They say the best criteria of Business Analyst’s success are a happy customer (business) and a happy engineering team. But what does it mean? How can we break down happiness and measure it? These are precisely the questions my 8 BAs team and I tried to answer this year. The result was a surprisingly efficient pair of surveys we developed and have been using through 2020. This article will explore the core principles behind them, share their content and some options as to how the results can become actionable.
Moving Toward Quantitative Markers
The usual way to gather feedback from customers and the team is to ask them once in a while, plain and simple. In our case this would happen at the end of an assignment/project, or whenever an employee was due for a level up. There is a significant problem with this approach. I’m sure many business analysts have encountered it at some point. You ask for feedback from a developer or a product manager, and their reply is ‘everything is excellent, the communication skills are great!’. And that’s it!
Too often we are faced with a lack of understanding of the BA role. Most of our efforts are invisible and taken for granted, as a result business analysts are perceived as just ‘people who talk a lot’. This means an effective request for feedback is also an educational endeavor. The request itself should contain information about what to expect from a BA. This is why we decided to go with closed-ended questions each of which describes a key activity of a business analyst and offers to rate it.
This approach would allow for a quantifiable result, a sort of rating for each BA that could be seen dynamically through time. In our case constant rotation of BAs and POs between different streams and teams ensures even distribution of ratings - each PO has a chance to work with several different BAs over time which allows for comparisons and evens out differences in subjective ‘scales’.
Small Survey - Big Turnout
Now that we knew we wanted to design a survey, we needed to make it as efficient as possible. When it comes to surveys it is best to go small. As small as possible. So people could be lured in with a promise of a 3-4 minute effort (and this promise would be kept). And so the team actually goes through with filling it out. I can’t even count how many times I quit an online study when it tried to feed too many questions.
Our goal was to fit everything in one page, no scrolling. When we added the formal questions (which BA is being evaluated), the title and description, deducted space for the ‘submit’ button - we got room for 3-4 questions. Sounds impossible to wrap BA work into only 4 questions, does it? Well, give us a chance.
Choosing the Language
It’s not enough to just ensure as many answers as possible. The quality of answers matters too. I prefer a personalized approach to crafting questions - when people read the proposed answers in first person. The trick to it is finding major points of contact between stakeholders and business analyst’s deliverables, and wrapping them up into an informal statement. For example:“I see/read stories for the first time when I start working on them”. This is something people can relate to, internalize it and give an effective answer.
We separated out a survey for our teammates and another one for the product people. This way we could avoid being generic and place narrow targets on key expectations. Our team is Russian speaking, so naturally their survey was in Russian despite the company culture to have all correspondence in English.
Our Surveys
Having gone through key decisions that went into designing our version of BA feedback surveys, we can dig into the content.
Survey for the engineering team taps mostly into the topic of enabling delivery with INVEST user stories and efficient clarifications:
1. How well does the BA explain the business value of changes?
where 10 rate ~ ‘I always get the benefits of features for end users’, and 1 rate ~ ‘No idea why or for whom I we deliver stuff’
2. How productive are grooming sessions?
where 10 rate ~ ‘Stories are empty, I have many unresolved questions afterwards’, and 1 rate ~ ‘I can get all the details of stories and estimate them’
3. How clear is the sprint scope at the beginning of it?
where 10 rate ~ ‘I fully understand the sprint goal and what needs to be done from my side’, and 1 rate ~ ‘I see/read stories for the first time when I start working on them’
4. How well are additional questions resolved during dev/testing?
where 10 rate ~ ‘Open questions overwhelm me, there is no help from BA’, and 1 rate ~ ‘BA quickly resolves my questions and unblocks my progress’
Survey for the product team explores backlog preparation and requirements quality:
1. How do we divide responsibility for writing requirements with the BA?
where 10 rate ~ ‘The BA produces all of the requirements, I only review them’, and 1 rate ~ ‘I produce all the requirements, write stories and specs myself’
2. How complete are the requirements BA provides?
where 10 rate ~ ‘10 - requirements are very detailed from the start, they consider all dependencies and impact areas’, and 1 rate ~ ‘In majority of cases most of the scope is discovered after the work has started, work is delayed due to lack of predicted requirements’
3. How ready are the sprints at the point of planning?
where 10 rate ~ ‘We always have ready stories for 1+ sprints ahead‘, and 1 rate ~ ‘Backlog items for the sprint are empty/not ready when it comes to planning the next sprint’
4.How hard is it to review the requirements?
where 10 rate ~ ‘Everything is compact and clear, review takes seconds’. and 1 rate ~ ‘I struggle to read and understand what I'm asked to review, requirements are scattered and inconsistently formatted’
As you can see, the questions and answers are calibrated to fit our processes and deliverables. So this content may not be copy/paste-able for all cases, but hopefully it shows the idea behind this way of gathering feedback.
Timing is Everything
The surveys are sent out on a regular basis - every quarter. It establishes a habit to give feedback, helps not to put issues on hold for a long time, and provides a regular reminder of what is to be expected from a BA. New BAs get compared to old ones, core BAs get a long living evaluation that changes over time.
Another side to asking for feedback often and with consistency is an opportunity for appreciation. The surveys have comment sections which served as boxes for suggestions as well as places to put positive feedback and cheers for the hard work a BA is doing.
What’s Next?
When the survey results are in, I found it best if each BA gets acquainted with their own results. Individual feedback can be discussed with the lead/manager in 1-2-1 format to dig into the reasons for some extreme ratings - good or bad.
General rates for the whole team could be used as one of the targets for BA performance (eg overall ratings no lower than 9). The graphs can be reviewed on the BA catch up. There were times in our team’s experience with this approach when we saw some major unhappiness of the whole team with some of the aspects of our work. For example, this way we found out most of our teammates didn’t understand the sprint scope and value of sprint as a tool. When we discovered this, we gathered for a few root cause analysis sessions, held additional interviews and surveys. After this we came up with several solution options and presented them to our leadership to get a go ahead on implementation. The effectiveness of our measures was seen next quarter.
And that concludes our short tale of one BA team working out an approach to measuring stakeholder happiness. Make it short, simple, relatable, educational and regular. Hope our experience will help our fellow Business Analysts move toward better feedback and happier teams.