» 5 Top Mistakes Management Makes With BA Performance Measurement Programs (And How to Avoid Them)
When used properly, a performance measurement program can help BAs and their managers identify specific improvement priorities, clarify responsibilities, and drive the desired behaviors required to achieve business goals. However, in practice it's rare to see managers taking advantage of the benefits a good performance measurement system can offer for organizations interested in achieving improvement objectives such as reducing the cycle length of the requirements process or minimizing the risk of IT projects not meeting the expectation of their stakeholders.
Below are the 5 top mistakes I have identified in companies implementing performance measurement programs for their business analysts, accompanied by tips to prevent these mistakes from hindering your performance improvement goals:
Failing to identify performance improvement objectives before selecting measures
In the children's book Alice in Wonderland, Alice had no idea where to go. When she asks the Cheshire cat which road to take, the magical cat offers a helpful reminder to pin down your destination first.
The objectives of a performance improvement program may be established by the BA team, or come from senior leadership, but they must be defined at the beginning of any performance measurement effort. An effective way of identifying appropriate performance improvement objectives is to listen to your business and technical stakeholders. Are there complaints about the time it is taking to complete the requirements for new projects? Do developers end up ignoring requirements documents and relying on conversations with stakeholders or on test plans to better understand software requirements? Such issues are excellent starting points for establishing good objectives for a performance improvement program for your business analysts.
It's very easy to create measures that are meaningless nonsense. Just because something can be measured, it doesn't mean it should be. Some managers insist in collecting every single piece of data just because it’s available, or using a standard set of metrics that doesn't reflect the organization's improvement priorities. An effective performance measurement program will include only the metrics that matter most for your particular performance objectives. Start from the concerns you have and the outcomes you plan to achieve in order to identify the few critical measures that will keep your measurement system focused and actionable.
Succumbing to the tyranny of conformance
Too much emphasis on adherence to documentation standards will have individuals working toward this objective at the expense of much more important goals. As a commenter pointed out in this article,
“If we have one set of measurements for everyone, then we will end up asking some teams to produce documentation just so we can measure it, and not because the documentation is needed for the project! I have personally experienced this (unfortunately it has been often in recent years), where I was REQUIRED to produce documentation that was useless for the project, but had to exist so that measurements could be made of my team’s effectiveness.”
Metrics about compliance with requirements documentation standards are rarely needed. Managers should be focusing instead on fostering true understanding of the necessary processes and policies by means of training, coaching and mentoring. When adherence to documentation is truly relevant for performance improvement, this objective should be measured against specific agreements, rather than against a generic set of standards that may not be applicable to all projects.
Measuring "long work" instead of "hard work"
Seth Godin recently wrote about how, for a long time in our history, long work was required to succeed:
“Long work is what the lawyer who bills 14 hours a day filling in forms does. Hard work is what the insightful litigator does when she synthesizes four disparate ideas and comes up with an argument that wins the case--in less than five minutes.”
Measuring long work (e.g., total number of requirements, number of documents produced, or hours worked) is much easier than measuring hard work – the work that avoids the consequences of failing to identify the correct business problem, or producing incomplete or ambiguous requirements—such as building a software product that is not fit for purpose. However, the effort required to identify meaningful measures that put a clear focus on priorities will be rewarded with valuable insight about performance problems and opportunities faced by the BA team, allowing data, not opinions, drive your improvement actions.
Using performance measurement for motivating or controlling behavior
We all know the consequences of using performance measures to punish or reward behavior: it tunnels employee vision, reduces the urge to experiment, causes sub-optimization, and leads to gaming, creative accounting, and fraud. Using performance measurement to control or motivate behavior will only prevent BAs from experimenting and suggesting new solutions for business problems. There is a much better use for an individual performance measurement system: to help business analysts and their managers scale successes, and in the case of failed performance, go from “we have a problem” to “there is a performance problem, and this part is caused by an inadequate process, this by a competence gap, and this by lack of management support for the BA work”.
Completely decoupling individual performance measurement from rewards will avoid problems such as unhealthy competition for personal gain and distrust of the performance measurement system. If individual measures aren't used as a direct basis for reward, the receptivity for learning from measurements, as well as the willingness to experiment and innovate, will be significant higher.
Forgetting to provide frequent feedback to individual BAs
Many managers save feedback to their annual performance reviews, but to be effective, feedback about individual performance must be frequent and consistent. BAs and their managers should be having frequent discussions about current performance levels and trends. In effective feedback processes, business analysts learn what they need to do differently to achieve the expected results, managers learn what type of guidance, support and oversight the BA needs to get there, and everybody develops a clear understanding of what must be done to solve problems and scale successes.
Should individual performance measurement be banned because of these common mistakes?
It's a beaten expression, but one that perfectly translates what has been happening in many organizations that gave up on measuring the performance of their business analysts: they are throwing out the baby with the bath water based on inexpressive (if not damaging) results obtained from flawed attempts at establishing a performance measurement system in the past.
In his book Agile Estimating and Planning, specialist in agile projects Mike Cohn recommends performance tracking to be performed only at the team level, arguing that measuring progress at the individual level leads to a variety of problems:
“For example, if finishing a task early results in a programmer’s being accused of giving a padded estimate for the task, the programmer will learn not to finish early. Rather than finish early, she won’t report a task as complete until its deadline."
There you have a perfect example of mistake #4 (“using performance measurement for motivating or controlling behavior”) causing a useful concept to be entirely dismissed only because it's being used in the wrong context. Monitoring performance in an individual basis is important for establishing accountability, providing feedback, and allowing actions to support the development of individuals and the organization. The combination of the individual, department, and process measures allows managers to achieve a much richer understanding of what is affecting performance, and initiate corrective or other actions to improve execution.
Instead of forgoing the benefits of individual performance measurement just to avoid the misuse of performance data, managers should be working on eliminating the causes of dysfunctional performance measurement initiatives by definining clear improvement objectives, identifying the metrics that matter most for these performance objetives, and ensuring that metrics are used for learning and improving, rather than for controlling or motivating behavior.
Author: Adriana Beal, MBA
Adriana received her B.S. in electronic engineering and MBA in strategic management of information systems from two of the most prestigious graduate schools in Brazil. She offers consulting assistance throughout the software development life cycle for organizations implementing large, complex software projects, and for the past 10 years, has been providing consulting services to a diverse client base, including several Fortune 100 companies. She recently published an ebook on Measuring the Performance of Business Analysts, with all proceeds going to a foundation helping underprivileged children in India. She is also the founder of http://projeto100.org/eng, a movement making significant changes in the lives of families living in poverty in Brazil.
If only the idea of documentation for documentation's sake would go out of style. Of course, that will happen when Waterfall goes the way of the dinosaurs and we start talking to each other instead.
cblapp, I certainly agree that documentation created for the sake of documentation (as described by the commenter I quoted in the article) must go away once and for all.
It's also important to remember, though, that 'talking to each other' cannot replace value-adding documentation. Even in Agile projects, there are many valid reasons to produce documents, some of which I discuss in this article: http://www.bridging-the-gap.com/the-positive-influence-bas-have-in-the-quality-of-documentation-in-agile-projects/
Thanks for leaving your comment!
Cblapp - it's not waterfall that creates documentation for documentation's but how projects are planned and managed. It's lazy to assume that a standard set of documentation is required in any project be it Waterfall or Agile. If the approach is considered and agreed during the planning phase including what documentation is required for each phase then it is far less lkely that redundant documentation is created.
I would like to know your opinion...
Would you agree that measuring performance with such mistakes is better than no measurment at all?
What if this would be the only way how good analysts could show the quality of their work in an environment where everything is left to the individual and a manager does not actually track their work or improvement?
@lada: "Would you agree that measuring performance with such mistakes is better than no measurment at all? "
That is a wonderful question! My answer is: no, I would not agree that measuring performance with these mistakes is better than no measurement at all.
Recently I was asked to informally help a BA group of a large company to improve their performance measurement system. After a few questions, I could see that they were willing to change the system to start providing frequent feedback (avoiding mistake #5) but were not prepared to let go of the other mistakes (specially using measures for reward/punishment). I told them that if that was the case, my recommendation was to stop measuring the performance of their BAs altogether, because the risk of their measurement system not adding any value, and having a negative impact on the team, was very high.
You ask, "What if this would be the only way how good analysts could show the quality of their work in an environment where everything is left to the individual and a manager does not actually track their work or improvement? " -- well, in that case, you should celebrate the opportunity to come up with better metrics (that measure "hard work" instead of "long work", focus on value-added instead of adherence to documentation, etc.). Start tracking measures that tell the real story -- reduction in the number of change requests due to better requirements, increased customer satisfaction, etc. -- and use the data to both monitor the progress and show the positive impact of the analysts' work.
If you focus on measuring how many documents or requirements were produced, or how many hours were dedicated to analytical work, then you won't be learning much about opportunities for improvement, nor providing evidence of the quality of your work. Good luck!
Rocognise this, those un-efficient way of measuring just to have it measured because it is possible. Where my experiences differs in many projects is that measuring not really is used for evaluation of the BA. In my experiences (in The Netherlands) BA's that NEED to be evaluated, are mostly "swimming" in the organisation and the mentioned documents to be written are just written because of any conformity or standards that are followed deeply (let's say, any document specified by Prince II or LAD NEEDS to be written).
What i miss in my experiences is real goal-setting. And, when it is set, it is forced to change tomorrow. It takes much time and energy to convince clients to be consistent on the goals and just to document what needs to be documente. The last thing can help reaching goals earlier. In 1969 man flew to the moon, calculating with two digits behind the comma, on slow computers an much paperwork. Prince II did not exist, and with good comon sense a giant result was set. That is not impossible anymore but please, do not think interesting books with methods and methodoligies are bibles. Use your brain!
Thanks for leaving your testimony -- I'm sure many BAs and BA managers will identify with what you wrote. And yes, as with anything else, there are no silver bullets, and it makes no sense to copy methods (or metrics) from other organizations or projects just because they worked once--most likely under different circumstance).
To quote Alistair Cockburn,
" If you follow a methodology out of the box, you will have one that fits some project in the world, but probably not yours."
Only registered users may post comments.