Foreword
Business analysts’ performance and how we could measure it appeared to be a hot topic of interest for the Bulgarian Business Analysis community.
“If you can’t measure it, you can’t manage it” (Peter Drucker) – this statement hides lots of questions for the Business Analysis managers, like what to measure, how to assess, quantifiable or qualifiable metrics to use? These questions were all raised at the IIBA Bulgaria Christmas party 2015
It was one of the best chill out brainstorming session the IIBA Bulgarian Chapter has ever had. Red wine, couple of Christmas cookies and several discussions on different BA topics. So pleasant atmosphere at great inspiring place. Somewhere among all conversations held, one outstanding issue arose – how do we measure the business analysts? That simple question though does not have a simple answer. And somebody proposed – why don’t we organize a workshop to discuss that question? Why not?! This is how the idea for ‘KPIs for Business Analysts’ workshop was born.
BABOK Guide® KPI definition
BABOK® Guide introduces a KPI definition relevant to performance measurement of solution and solution components.
"An indicator displays the result of analyzing one or more specific measures for addressing a concern about a need, value, output, activity, or input in a table or graphical form. Each concern requires at least one indicator to measure it properly, but some may require several"
In this perspective, a good indicator has six characteristics:
- Clear
- Relevant
- Economical
- Adequate
- Quantifiable
- Trustworthy and credible
BABOK® Guide definition outlines all these characteristics to be valid for a good KPI from product/solution perspective rather than evaluating the process and practices to undertaken to achieve this output.
The Workshop day
Scheduled for Saturday afternoon, the workshop ‘KPIs for Business Analysts
’ gathered 25 enthusiasts, currently acting as Business Analysts, Business Analysts Managers and Project Managers. Warm greetings, small talks and shortly the workshop started…
Agenda
14:00 – 14:10 Opening speech
14:10 – 14:15 Setting ground rules
14:15 – 14:20 KPI definition
14:20 – 14:40 Participants introduction
14:40 – 15:00 What is a good KPI for business analysts?
15:00 – 15:10 Coffee break
15:10 – 16:00 Discussion panels (3 separate work groups)
16:00 – 16:20 Coffee break
16:20 – 17:20 Share discussions output
17:20 – 18:00 Summary and conclusion
All participants shared different motivation to join. ‘To learn something new’, ‘to share my experience’, ‘to learn from other’s experience’, ‘to meet fellows’. And all these expectations were covered!
The Workshop output
What is a good KPI for business analysts?
As a result of several discussions held the workshop participants have agreed on a list of KPIs characteristics which amplifies the
BABOK® Guide definition...
- Agreed - all stakeholders should agree on the business analysts KPIs. This might be similar to the BABOK's KPI characteristic for "Communicated", but not exactly. Here "Аgreed" KPI means "Communicated" KPI which has got feedback and reached a consensus on its definition
- Calibrated - scaled with expectations, scope and goals
- Proper to the business environment - a KPI should be applicable to the relevant context of usage - waterfall vs. agile process, BA position in the organization - team assignment (IT/Business team), internal/external BA, level in the organization (junior, regular, senior), level of business analysis (system analysis, business systems analysis, enterprise analysis and etc.)
- Relevant to the assessment purpose - a KPI target purpose should be clearly stated - to predict (to set leading indicators) or to track (to set lagging indicators) the business analysts work. Leading indicators are those that predict the future. Lagging indicators are those which reflect facts that developed in the past.
- Weighted – KPIs should have its own weigh and priority may not weigh equal
How often the KPIs should be consumed and refined?
Two main cycles of consuming and refining KPIs have been identified:
- For in-house business analysts
- 6 months for middle management
- 12 months for agents in operations
- For outsourced business analysts
- each deliverable should end up with retrospection of the achieved
KPIs for Business analysts’ definition
It has been observed that KPIs are commonly defined with negative perspective like the following examples:
- Could we consider escalations as s KPI?
- How many times a BA task was rejected and returned for rework?
Despite, some examples for positive measures can be:
- How happy are developers and business people from the BA work?
- How often stakeholders thanks his Business analyst?
Benefits from KPIs for Business analysts’
KPIs for business analysts are most often used to:
- assess performance of the role
- blame somebody
List of KPIs for Business analysts’
Finally, the target of the workshop has been achieved and the following list of KPIs have been defined and put in a proper category:
| Category |
KPI
|
| Communication and Planning |
- Percentage deviation from BA estimates
|
Requirements Management
|
- Number of affected processes that were not changed (Any existing process that should have been enhanced as part of the change that BA worked on)
- Number of organizational standards not followed
- Average number iterations (compared to predefined standards)
- Has the BA managed to prioritize his requirements? - Yes/No
- Requirements prioritization
- Percentage of requirements prioritized
|
BA Skills and Knowledge
|
- Number of BA tools & techniques used
- How successfully are the tools & techniques used?
- Does the BA suggest solutions? - Yes/No
- Percentage of accepted suggestions for improvement that the BA raised
- Does the BA learn and improve his skills?
- Number learning hours, preferably related with results from learning
- Number training (knowledge transfer) hours, preferably related with results from training
- Number self-improvement hours, preferably related with results from training
- Does the BA applies lessons learned? (or repeats same mistakes)
- Number of reusable requirements logged in
- Adaptivity with the organizational culture
- KPIs for BA organizational skills:
- Number of meetings to number of conclusions and agreements
- Number of reviews and comments to number of stakeholders
- Percentage of rescheduled meetings
- Percentage of deadlines met by stakeholders
|
Stakeholder Satisfaction
|
- Subjective opinion on BA skill of stakeholders (Stakeholders feedback)
- Stakeholder satisfaction
- Were the requirements clear? (are the requirements clearly documented and visualized)
- How would you evaluate the BA’s communication skills?
- How would you evaluate the subject matter expertise?
- How would you evaluate the completeness of the analysis? (did the BA managed to involve all stakeholders)
- How would you evaluate the proactivity of the BA?
- (As a BA) Was my opinion taken into consideration?
- How would you evaluate the engagement level of the BA?
- Open field for recommendations - number of unique recommendations
- Was the BA prepared for each meeting/interview he schedules
- (Percentage) of the implemented features that are actually used after implementation
- Number of satisfied business needs
- How long did it take for the solution to be adopted?
- Number of issues escalated for consultation (this would measure the BA reputation in stakeholders)
|
Change Management
|
- Number of change requests (that are not initiated because of a Business change) within project scope:
- on project milestones
- after deployment
(This KPI would measure missed requirements, stakeholders, systems that have impact from the change)
- Number of defects on requirements bugs)
|
Effectiveness
|
- Contribution - Value of benefits the BA brought in with his changes. Any innovations implemented by BAs? (in terms of process and organization of work)
- Does the BA manage to stick to budget and needs? (avoid Gold Plating)
- Velocity of execution - KPI that tracks velocity of a BA team
|
Author: Mariana Mitevska, President, IIBA Sofia Chapter, Bulgaria