The Modern Analyst Blog for Business Analysts

Jarett Hailes
Jarett Hailes

Mitigating the Risk of Story Point Drift

In many Agile projects requirements are not typically written in the form of a formal requirements document. Instead, a collection of concise but effective means of describing what must be built called user stories are often used. User stories describe the behaviour, performance, or interface of a system from a customer’s perspective. A typical user story might look something like this:
As a potential customer I want to be able to view books based on the search criteria I entered.
User stories are not only effective requirements management artifacts, they are also essential to estimate the scope/size of the project and to track the progress of the team.  When determining the size of the project, teams estimate the level of effort required to complete each user story and then aggregate their results to come up with their estimate for the scope of the project (for more details on how to estimate level of effort in Agile projects, see Mike Cohn’s excellent book on the subject).
 
Agile teams will often use a unit-free type of measure called story points. The advantage of using story points is that their inherent value is relative. Instead of trying to come up with absolute values typically related to time (for instance, how many days will it take to complete feature X), story points are only concerned about the relative effort or complexity of a story compared to the other stories that have been or need to be completed. When the total number of story points for the project is combined with a team’s velocity (the number of story points completed per iteration period) project stakeholders are able to get an increasingly accurate picture of the size of the project and the time it will take to complete given the current team.
 
Initially when the team forms they will estimate story points for some number of baseline stories. For instance, the team may have the following 3 stories as their baseline:
  • As a user I want to browse the book collection by subject: 8 points
  • As a user I want to be able to save my payment information: 2 points
  • As a user I want to be able to recommend my favorite books to people: 5 points
 
From this baseline other user stories are estimated in terms of their perceived relative amount of effort to accomplish. For larger projects, most user stories are estimated as epics, larger user stories that will be broken down later on in order to actually tackle development.
 
Over time, new user stories will be added to the product backlog, others removed and some will be altered to reflect changing requirements. All the stories that are added or changed to the backlog will require story point estimation. 
 
One of the potential risks that exists when using story points is what I call ‘story point drift’.   Story point drift is when user stories that had a given story point value at the beginning of the project require substantially more or less effort than stories with the same story point value later on in the project for the given amount of effort required to complete a story.
 
For instance, let’s say I have the following user stories estimated later in the project:
  • As a user I want to be able to have my bank account linked to my login and set up a monthly withdrawal plan: 8 points
  • As a user I want to have a theme editor that can customize the look and feel of my online affiliate store: 5 points
While the above user stories may accurately represent their relative size compared to each other, in comparison to the stories that were estimated at the onset of the project it appears as though the latter group’s story points under-represent their complexity and level of effort required. I would suspect that the amount of work to enable automatic bank transactions is substantially larger than the amount of effort required to build a browse by subject feature.
 
I find the risk of story point drift increases on larger or longer-term Agile projects. Story point drift can occur for several reasons:
  • The team’s collective memory is short term: when a team begins estimating new stories, they typically draw upon their experiences from recently completed stories. If some of those stories were misclassified (either required materially more or less relative effort than was represented in the story point estimate), then the team can end up believing that these recent stories are the new norm for the value of the story points and use these as references going forward, which skew the story point values from the original baseline references.
  • The team’s complement has changed: it is not uncommon for project teams to change over time. I’ve noticed that even as teams get larger or smaller that a team’s velocity can appear to remain constant. When investigating this, I’ve found that it’s often because the team begins to estimate stories with fewer points, since they now feel that the story is ‘easier’ since there are more people to work on the project. As a result stories that might have been considered 8 points previously are now being estimated as 5 points. As a result, despite the fact that the team is likely getting more done given their drift in estimates the team’s velocity appears unchanged.
  • Baselines are not referred to: Agile projects usually thrive by the minimization of overhead that does not lead to customer value and by adapting to non-ideal situations. For example, Agile teams rarely wait for everyone to show up for a meeting – the meeting is time-boxed and starts and ends on time regardless of who is missing. However, at times Agile teams may forget to bring physical objects like their reference stories to meetings. Sometimes teams will try and get through the meeting based on recollection rather than actual reference. Without the actual reference stories and point values in hand, the resulting estimates can be somewhat skewed.
 
When I’ve seen story point drift it has occurred in small increments over time – you don’t realize that there’s been a dramatic deviation in the estimates until sometimes months later. Story point drift can cause issues with resource planning and progress and time to completion estimation.   Here are a couple of strategies I’ve used to help mitigate story point drift:
  • Bring mixture of original and recent stories as the baseline for new estimation: it never hurts to hold up your original estimates when doing relative comparisons. Having some recent examples are also helpful, in particular since your initial estimates were likely only for a few potential story point values. Having 1 or 2 stories for each possible story point value can also be helpful when discussing what level of estimate to assign to a new story, or when individuals are deadlocked in a session of planning poker.
  • Know when certain stories should be re-estimated after implementation: once in awhile you will encounter a story that was a lot more (or less) effort than you initially thought. If the level of effort was sufficiently different from another story that was given the same amount of points, you may want to re-estimate the story so that it does not affect your team’s perception of the value of a certain number of story points. I typically would only re-estimate a story once a project has been going for several Sprints – early on you may discover that many of your stories take less or more effort than you thought and you’ll be tempted to resize those stories. However, as long as the stories are taking about the same amount of relative effort, then there’s no need to re-estimate. So if you thought your 3 point stories would take a day to complete but they’re taking the better part of a week, check to see how your 5 and 8 point stories are taking. If those are also taking a fair bit longer than anticipated, then don’t re-estimate.
  • Spend a little time every few Sprints and analyze relative stories: the ScrumMaster or Project Manager can review some of the completed stories over time to look for possible story point drift. If some drift is detected, bring it up with the team and see what they think. If the team agrees, then re-estimate the stories that appear to be out of what in terms of relative effort.
User stories and story points can be a very good way to manage the requirements and estimation for a project. Keeping an eye on story point drift will ensure that the team has a good handle on the project’s rate of progress and estimated time to completion. With diligence this method of project estimation can be a very accurate and effective tool for Agile teams.
 
-------
Jarett Hailes
Larimar Consulting Inc.
http://www.larimarconsulting.com
This entry was published on Jan 17, 2010 / Jarett Hailes. Posted in Project Management, Business Analysis, Estimation, Agile Methods. Bookmark the Permalink or E-mail it to a friend.
Like this article:
  3 members liked this article

Related Articles

COMMENTS

Divya posted on Thursday, January 28, 2010 6:08 PM
The retrospectives conducted after each sprint tend to give an idea about the drift if there was any. Taking into consideration the experience gained out of each scrum, it may be possible to re-estimate the points hence forth. I have worked in an Agile environment and the drift was something we encountered a couple of times. However, with more experience of working using Agile and constant re-valuation, we were able to control the drift.
Great article. Thanks.
Divya
Jarett Hailes posted on Thursday, January 28, 2010 11:36 PM
Thanks for your comment Divya. I agree that the retrospective is a good opportunity to review whether stories within the lines of what was expected at the onset. There are some risks of re-estimating story points too often as well. Mike Cohn's Agile Estimation and Planning book has some good guidelines on when to re-estimate story points.
Jarett Hailes
Marc Thibault posted on Monday, February 15, 2010 7:39 PM
In the case of interesting disparities, how do you know what to allocate to story points and what to allocate to velocity?

Late in the project you can use project-wide points over time to-date as a broad norm but I would worry about applying it to a specific couple of people working on a specific story. The bigger and more dispersed the project, the greater the variability in velocity you're going to get and the more unreliable it becomes.

When programmer productivity can vary by an order of magnitude, not just between programmers but also by the same programmer in different situations, this would seem to be an interesting problem.

Marc Thibault
Only registered users may post comments.

 



Blog Guidelines

» What is the Modern Analyst Blog and what are the Benefits of Contributing?

» Review our Blog Post Guidelines.

» I am looking for the original Modern Analyst blog posts.




 

ModernAnalyst.com LinkedIn Group ModernAnalyst.com on LinkedIn: connect with fellow business analysts in order to develop and expand your professional network.
Learn More...

Copyright 2006-2024 by Modern Analyst Media LLC