Training development projects historically have tended to apply the following scalar or product breakdown structure:

  1. Curriculum/Learning Path

  2. Course/Learning Experience

  3. Lesson/Learning Experience

  4. Learning Objective/Performance goal

  5. Teaching Point/Key Learning Point

Because we spend much time persuading professionals in the training domain, rather than decision makers with a primary proficiency in the software domain, we made a design choice with our approach to Lean-Agile to blend as much from their familiar training domain as possible.

Software is more varied and abstract and the product breakdown structure varies more often depending on the intended functionality. So the software Agile literature proposes many ways to decompose a project into similarly sized chunks.

An Aside on Estimating Relative Sizes

Often, project leadership receives flawed estimates when they ask a development team to estimate in hours. To avoid that outcome, many Agile groups use relative estimation to estimate their work effort. This method is very different from estimating the exact or absolute effort using hours as the unit of measure.

Let’s be clear, nothing in Agile requires using relative estimation techniques. However, relative estimation has become the most popular method in Agile. We can use any desired estimation method with Agile. With that caveat out of the way, here is why many Agile practitioners use relative estimates to predict their own work effort.

Why
  • Estimation is hard.

  • We need estimates to set goals for scope, schedule and budget to track progress towards the desired results.

  • We need estimates to make trade-off decisions.

  • We want fast estimates. Estimation is time consuming, with many absolute estimates involving substantial rework during execution. Lean calls this waste. Relative estimates involve less waste.

  • We need an effective technique for estimating emergent requirements. Scope often changes.

  • Complex projects have many moving pieces with hard-to-quantify dependencies.

  • Humans are better attuned to pattern recognition, comparing two things in relation to each other. People tend to more accurately estimate relative sizes.

  • People tend not to be good at correctly estimating with absolute values, like hours.

  • Optimists tend to underestimate. Pessimists tend to overestimate. [1]

  • "Comparing is much quicker and more accurate than deconstructing." [1] By deconstructing, we mean to break down a work item into constituent tasks.

  • People agree faster on relative estimates like "that is twice the size of the other one". If required to use absolute values, they may spend more time arguing over whether it was 15.75 hours, 16.25 hours or 17.5 hours.

  • In golf, at the tee box, we don’t estimate 210 yards and 7 inches. At that early point in the process we only estimate in yards. Only later at the putting green do we estimate in feet or inches.

  • At the beginning of the project, we are estimating with the least information we will ever have in the project. Estimates for six or more months out tend not to be accurate.

  • No matter how much effort we put into an estimate, an estimate is still an estimate.

So rather than absolute values, Agile practitioners in the software discipline have largely adopted relative values instead, using various methods. Because this is so different from traditional hourly estimates, let’s discuss how this is often done.

How
  • The most popular relative size value sets in software development is called story points, where we size work items, or user stories, using a variant of Fibonacci numbers as the unit of measure.

  • Another popular relative sizing value set in software development is T-shirt sizes, such as XX Large, X Large, Large, Medium, Small.

  • Relative estimation is performed at the product backlog level rather than at the iteration backlog level.

  • Iteration-level estimation can be estimated in hours like traditional projects because a single iteration is close in time, typically days not months.

  • Multiple experts estimate at the same time (The Delphi Technique), like judges at Olympic Ice Skating events.

  • Estimate outliers are verbally justified by the expert making the assertion.

  • Averaging individual expert estimates lead to better estimates.

  • Planning Poker®.

If we will not use hours as the unit of measure, then we need a substitute. The Agile community in the software discipline uses size "points" to size the effort for each work item. Absolute estimates use hours, numbered using a scale of 1–100, with a granularity of one. Relative estimates use a points scale that is split up into a fixed set of 10–12 "buckets" when using modified Fibonacci. Relative estimation intentionally uses a rougher granularity with almost ten times fewer choices for the estimators.

The Fibonacci sequence provides a pattern for sizing things, whether effort or physical objects in nature. The real Fibonacci sequence is named after Italian mathematician Fibonacci. Applications of the Fibonacci sequence appear in nature, such as the relative size between the whorls in a nautilus shell, plants, storms, and galaxies. Fibonacci scaling is fascinating in nature, but beyond the scope of this book. See wikipedia for more information (https://en.wikipedia.org/wiki/Fibonacci_number).

FibonacciInNature
Figure 1. Fibonacci in Nature [2]

Note that actual Fibonacci numbers are 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144 and so on.

Many in the software development community use a pseudo Fibonacci sequence to relatively estimate a work effort. Note how easily we can distinguish between a 3-point sized effort and a 20-point sized effort.

So to facilitate group estimation, Mountain Goat Software invented a game-like estimating activity and called it Planning Poker®.[3] With Planning Poker®, each team member estimates a particular work item relative to a "normal" work item using a pseudo Fibonacci number. Like in the game rock, paper, scissors, all players should reveal their choice at the same time to temper peer influence. To help team members each reveal their estimate at the same time, Mountain Goat Software developed a card deck for Planning Poker®.

Modified or pseudo Fibonacci numbers use a similar size progression on playing cards that all team members can use to relatively size the effort from the work item up. Mountain Goat Software has popularized Planning Poker® with their cards which include: 0, 1/2, 1, 2, 3, 5, 8, 13, 20, 40, 100, and infinity.

PlanningPokerCards
Figure 2. Planning Poker® Cards by Mountain Goat Software [4]
Why Modified Fibonacci?
Many teams do not cling to true Fibonacci because in the upper regions "34", "55", and "89" imply a certainty that simply doesn’t exist [in estimating]. Stating "40" instead of 34 or "100" instead of 55 transports a coarse-grained granularity that reflects the uncertainty in these regions. Simply speaking, the team tells their Product Owner by showing "40" that "this item is too big and has to be refined" while "100" basically means "this will never fit into a Sprint [an iteration], more refinement is needed".[5]
— Dominik Maximini

The key point here is that the team’s relative estimates are bottom-up from each team member, and they are relative to a normal work item of, let’s say 8-points. This point estimation may be confusing on first exposure. Because the team doing the work will estimate the points, their estimate is tailored to their team.

For more visually inclined readers, the following figure uses colored blocks to see the size in comparison to the other size buckets. Size 8 is in orange because it is the comparison size for relative sizing against all the other sizes shown in blue blocks.

VisualSizePointsScale
Figure 3. Estimating Relative Size with Size Points

The smaller size points indicate lower uncertainty/risk and higher reliability/precision. Smaller size points also tend to indicate more confidence by the team. While the higher numbers indicate higher uncertainty/risk and lower reliability/precision. Higher size points also tend to indicate less confidence by the team.

Back to Training Scalars

When coding is significant in a courseware product development project, for example in custom simulation development, story point estimation methods from the software development domain are used too.

However, if the courseware does not include significant custom code, then we have found that many training professionals prefer to use a scalar or product breakdown structure they are already familiar with, so we have successfully used:

  1. Course

  2. Lesson

  3. Learning Objective (LO)

We have not felt the need to go as granular as teaching points; although, that is possible. For Agile, we have also found is harder to apply to analysis projects, the A in ADDIE, that deal with curriculum design because we typically staff those with a single instructional/LX designer, making many of the Agile concepts more overhead than helpful with a one-person team. We have had success applying LOs as the good enough or effective granularity or small-batch size for purposes of persuading, negotiating, and implementing learning experience development projects. LOs also vary in size, but typically they are more similar to other LOs than lessons are similar in size to each other. Using LOs allows us to meet the Lean principle for even, continuous flow through the system by using similarly sized work items.

Estimating in some organizations is based on historical data of similar projects. Learning experience development that is estimated by traditional student seat hours may be difficult to switch all at once to another method like T-shirt sizes or Story Points. Training groups may not be used to estimating each work item card from the bottom-up. Initially, you may need to continue estimating by student seat hours and then gather work item card data to provide a basis for future estimates for future projects.

Or you may be okay trying to use Planning Poker® cards using story points.


1. Reproduced with permission from Ilan Goldstein, Axis Agile, Copyright 2013. http://www.amazon.com/Scrum-Shortcuts-without-Cutting-Corners/dp/0321822366
2. Reproduced with permission from Tim Nolan, Agile Government, Collin County Texas, http://www.slideshare.net/TimNolan3/agile-government
3. PLANNING POKER ® is a registered trademark of Mountain Goat Software, LLC
4. Reproduced with permission from Mountain Goat Software.
5. Reproduced with permission from Dominik Maximini, Estimating relative sizes, http://scrumorakel.de/blog/index.php?/archives/48-Estimating-relative-sizes-e.g.-story-points.html
Image

Line By Line

Here a Little, There a Little, Layer by Layer.

Back to Overview