Agile Development & Testing

When adopting an agile approach to producing software, it is essential that folks from different functions come together to work as one team. Of prime importance is the coming together of the builders and breakers. Both developers and testers need to work closely together as one team to produce quality software. Development and testing go hand-in-hand rather than testing playing catch up with Development. This implies an incremental approach to testing work-in-progress artifacts as opposed to testing completed pieces. Build something-test it quickly-provide rapid feedback-incorporate feedback-build more-test some more ….

In such an approach too, it is easy to retain the functional distinctions in terms of assuming that testers do the testing and developers just code. To achieve greater quality, the emphasis must be on building in quality and prevention rather than just detection. Development is equally if not more responsible to build in quality and adopting quality practices that minimizes defects. Testers would partner with Development to quickly understand, test and provide feedback.

For Development to prioritize building in of quality, there needs to be shifts in the way customer reported issues are dealt with. When a customer reports an issue which wasn't found by internal testing, does the "blame" rest on the tester who did not find the issue or would you rather question the developer who built that piece of code with the defect? It isn't about blaming one or the other. Instead, the responsibility must lay with both. The developer whose code has the defect is as much responsible for the issue as the tester whose tests did not catch the issue. 

Also, there needs to be an organization wide emphasis on testing. When Development produces an artifact, there needs to be demonstrated evidence of tests having been run with "satisfactory" results. In fact, reporting of test results for any artifact being delivered by Development needs to be a requirement rather than something that is desirable or good to have. Effort/project estimates for producing software must include testing and time required to implement quality practices.

Ideal time in Agile estimation

In an earlier post we looked at agile estimation using story points. Let us now look at another unit used in agile estimation, the ideal time.

What does ideal time mean and how is it different from the standard calendar time used to come up with schedules and determine project completion?

The standard calendar time is also referred to as elapsed time. In this post, let us refer to this as calendar time since that is a term that is more widely used. Ideal time is what the term implies - the ideal time it would take to complete an activity if there were no overheads. To further clarify, overheads refer to the time spent on unplanned activities such as checking emails, phone calls, attending meetings, switching between tasks, interruptions of any kind, etc. which are very much existent in any team member's day. For example, in an eight hour working day, the ideal time available to work on a planned task may actually be between four to five hours after accounting for overheads.

When seeking time based estimates from team members it is important to be clear on the unit used. Are your team members estimating in ideal time or calendar time. When a team member says it will take X days to complete an activity or story, does it mean X days exclusively working on nothing but this particular activity? Managers who mistakenly think that any time based estimates refer to calendar days may map the estimate to a calendar and have the wrong expectation.

When estimating in ideal time the assumption is you will solely and fully work on the particular task for the estimated duration without any interruption plus everything you need (such as tools, people, dependencies, etc.) are readily available for your use when you need it. Ideal time estimates are agnostic to the level of overheads in a particular environment and may be used to derive duration similar to how Story points are used.

Story points & Agile estimation

An important element in agile estimation is the use of story points. Here's an overview of story points and their usage.

So, what are story points?

A story point is an abstract and relative measure of size of an user story. Estimating size in story points should take into consideration all of the activities (such as design, development, testing, etc.) required to complete the user story, plus the complexity involved, any uncertainties and risks.

What scale is used to assign these values? And, what do the story point values mean?

First, the story point scale is chosen by the agile/scrum team. It may be t-shirt sizes (small, medium, large, extra large, ...) or Fibonacci series (tends to have a good spread) or a series of numbers. The requirement is for the agile team to agree upon and have a common understanding of the scale. The values by themselves are not important. The relation of one value to another is of significance. For example, if we have a scale with values such as 1, 2, 3 and so on the expectation is that if a story is estimated to have a story point value of 2 it implies that this story is 2 times the size of a story that has a story point value of 1.

When we talk of estimating "size", the general confusion is to associate "size" with "duration" or "effort". An user story can have varying effort estimates based on the individual implementing it. For example, an experienced person may take 1 hour to do a particular job whereas a less experienced or fresh person may take 5 hours for the same job. However, despite the varying effort estimates, the size of the job does not change. Story points are a relative unit of measuring this size. The confusion around the three terms - size, duration and effort lead to these being used interchangeably when they actually are distinct entities.

Duration is a derived metric while we estimate size. Before getting into the how of deriving duration, let us look at another term - Velocity. Velocity refers to the number of completed story points per iteration or the sum of the story point estimates for delivered/completed stories in each iteration. For example if an agile team completed five stories in a particular iteration wherein each story was estimated at 4 story points, the total story points completed in the iteration would be 5 x 4 = 20 story points which is the velocity of the team. Since velocity can fluctuate for various reasons, it is advisable to consider velocity across multiple iterations to obtain a stable measure. To derive duration, sum up the story point estimates for all planned stories in a project and divide this by velocity. This number represents the number of iterations required to complete the planned set of stories. Given the number of iterations required to deliver the planned set of stories and the duration of each iteration, we can easily find out the schedule for the project.

Agile Planning

Does the term agile planning seem to be an oxymoron? Do agile teams really spend time planning? Well, actually they do. And the fact may be that agile teams actually spend more time planning than non-agile teams. OK, if agile teams indeed spend so much time planning, why isn't it evident like the planning done by traditional/non-agile teams? There’s a difference. And the difference is in the “when” of planning. Traditional teams focus on doing almost all of their planning upfront. Planning, or rather complete planning often precedes the “doing” in a traditional model. And by “doing” I mean the actual activities resulting in producing deliverables required by your customer. Contrast this with agile teams, wherein planning occurs throughout the project. So, how does this work?

At a fundamental level, agile planning acknowledges the existence of unknowns.

The upfront planning in agile is done based on what is known while acknowledging the existence of unknowns. With this limited upfront plan, agile teams jump right in to the “doing”. This approach enables agile teams to gather new information as they produce which in turn contributes to reducing the unknowns. New information gathered is used to revise plans. Agile planning is a flexible exercise that embraces adaptation of plans through the project as newer information is obtained and unknowns are progressively reduced.

Are unknowns important? Wouldn't an exhaustive upfront planning exercise clarify any and all unknowns? When we do not acknowledge unknowns we assume that the requirements available at the start of the project are complete and accurate, we (and our customers) know exactly what is needed and customers will not want any changes to defined requirements or add any other requirements. Even if we were to assume that the requirements are truly final, do we really know everything required at the start of a project to prepare a plan that need not/should not revised? Agile planning strives to iteratively and progressively reduce unknowns.

Agile planning is an iterative process of planning, followed by execution, (re)planning again while adapting to any new knowledge gained, then followed again by execution, and so on. In each iteration a working artifact is generally available to "show" customers or product owners (the ones who provided the requirements). Iterative planning enables clarifying any incorrect assumptions or estimates made earlier. 

Another aspect of agile planning is the focus on delivering features rather than just completing a checklist of activities. This enables bringing together multiple functions to collaborate in pursuit of delivering a feature or set of features in each iteration. An impediment in any area tends to impact everyone else and hence there is the joint effort in speedy resolution.

Agile teams plan at multiple levels depicted in the planning onion shown here.

Each level of the onion contributes to defining goals, setting time frames, ownership and resourcing for the level below. Agile teams are generally involved with Release, Iteration and Daily planning. 

A release represents a set of user stories/prioritized features to be produced and delivered as part of a new release. Release planning sets the high level goals for the release, time line, scope and resourcing. A release progresses through one or more iterations.

Iteration level planning looks at the stories/features to be delivered in each iteration (typically of 2-4 weeks duration). Daily planning looks at activities to be performed each day in pursuit of the goals for the iteration. Agile teams generally meet for a short stand-up meeting every day to discuss tasks performed during the previous day, tasks planned for the day and bring up any impediments that may be affecting their work. The other three levels - Strategy, Portfolio and Product are generally owned by Senior Management and Product Management.

Agile planning is about simplicity and flexibility. Flexibility to adapt to changing requirements and make needed course corrections in the plan. Does this flexibility mean that agile teams cannot commit to delivery dates? No. Agile teams are capable of providing realistic completion dates and actually meet them. When plans are changed, dates do not necessarily have to be changed. Changing scope of features, dropping non-essential or lower priority features, adding resources, etc. may be resorted to for maintaining dates.