Starting with Startup - Agile Acceptance Challenges

When starting with the agile development (Scrum as base in our case) there are always some challenges that you will meet at the beginning and during the road.

I will try to explain structure of the team and acceptance of different agile parts.

Team

We have started with team of 6 (including me). Two of us already had experience with Scrum, one of us had experience with classical water-fall models and three had no experience with project management models. Few months later the team was extended with the tester.

The structure of the team was (is) following:

3 senior developers - 2 with agile experience and 1 with water-fall experience

1 intermediate developer - experience only with ad-hoc project management model

1 junior developer - no experience with any project management model

1 designer - no experience with any real project management model

1 tester (few months later) - experience with water-fall models

So let see how this team went through agile adoption.

For each part I will split acceptance into three parts:

- with agile experience - acceptance of the team members who had agile experience

- no experience -acceptance of the team members that had no experience with project management methodologies

- water-fall experience - acceptance of the team members that had experience with water-fall project management models.

Writing requirements (user stories)

With agile experience

Team members that already had agile experience had no issues in writing user stories.

No experience

Team members without experience with project management methodologies accepted user stories as the form of writing requirements. At the beginning there were some issues to write user story of the right size (sometimes they were too small or too big) but they got the right form very quickly.

Water-fall experience

In the case of developer user stories have never been really accepted. My feeling was that there was no even wish to understand what is advantage of user stories. User stories were constantly too big (e.g. As admin I can do anything). Actually this colleague have never accommodated to agile development and left the company later.

In the case of tester acceptance of user stories came much faster. For him it is much easier to understand what to expect when user story is implemented. It is obvious that tester prefers user stories than complex use cases or other functional specification.

Estimation

With agile experience

Surprisingly here we needed some time to accommodate. The reason was that I have decided to use story points as estimation method. But the team member with agile experience was used to ideal person days estimation. It was confusing for him for some time. But at the end estimation model with user stories was fully accepted

No experience and water-fall experience

After some turbulences at the beginning estimation model was accepted. The most issues were regarding over-estimation and under-estimation. But with help estimation became better and now is constant.

Meaning of DONE

Meaning of DONE in agile is very interesting. You should finish user story in the way that in the future you should not come back to that user story. User story should be ready for production.

With agile experience

Anybody with agile experience should not have problems to get meaning of DONE. At least that was the case with us.

No experience

At the beginning it was not so clear what DONE means. There were questions like: Is it already DONE? Can I improve this? I have such good idea, can I add it?

All the time I have insisted only on implementing user story and focusing on tasks. Very soon they realized what done means and actually realized why DONE is very helpful.

Water-fall experience

Here situation was much more interesting.

Developer with water-fall model experience had constant problems to get work DONE. There were always small tasks that can be easily finished later. There were always some missing features because they were not specified enough in the user story description. The "wrong" influence of the water-fall could be felt most of the time.

But tester was much happier. He read user story and had understanding what to expect from it. This way he can test and easily see if it is DONE. There are no answers like: I will do it later. It quite simple - if you sent user story for testing everything must work.

Testing

Under "testing" I mean writing unit and integration (as we use Grails) tests. Here situation differs from team member to team member. Some of them understand and know how to write good tests and some of the are still fighting to get the tests right. The most common issues are like:

- writing test that tests too much functionality and spread through number of classes or services.

- writing test that tests method not functionality

Conclusion

My feeling is that team accepted agile development very good. For team members without project methodology experience agile acceptance was easy and what is the most important natural. They have feeling that it is the right way to do the work. The most problems had a developer with water-fall experience. But the proof that having water-fall experience does not mean it is impossible to accommodate to agile is new team member that also came with water-fall experience but is accepting agile approach very good.

What is your experience in accepting agile?

Starting with the Startup - Setting Up Development Environment

After we have set basics for using Scrum we can start with our first sprints. Our first sprints were devoted to choosing and setting up development environment.

So let explore what do we know so far about the project (of course I knew much more than I can write in this post).

* Our main focus are desktop and mobile web applications

* Our core business is in the field of affiliate networks

* We will use agile process or to be more specific Scrum

So I will jump directly into decisions we had to make and what have we decided:

Web development framework

Basically we have been deciding between two choices: Grails and Wicket. Who is reading my blog from time to time probably already knows answer to this. We have decided for Grails.

Build process

Although Grails have nice build commands we needed something that is easier to integrate with the builds of other projects. Basically we have been deciding between maven and ivy. As Grails supports ivy much better than maven we have decided for ivy.

Continuous build process

More or less we have not been thinking too much here. We have decided for Hudson. Hudson is easy to install, has great front-end for management and number of very useful plugins. One note: although Hudson has Grails plugin we are not using it. We are using ivy in the Hudson to build the Grails projects.

Bug tracker

Although we would be happy not to have bugs that is not reality. So we needed bug tracker. As most of us had experience with Mantis we have decided that our bug tracking tool will be Mantis

Libraries repository

We will be depending on some external libraries. As we don't want to have them stored on Internet we needed some tool that handles dependencies. We have decided for Nexus.

Source code control

Although there are new source code control system like git we have decided to use probably the most common one Subversion

Mobile web development framework

After lot of investigation you find out that the market with the mobile web application frameworks is quite poor. The only available and really useful open source solution is: WALL NG. Luckily we have managed to almost fully integrate WALL NG with the Grails.

IDE Tool

Although we didn't have any experience with InteliJ Idea as this is currently only tool with good support for groovy/grails we have decided that our IDE tool of selection is InteliJ Idea.

In-house Grails plugin repository

As Grails has number of useful plugins and we knew we will be using them. This leads that we need local Grails plugin repository. Grails repository is set in the Subversion.

Having setup like the one mentioned above should give you fluent build process that is easy to repeat in the any moment. Additionally Hudson will ensure that your build is in the fit condition because it will report all build and test breaks.

Starting with the Startup - Start Using Scrum

After Scrum has been selected as process and the team have been educated about agile and Scrum we can start with setting up development environment. Very important thing to notice is that we can start using Scrum right away. Yes, Scrum can be used for tasks where you need to make decisions, investigation, setup of development environment and similar tasks.

As quite often those tasks are not too much time consuming and you want to review current status more often it is best to start with short sprints. E.g. we decided to make one week sprints. This means that you can review your progress, adapt to new findings and adjust your directions on the weekly basis. When you are deciding about length of the sprint you should ask question: For what time period can we be sure that we will not change our minds?

The first thing we did was not part of the sprint but we did that with the purpose. We wanted to select some Scrum tool.

More or less there are two basic options:

* Using excel for product backlog and sprint backlog
* Using some of existing (commercial or open sourced) Scrum tools

We have decided to use ScrumWorks basic edition with the addition of the big pin-board.

ScrumWorks enables you to have product backlog, sprint backlog with the burndown chart, and some advance reporting options. Pin-board is used for user story cards. Idea is to split the pin-board in columns (e.g. todo, progress, testing, done) so everybody can see the status of the current sprint by simply looking onto it. If you are asking: isn't Scrum tool enough, I would say no. The reason is that if you are not part of the team, to check the status of the sprint it is enough to check the pin-board. Without pin-board you would had to open Scrum tool and this is already enough to give it up on checking status of the sprint.

One of the most important features of the agile and Scrum is transparency. We want to be aware of the real situation in each moment. If the project or sprint is late we want to find it out as soon as possible.

So we have prepared our Scrum tools and now we are ready to start sprint(s) for setting up development environment.

So how we set up development environment will be described in the next post.

Starting with the Startup - Educating the Team

In the previous post I have explained why we have choose agile approach or more directly Scrum. But before you start using agile approach it is very important to explain what is agile and Scrum to the whole team and of course to the management.

In our case we are lucky because we have full support from the management. It means there are no "artificial" obstacles in implementing Scrum. The only real obstacle are we. So before going on I made presentation about Scrum, team members did some reading and then we discussed about the whole approach.

So when you are educating team do not forget to explain to them:

• Cores of the agile and the Scrum

And especially take care to explain:

• Why everything in Scrum is time-boxed
• What is user story and what are the differences between user story and use case
• What means DONE
• What is goal of the sprint
• What is velocity and why it gives better estimation when things will be done than "classical" approaches
• Why testing is so important in the agile approach
• What is burndown chart and how it helps the whole team

It may be clear why these points are important but I will shortly explain why I think it is important to take care there is full understanding of them.

Time-boxing - very often developers are used to implement some tasks as long as they want. Even after deadlines. In Scrum everything is time-boxed because this way everybody have to be focused that features have to be finished in proposed time. So it is not possible to change architecture forever. In one moment you have to decide and do implementation.

User story - in lot of companies and even in schools developers are used to thing in use cases. While use cases are good very often they put limitation that need not to be set in given moment. Classical examples are: then user clicks the button.... How the use case knows there is .... button. User story explains functionality that user wants to achieve without too much details.

Done - Done must mean: I am confident that feature implemented can go online. It is tested, written in good quality and I believe there are no bugs. This should be approach for all features that are marked done. You should not return back to this feature.

Goal of the sprint - you have to set goal if you want to have good feeling at the end of sprint. How could you know if your sprint was success or failure without clear goal?

Velocity - velocity is number that based on the experience (yesterday's weather) tells you how much team can implement in the given period of time. Velocity is better than "classical" approaches because it covers meetings, phone calls, unexpected bugs and everything that is happening more or less regulary.

Testing - if there are no tests how can you change the code and be sure that you didn't break any of the existing functionality. Well you cannot and you have to check everything after each change. If you have lot of tests you just run them and find out immediately if you have broke some existing functionality.

Burndown - is great tool that tells you without mercy you current status. It will tell you if you are doing good or if you are doing bad. Just have it in front of eyes.

Conclusion: If you have decided to take agile approach do not forget to educate developers what is agile. It is better to do it at the beginning then fixing problems in the middle.

Starting with the Startup - Selecting Processes

In the previous post about Starting with the Startup I explained that as our first step we have decided to choose processes. Of course one of the most important decisions to make is to choose right management methodology (framework). The product owner (managing director) was coming from the company with the strong full life cycle (waterfall if you prefer this way) process. He had no idea about any other processes and he was used to work in that way. Even all the next steps he planned were in that direction. There was plan to hire external consultant that will more or less write "book with the boxes" that we will implement. And that will be our solutions.

The good thing is that although he didn't know for something else than waterfall methodology he didn't like waterfall approach.

So as you may guess it was not hard to explain him what is agile, what is scrum and what are advantages of the agile approach compared to waterfall approach.

For those who don't know difference just shortly: in waterfall you define and plan the whole project in advance and then just implement what was planned. No changes are allowed or complex change request processes are defined to introduce new requirements. In agile style you create vision and rough plan of your project and then implement your project in increments and iterations. You allow project to grow and change in the way according you new discoveries or business needs.

Selecting agile approach out of the box would be mistake because for any decision you should have a good reasons. So let analyze situation we have.

• We are startup company with small number of developers.
• We have rough idea what is our goal
• Our main focus are desktop and mobile web applications
• The market where we are moving is changing rapidly
• We are not sure and not aware of all the features we will need in our applications

Possibility 1: Waterfall (or similar strong processes) - if we choose this path it means that we need to define all the features of our applications. Oops, we are still not sure how many applications we will have. Not to talk about features of those applications. Ok, but if we analyze problem carefully maybe we would be able to define all the features in all the applications. But the next problem is: market is changing rapidly. Are we sure we want to create detailed plans too far in the future knowing that market will change in the mean time. Decision: this is not the way we should go.

Possibility 2: Agile (Scrum) - I would say this is the direct hit. Agile approach is good to use if you have clear vision but not clear plan. If you have small (up to 9 persons) collocated team. If you requirements can change fast according to new discoveries and you need to adjust your goals according to market changes. Well if you check the list above agile is covering situation we have.

Decision: So actually it was not so hard to make the decision. We have decided to take agile approach implemented with Scrum.

Starting with the Startup

Few months ago (October last year) I have joined the startup company. An what is the real luck I have joined them really at the beginning. So when I came on my first working day there was server room prepared, tables with computers and few developers that joined the company a month before me. The core business of the company is (will be) affiliate networks and mobile web applications. It is quite young and very innovating area of the current industry trends. If you don't believe me Google it :)

The great thing in joining startup right from the beginning is that you have big influence on the whole process. It is never boring and commonly it is taken care that the team is created from great people and developers. And of course it does not mean that big and small companies do not have great people and great developers.

So when the team is established, servers are prepared and there is rough idea what needs to be implemented you can jump right away into development. You install Eclipse, and start writing code. And doing it this way you will fail in a few months :)

First steps to be done are to establish some basic processes. As my part is mainly considered to development processes I will go through development processes and development environment setup.

Well our decisions I will explain in the coming posts.

Scaling Scrum

My first contact with Scrum was related to "normal" Scrum: one team, at one place works on one project of average size. As I continued to investigate Scrum I have noticed that number of people are asking how to scale Scrum to larger projects and multiple teams. Question sounded so normal. Can I apply it to large projects?

Even now when I am explaining the Scrum this question is rising very often. Scaling seems like one of the most important things regarding Scrum. If we are not able to scale it, it is not worth to try it. Once I was asked this question by manager although I new they do not have any big project.

But as longer I am with Scrum this question seems odder to me. How many of you had a chance to work on such big projects spread around the globe? How many of you had a chance to work on projects that cannot be implemented by one team? I have been in small but also in very big companies and I didn't see project larger than 50 team members. Maybe I was in wrong projects :)

For me question is it possible to scale Scrum is not important. If Scrum can be successfully used on mid-size projects but it is not possible to scale it I would use it anyway on those mid-size projects. It is easy framework, it gives excellent results and usually people like it. So why should I bother with scaling anyway. I will think about that when I will need to lead on such big project.

So what is conclusion? Yes, it is possible to scale Scrum.
There is number of articles and books around the Internet. Just go and read them. But if you have really big project, then get somebody who has experience using Scrum on scaled projects.

As long as you are working on "normal" projects, you don't need to scale Scrum.

If you are new to Scrum, try to use it on small or mid-size projects. Don't worry about scaling. It will take some time until you are able to use it correctly on small projects and it will take much longer till you will be able to scale it without experienced Scrum Master.

Scrum: Lessons Learned

It is a year as we have started using Scrum as project management framework. At that time I was starting with a startup company and it was on me to decide and propose next steps. Somewhere at that time I read Scrum and XP from the Trenches and found implementing scrum web site. For me decision was clear. Most of the colleague had some experience with "kind of" agile software development. Unit testing, CI, SVN has been part of life. But Scrum, iterations, user stories have been something new and something we had to learn. Although I have studied lot of books we started with baby steps. Today I believe we are children but we have much more experience than year ago.

During our sprints we have made some common mistakes, we have made some less common mistakes and I will try to summarize what we have learned.

First let me shortly explain our situation. We are small startup company with number of projects. Therefore it is not possible to have consistent team size per project. Further on, very often we are not able to run more than two-three consequent sprints on the same project. Althout these are some Scrum pre assupmtations, I believe Scrum helped us a lot and we wouldn't be where we are if we would go without Scrum.

So what are the most important lessons learned so far:

• Product backlog - although it is stated in all books and articles we very often failed to have prioritized list of features to implement called product backlog. When you don't have product backlog you are never sure about your future steps. So, always have product backlog with features to be implemented.
• Product owner - when there is no real product owner, and this is unfortunately very often case in our situation, somebody else (product owner proxy) must take responsibilities of product owner. It our case it is me (playing role of developer, scrum master and product owner proxy).
• User stories - although Scrum does not explicitly requires to express requirements as user stories, we found out that user stories are the best way to express requirement. It is easy to prioritize them, it is easy to understand them and it is easy to know when user story is implemented. At the beginning we used more generic terms for describing what we want to implement. That got us into position of never ending requirements (requirements that are never implemented).
• Sprint planning - at the beginning we had short sprint planning session. We just listed what we want to implement without discussion. Consequence was that we didn't had common design understanding, we had different opinion of DONE. As we used longer sprint planning sessions (4-5 hours) those understanding got better.
• Unfinished user stories - do not let sprint to finish with not all planned user stories implemented. If this is happening too often somewhere you are doing mistake. Lower your velocity, make better definition of DONE. If team is used that sprint can be finished but not everything is implemented this will repeat again and again. At the end of the sprint everything must be implemented otherwise sprint is failure.
• Code quality - very often you want to finish tasks as soon as possible and therefore underestimating user stories. Then what usually happens is that at the end of the sprint code is not written with production quality and later on you have to return to that code. Actually what happened here is that you didn't finish user story. Therefore, give yourself enough time for user story implementation. Do not forget refactoring, discussions, integration tests, nice user layout and experience. Only if you satisfy all these requirements you will implement user story with production quality and will have potentially shippable code at the end of sprint.
• Estimation - even after a year, we are still making mistakes when estimating. We can noticed improvements but we are still unsatisfied. For this problem for us there is still not solution.
  

And for the end, remember, Scrum is team work. Scrum master is here to help and protect the team, no to command the team.

Daily Scrum - (Wrong) Order of Questions?

Daily scrum meeting is time boxed meeting held on the each day of the sprint. This meeting is short (should not take more than 3 minutes per person), is informative and should help team to organize. Daily scrum is for developers and only developers are allowed to speak during daily scrum. All others are welcomed but they are not allowed to speak.

During daily meeting each developer, one by one, answers to three questions:
1. What did you do yesterday?
2. What will you do today?
3. Are there any impediments?

Simple enough. Short enough. Very important.

The main goals of the daily scrum can be summarized as:
- everybody are informed what is happening
- everybody are aware about possible risks
- team can organize what will implement up to the next daily scrum.


The second question (what will you do today) and the third item of daily scrum summary (team can organize work) leads me to the point that order in which questions are asked is wrong.

If the user stories are small enough, everybody can promise what they will implement up to the next daily scrum. But very often that is not a case. Features are bigger, contain number of tasks, and having more than one developer on a feature in lot of cases can increase overall productivity.

So it seems to me that in the first round, all developers, one by one, should be asked two questions: "Whad did you do yesterday?" and "Are there any impediments?"

When everybody answers these two questions, team all together should discuss and decide who will implement what up to next daily scrum. This way, team selects the best possible way to achieve the final result, sprint goal. Also this way daily scrum gives you even better satisfaction of rule: team can organize work for the next working day.

Up to recently, we have been doing daily scrums by the book. Everybody had to answer three questions asked. But very often, it happened that after daily scrum, when we started with implementation, we changed our promises of what we will implement because we noticed there is much better path to increase productivity.

Because of that we changed order of questions as described above. And now, at the end of daily scrum we decide who will participate on which feature. To me this seems as better approach. After some time, when we have used to this approach, our daily responsibilities are better organized and we have feeling that we are doing it right.

What I would like to mention one more time is that I am aware that if user stories are small enough this approach need not to be necessary, but when user story is bigger and contains number of tasks, organizing developers to work on one feature can increase productivity. Even when user stories are small enough, and they are more or less independent, very often there is indirect dependency between them (e.g. architectural), and when this dependency is noticed by the developers having them work together on these user stories can increase productivity (in the worst case you will need less refactoring)

Working with User Proxies

My short summary of the book User Stories Applied by Mike Cohn.

In the previous post we saw different possibilities how to gather user stories.

The best way to gather user stories is to work with the users of the software to be developed. But sometimes, you are not allowed to communicated with users directly or in the worst case you even do not have users :)

In this case you have to work with user proxies. User proxy is a person that represents user but is not a user. There are different types of user proxies, and can come from different contexts. So user proxy can be:

• User's manager
• Development manager
• Salesperson
• Domain expert
• Marketing group
• Former users
• Customers
• Trainers and technical support
• Business or system analyst

User proxy can help you a lot, but you must be aware that user proxies do not have all necessary knowledge. So for example, user's manager knows something about system but will use only small part of the system. Risk is that you will develop system that will suite perfect to manager but will not be designed well for the real users. Similarly salesperson usually can explain you high level overview very well but can have problems with details. And there are similar kind of problems with all user proxies.

What is very important to remember: When working with user proxy if possible you should try hard to get your way through user proxy to the real user when ever it is possible.

To produce good software that meets user requirements you have to work with users and avoid user proxies when ever possible.

Gathering User Stories

My short summary of the book User Stories Applied by Mike Cohn.

In the previous post we saw why it is important to define roles and some techniques how to describe them.

When we have specified roles we need user stories. User stories need to be gathered somehow. Of course, the best practice is to gather user stories from user or customer. In any case, one should avoid to try to define user stories without user/customer.

Following techniques can be used to generate user stories:

• User interviews
• Questionnaires
• Observation
• Story Writing workshops

User interview is usually preferred style of user story gathering. Although interview seems simple way, usually it is not so easy. You must be able to lead interview but also to ask right questions.

Questionnaires are not so famous style of user stories gathering but can be useful if you have lot of users. In this case survey can be very good to gather information about features users would like to have.

Observation is simple but no always possible. But observing user how she interacts with your software can give you real understanding about her needs and how software should be changed to help her.

Story writing workshops can be very rapid way for generating number of user stories. These workshops are very productive but quality of workshop can depend on the meeting leader. For me, story writing workshop with right persons on the workshop is preferred way for gathering user stories.

Difference between iterative and incremental development

Do you sometimes also have problem to make clear distinction between iterative and incremental development? What if you should explain difference to customer or friend?

Hopefully these two pictures taken from Don’t know what I want, but I know how to get it will help you understand it.

When you work incrementally you are adding piece by piece but expect that each piece if fully finished. One picture is worth of thousands words :)
When you work iteratively you create rough product or product piece in one iteration, then review it and improve it in next iteration and so on until its finished. One picture is worth of thousands words :)
So is Scrum incremental or iterative?

It is iterative by the definition as we are running sprints. We are improving application iteration after iteration.

But I believe it is also incremental because user stories finished in sprint should be of shippable quality and therefore we should not return to them in next iterations.

So Scrum is iterative incremental process.

Is this your opinion too?

User Story - Define Roles

My short summary of the book User Stories Applied by Mike Cohn.

In the previous post I summarized basic attributes of good user stories (INVEST attributes). In this post I will summarize while user roles are important.

In the most of cases in each software systems there are different roles that are interacting with it. Before you start writing your user stories you should think about roles that will interact with your system.

This way you will avoid situation when all your user stories start with "As an user...."
This way you will have better understanding what capabilities you system should have.
Having roles it may very easy happen that you will discover user stories that you would probably forget in the first moment.

How do you come to roles:

• Make brainstorm about initial set of roles
• organize initial set of roles
  
• consolidate roles
• refine roles

Of course, brainstorming should not take more than hour. After this hour you should have better ideas of user roles supported by your system.
Having roles you are able to write better user stories because very often role gives additional clarity to the user story.

There are some additional techniques for defining roles: personas and extreme characters.
Personas and extreme characters can give another view into you system but you should use them only if your project really requires that.

User Story should I N V E S T

My short summary of the book User Stories Applied by Mike Cohn.

In the previous post we saw what is user story. Now I will summarize what are the main attributes of good user story.

The main attributes of good user story are:

• Independent
• Negotiable
• Valuable
• Estimatable
• Small
• Testable

User stories should be independent as much as possible. If they are not independent it can be a problem to estimate and prioritize them.

User stories should not be contract between developers and customer. It must be possible to achieve agreement what is part of the user story.

User story must present value to user or customer.

It must be possible to estimate user story. Usually it is not possible to estimate user story if it is too big or is full of domain specific terms.

User story should not be too big nor it should be too small. User story should be just right in the size. How to achieve right size of user story? Well it depends on team an experience.

It must be possible to test user story. If it is not possible to test user story then you are not able to tell when you have finished with user story implementation.

User Story

My short summary of the book User Stories Applied by Mike Cohn

User story describes functionality that will be valuable to user or customer of a software.

User story consists of three parts:

• Description
• Conversation
• Tests

As a general rule, user story description should be written without using technical or domain specific language. It must be understandable for both, developers and customer.

Conversation is short remainder about details of user story.

Tests are used to determinate if user story story is implemented and if user story is implemented correctly.

For me very nice example of user story in the book (although it is constraint) is:
A repeat customer must be able to find one book and complete an order in less than 90 seconds.

This user story has all important features of user story:

• It is easy understandable for developer and customer
• It does not use any technical or domain specific language
• It is independent
• It is easy testable

You can compare this user story to some more technical requirements like: search in database should be performed in no more than 2 secs. User must be able to navigate easy through pages. From these technical requirements you are not able to decide what is real intention of requirements.

For me this is already enough to be ready to give a user stories a chance :)

Book: User Stories Applied

I am just reading the book User Stories Applied written by Mike Cohn. Book is excellent and although I have read lot of articles and blogs about user stories on the Internet I found so many new and important information for me. Everybody interested in agile software development should read this book too. As I like to have the most important statements from the book written somewhere I will place them on this blog.
This will be something like book review done chapter by chapter. But do not expect that mentioned summary of the book will be enough. If you will like the summary then for sure you have to read the book.

The list of posts I wrote about this book is:

• User Story
• User Story Should INVEST
• User Story - Define Roles
• Gathering User Stories
• Working with User Proxies
  

Start estimating in ideal person days

I am aware that this title is in contradiction with my previous post. But in previous post I was more idealistic than I should be. The main problem when estimating complexity of user story with some abstract value is that you are able to give first estimation when roughly project will be finished only after first sprint.
When I woke up and came back to reality :) it became clear that in the most of the cases this is not acceptable. Customer very often wants to know at least some rough estimation about release plan, when project will be finished. Shortly, she wants some time based estimation.

From that point of view, you should estimate first set of user stories with story points that are equal to ideal person day. As soon as you have some first set of estimations you should stop thinking in ideal person days and use them as story points. It means, when you are estimating new user story, use previous estimations for complexity comparison. This way your estimations will be based on complexity factor but later on (after few sprints) your velocity will tell where are you standing according to plan.

Estimating in story points vs ideal person days

If you use Scrum or other agile methodology you are probably aware that estimation of requirements should be done in story points. If you have investigated further, you should be aware that story point should be some abstract value that is expressing complexity of feature to be implemented and story point should not equal to ideal person day. Some good reasoning you can find here. Very often, the usual mistake is that projects use story point as ideal person day. We did the same mistake too.

There is one additional interesting consequence if you use story point as ideal person day. It can be important if you are using Scrum as a development methodology. When using Scrum, very important value for you and for your customer is velocity. Velocity is number of story points team is able to implement in fixed period of time. Usually, as project goes, velocity should increase. This does not happen all the time but happens very often.

But if your user story == ideal person day, what may happen is that in reality your velocity will increase but it will not be visible in velocity value. How this can happen.

Let say that you have estimated feature F1 with 5 SPs at the beginning of the project. What is good to notice is that at the beginning of the project you are new in the project and technology. So you have estimated first set of features with SPs (ideal person days). After a two or three sprints you are aware of velocity and roughly you know what you can expect in the future. But as your project goes on, requirements are changing and you have to reestimate changed and estimate new requirements.

But now you have much more knowledge of the project. And here comes interesting situation. You have a new feature F10 you need to estimate. Feature F10 is very similar to feature F1, but because you already have experience with the project you will estimate F10 with 2 SPs. So what we have now is that two features of similar complexity are estimated with different ideal person days. If this situation repeats for set of features, you will estimate them with the less time for development. When you include them into sprint, your velocity will remain the same but in reality you have implemented more features.

If you are good or always bad same way in estimation, your velocity cannot change during project lifecycle because you are changing value of story point. This means, nor you nor your customer are not able to notice improvements in efficiency because velocity is same (or slightly changes) all the time.

If you use first set of features as a reference value for estimating new features, you are not changing value of SP, and if you develop features faster than at the beginning of the project, you will see that in velocity.

So what will be conclusion?
Never estimate in ideal person days. Always use story points as abstract value expressing complexity of feature to be implemented.

Now, when you are estimating user stories with story points and expressing their complexity you are able to calculate and estimate your velocity.

But what should you do if you are in the middle of the project and you are already estimating in the ideal person days. Solution is easy. Stop thinking in ideal person days, select representative set of features with different complexity and use them as reference for future estimation. This way you make transition from ideal person days to user stories immediately and more or less painlessly.

Velocity (not so) simple value

As defined in Scrum, velocity is simple and easily understandable value. Velocity is number of story points team is able to implement during one sprint. What is important to notice, velocity is value expressing only completely implemented user stories or features. User stories that are implemented only partially cannot be part of the velocity because Scrum recognizes only implemented and unimplemented features. So it is really easy. To calculate your velocity you should only sum story points of fully implemented user stories. And this works perfectly for the one sprint. And the next logical step is to use velocity from previous sprint as orientation point for velocity in the next sprint.

And here we come to "not so simple" part. Sprint as per definition is always of the same length. But what often differs is effort expressed in number of working days included within sprint. In the next sprint you may have bank holiday(s), somebody can have vacation, even you can have different number of team members or any other imaginable situation can change effort that will be part of the sprint. This means that effort included within the new sprint will be probably different than effort included in the previous sprint. And therefore you are not able to simply apply velocity from previous sprint(s) for the new one.

Solution to this problem is more or less simple and mainly is described in "Scrum and XP from the Trenches". You should record effort expressed as person days within sprint. One person day is one person day work on the project. So if 5 persons is working in the sprint that lasts for 2 working weeks (10 working days), effort expressed in person days is 50 person days. If one person had vacation 1 day long, effort within sprint is 49 person days. I believe it is clear enough how to calculate number of person days included within sprint.

When effort in sprint is expressed in person days, you have knowledge with what person days effort you are able to implement how many story points. Be aware that this maths will work only if you don't have big turbulences within team. If you are changing team for each sprint, then you actually don't have Scrum and therefor velocity as defined in Scrum is useless for you.

So what is the math. Let make an example.
(PD - person day, SP - story point).

Let say that for finished sprint 1 of 2 working weeks length we have following information:
Sprint 1: 50 PDs within sprint, 35 SPs implemented. This means that for 50 PDs our velocity is 35 SPs.

Let say that we know that in sprint 2 of 2 working weeks length we will have only 37 PDs (due the vacation and bank holidays). So what number of SPs we can expect to be implemented based on velocity from the sprint 1?
Equation is simple:
35SP : 50PD = xSP : 37 PD => x = (35 * 37) / 50 => 26 SP

So, for the sprint with effort of 37 PDs we can expect to implement user stories valuable of 26 SPs.

This equation should work in the most of the reasonable situations. At least that is my experience. But of course there are limitations. If you are changing the team too often, if you are changing technology, if you are changing the length of the sprint this equation will not work because all these situations changes preconditions used in the previous sprint(s).

Business value of Scrum

Usually when you read articles about Scrum you find out how it helps team to focus, to develop code faster, and with less bugs. You ship production quality product every few weeks, actually, after each sprint. Well, what you need is only product owner, product backlog, scrum master, scrum team, and other Scrum artifacts. When you find out and understand all benefits of Scrum you are starting with it immediately. But usually then you face with the first problem. Customer agrees to go with Scrum but is unable to take care of product backlog, cannot provide real product owner and actually has no Scrum orientation. And I have feeling that this situation repeats again and again.
From my point of view, reason for this is that customers very often do not want to educate them self about Scrum, follow strict corporate rules and number of articles describe benefits of Scrum only for software development teams, not for customers too.

If we take a look at mainly standard procedure within corporates, implementing project looks like this:

• they setup board for project
• board specifies requirements and creates budget
• they find responsible person that make details from requirements
• requirements are estimated and if they do not fit within budget, often project is not started
• if requirements fit into budget, project is started and board expects software product in some time frame
• after given time they receive product specified at the beginning. This product satisfies all requirements from specification, but the most often product is not what they expected, lot of functionality will never be used and after all, when they see product they can see that some requirements could be implemented much different (but they were implemented by specification)

Let us see how Scrum can help customers to avoid such situations. How would the same project run with the Scrum:

• board for project is setup
• board specifies project vision and setups initial budget
• responsible person, let call her product owner, specifies the most important features of project and development can start
• after very short time, 2-4 weeks, product owner is able to see first draft of application. She can review approach, notice some more important feature than planned sprint ago, change course of project if necessary.
  
• after a few months, the first simple version can be already deployed
• after this time, if board still things that project has future, additional budget can be assigned to the project. If they think that project has no future project can be canceled and not too much money is wasted
• project can go this way as long as really useful features exists. What is the most important, project can be canceled at any moment but valuable product will still exist. Project is finished when board or product owner decides so.

Let us try compare these approaches. Let call first approach traditional approach (waterfall :)) and the second one Scrum approach.

Budget and specification
With traditional approach you have to estimate budget of the whole project. To be able to estimate budget you need to specify most of the requirements with enough details. This means, even before project is started you must know how final product will look like because without that budget estimation is not possible. Well this is tedious, error prone work that usually does not give expected results. As you have to estimate all the future features, probability of big discrepancies is big.

With Scrum approach you can assign some initial budget to the project. This can be budget that is enough for few sprints, so it means no more than a few months. Project vision is enough, because what you really need to estimate is only the most important features of application. Don't tell me that all features of future application have the same importance :). As you have less feature to estimate, probability of big discrepancies is smaller.

I think that conclusion which approach is better is clear. At least, I would select Scrum approach.

Project development
So you have started your project with the traditional approach. In front of you is few months (or a years) of development, your team is struggling with requirements, ambiguous definitions, unclear descriptions, lack of support. No changes in specification are possible because it can influence budget, you have to stragle with hard change request process, you don't have time for it. Actually who cares if you new request is actually excellent idea, you should remember it during proposal phase. If you are lucky your project will be delivered on time and within budget. If you are lucky final product will be somehow similar to one you expected. Generally speaking, those projects are more or less always failures.

So you have started your project with the Scrum approach. In front of view is one sprint that will last from 2-4 weeks. After sprint you can review shippable product developed with production quality. You process with further sprint. After a few sprints your most important features are implemented and if you want you can even start using it. Your budget is almost spent so you need another meeting with a board. You present project to the board and they almost immediately after project kick off can see what a product will look like. If they like it they approve budget for next few sprints. But then you got excellent idea. It is real breakthrough within product. Well, this is Scrum :), no problem. Include it into next sprint and you will have it soon implemented. Thats all about complicated change request. All the time you are implementing the most important features of product. After some time your product is finished and you have product exactly by your expectation because you was able to review it and correct directions after each 2-4 weeks.

Well, which approach would you select?

Conclusion
I tried to have a look at the Scrum from business point of view. It is clear that Scrum is valuable for developers. But what is the most important in these days is business.

Business wants to minimize risk and maximize profit with minimum investment. That was the reason why waterfall model was introduced within first place. But as it turns out, Scrum can minimize risk and investment but at the same time maximize profit.