Friday, November 16, 2007

Velocity (not so) simple value

As defined in Scrum, velocity is simple and easily understandable value. Velocity is number of story points team is able to implement during one sprint. What is important to notice, velocity is value expressing only completely implemented user stories or features. User stories that are implemented only partially cannot be part of the velocity because Scrum recognizes only implemented and unimplemented features. So it is really easy. To calculate your velocity you should only sum story points of fully implemented user stories. And this works perfectly for the one sprint. And the next logical step is to use velocity from previous sprint as orientation point for velocity in the next sprint.

And here we come to "not so simple" part. Sprint as per definition is always of the same length. But what often differs is effort expressed in number of working days included within sprint. In the next sprint you may have bank holiday(s), somebody can have vacation, even you can have different number of team members or any other imaginable situation can change effort that will be part of the sprint. This means that effort included within the new sprint will be probably different than effort included in the previous sprint. And therefore you are not able to simply apply velocity from previous sprint(s) for the new one.

Solution to this problem is more or less simple and mainly is described in "Scrum and XP from the Trenches". You should record effort expressed as person days within sprint. One person day is one person day work on the project. So if 5 persons is working in the sprint that lasts for 2 working weeks (10 working days), effort expressed in person days is 50 person days. If one person had vacation 1 day long, effort within sprint is 49 person days. I believe it is clear enough how to calculate number of person days included within sprint.

When effort in sprint is expressed in person days, you have knowledge with what person days effort you are able to implement how many story points. Be aware that this maths will work only if you don't have big turbulences within team. If you are changing team for each sprint, then you actually don't have Scrum and therefor velocity as defined in Scrum is useless for you.

So what is the math. Let make an example.
(PD - person day, SP - story point).

Let say that for finished sprint 1 of 2 working weeks length we have following information:
Sprint 1: 50 PDs within sprint, 35 SPs implemented. This means that for 50 PDs our velocity is 35 SPs.

Let say that we know that in sprint 2 of 2 working weeks length we will have only 37 PDs (due the vacation and bank holidays). So what number of SPs we can expect to be implemented based on velocity from the sprint 1?
Equation is simple:
35SP : 50PD = xSP : 37 PD => x = (35 * 37) / 50 => 26 SP

So, for the sprint with effort of 37 PDs we can expect to implement user stories valuable of 26 SPs.

This equation should work in the most of the reasonable situations. At least that is my experience. But of course there are limitations. If you are changing the team too often, if you are changing technology, if you are changing the length of the sprint this equation will not work because all these situations changes preconditions used in the previous sprint(s).

Business value of Scrum

Usually when you read articles about Scrum you find out how it helps team to focus, to develop code faster, and with less bugs. You ship production quality product every few weeks, actually, after each sprint. Well, what you need is only product owner, product backlog, scrum master, scrum team, and other Scrum artifacts. When you find out and understand all benefits of Scrum you are starting with it immediately. But usually then you face with the first problem. Customer agrees to go with Scrum but is unable to take care of product backlog, cannot provide real product owner and actually has no Scrum orientation. And I have feeling that this situation repeats again and again.
From my point of view, reason for this is that customers very often do not want to educate them self about Scrum, follow strict corporate rules and number of articles describe benefits of Scrum only for software development teams, not for customers too.

If we take a look at mainly standard procedure within corporates, implementing project looks like this:
  • they setup board for project
  • board specifies requirements and creates budget
  • they find responsible person that make details from requirements
  • requirements are estimated and if they do not fit within budget, often project is not started
  • if requirements fit into budget, project is started and board expects software product in some time frame
  • after given time they receive product specified at the beginning. This product satisfies all requirements from specification, but the most often product is not what they expected, lot of functionality will never be used and after all, when they see product they can see that some requirements could be implemented much different (but they were implemented by specification)
Let us see how Scrum can help customers to avoid such situations. How would the same project run with the Scrum:
  • board for project is setup
  • board specifies project vision and setups initial budget
  • responsible person, let call her product owner, specifies the most important features of project and development can start
  • after very short time, 2-4 weeks, product owner is able to see first draft of application. She can review approach, notice some more important feature than planned sprint ago, change course of project if necessary.
  • after a few months, the first simple version can be already deployed
  • after this time, if board still things that project has future, additional budget can be assigned to the project. If they think that project has no future project can be canceled and not too much money is wasted
  • project can go this way as long as really useful features exists. What is the most important, project can be canceled at any moment but valuable product will still exist. Project is finished when board or product owner decides so.
Let us try compare these approaches. Let call first approach traditional approach (waterfall :)) and the second one Scrum approach.

Budget and specification
With traditional approach you have to estimate budget of the whole project. To be able to estimate budget you need to specify most of the requirements with enough details. This means, even before project is started you must know how final product will look like because without that budget estimation is not possible. Well this is tedious, error prone work that usually does not give expected results. As you have to estimate all the future features, probability of big discrepancies is big.

With Scrum approach you can assign some initial budget to the project. This can be budget that is enough for few sprints, so it means no more than a few months. Project vision is enough, because what you really need to estimate is only the most important features of application. Don't tell me that all features of future application have the same importance :). As you have less feature to estimate, probability of big discrepancies is smaller.

I think that conclusion which approach is better is clear. At least, I would select Scrum approach.

Project development
So you have started your project with the traditional approach. In front of you is few months (or a years) of development, your team is struggling with requirements, ambiguous definitions, unclear descriptions, lack of support. No changes in specification are possible because it can influence budget, you have to stragle with hard change request process, you don't have time for it. Actually who cares if you new request is actually excellent idea, you should remember it during proposal phase. If you are lucky your project will be delivered on time and within budget. If you are lucky final product will be somehow similar to one you expected. Generally speaking, those projects are more or less always failures.

So you have started your project with the Scrum approach. In front of view is one sprint that will last from 2-4 weeks. After sprint you can review shippable product developed with production quality. You process with further sprint. After a few sprints your most important features are implemented and if you want you can even start using it. Your budget is almost spent so you need another meeting with a board. You present project to the board and they almost immediately after project kick off can see what a product will look like. If they like it they approve budget for next few sprints. But then you got excellent idea. It is real breakthrough within product. Well, this is Scrum :), no problem. Include it into next sprint and you will have it soon implemented. Thats all about complicated change request. All the time you are implementing the most important features of product. After some time your product is finished and you have product exactly by your expectation because you was able to review it and correct directions after each 2-4 weeks.

Well, which approach would you select?

Conclusion
I tried to have a look at the Scrum from business point of view. It is clear that Scrum is valuable for developers. But what is the most important in these days is business.

Business wants to minimize risk and maximize profit with minimum investment. That was the reason why waterfall model was introduced within first place. But as it turns out, Scrum can minimize risk and investment but at the same time maximize profit.

Saturday, November 10, 2007

SCRUM and CMMI

This is my opinion about Scrum and CMMI and maybe I am getting it all wrong. We are using Scrum for almost a year now and we want to find out our CMMI level compliance. But let tell the whole story.

So you have read a few books about Scrum and started to use it within company. When your Scrum process is stable enough you decide to see your CMMI level. As usually you start with Google. After reading a few articles, maybe some pages from books it seams (at least to me) as a very simple process.
You are happy and think that making your company CMMI 2 or 3 compliant will not be so hard task. At the end you have saw so many tables that map Scrum and CMMI almost to the CMMI level 3.

But then you start to read CMMI specification. Two different types of CMMI approach, number of processes for each CMMI level. And then you realize as with all tasks, it is not so easy to accomplish it.

Recently, I got task to prepare ground for CMMI 2 and 3 compliance. Now we are using Scrum more or less successfully for almost a year. So as mentioned above, after some short investigation I thought this will be almost a piece of cake :) But then after some reading I found out that it is not so easy. You have to invest time and patients.
So I need a plan how to achieve CMMI 2 compliance. As we are using Scrum and we like it, idea is that we must achieve CMMI 3 compliance using Scrum. Because I am already on Scrum, I decided to create a plan. So this is my plan:
  • create initial "project development process guide" based on Scrum and agile software development
  • Study CMMI spec. so I am aware of pros. and cons. of it and have good overview
  • Create mapping and discover gaps between "project development process guide" and CMMI level 2
  • Update "project development process guide" so it is CMMI 2 compliant
  • Wait some time so company use to new CMMI 2 compliant "project development process guide"
  • Create mapping and discover gaps between "project development process guide" and CMMI level 3
  • Update "project development process guide" so it is CMMI 3 compliant
  • Use new "project development process guide"
I hope we will be able to achieve CMMI 3 compliance within next 12 months.

Currently I am working on the first task, writing "project development process guide" that is based on Scrum but covers little bit wider area. Although Scrum is more management than software development framework it does not cover areas like versioning system, continuous build system, quality management and other more or less known issues regarding software development. I am aware that we need this areas within our guide. So I want to specify some general rules (idea taken from Scrum) how to develop software projects and then create one implementation of this rules.
This implementation later on must be CMMI compliant but tools used to achieve it (bug tracker, SVN...) are not hardcoded so they can vary from project to project.

When this guide is prepared, we will use it for some time until we are satisfied and feel good about it. When this happens, first task from the sprint plan will be finished. After that we are ready to sail towared CMMI waters.

We will see what happens next.

Sunday, October 28, 2007

Checkstyle report with Maven2

Sorry if I will not be precise enough but I spent some time until I managed to configure custom checkstyle to be included within report generated by maven2. I want to share this knowledge so maybe somebody will need not spend so much time on this because I followed description on checkstyle maven2 plugin but that didn't work.

I have found solutions from the following mailing list archive. Follows short description.

Problem is following. I have maven java project and custom checkstyle configuration. I want to include checkstyle report into maven2 generated report. Solution is:

Create new maven project and into resources folder include your custom checkstyle configuration file e.g. my_checkstyle.xml.
Deploy this project to local or your intranet maven repository.
For this example groupId is com.my.checkstyle and artifactId is checkstyle and version is 1.0-SNAPSHOT

Then within pom of project for which you want to have custom checkstyle validation add following lines:

<build>
<extensions>
<extension>
<groupid>com.my.checkstyle</groupid>
<artifactid>checkstyle</artifactid>
<version>1.0-SNAPSHOT</version>
</extension>
</extensions>
</build>

<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-checkstyle-plugin</artifactId>
<configuration>
<configlocation>my_checkstyle.xml</configlocation>
</configuration>
</plugin>
</plugins>
</reporting>

Run mvn site for your project and your customized checkstyle report will be included within your project.

Note: For some reasons, not all customizations work. Plugin is reporting errors when module element contains subelements like suppressLoadErrors. Currently I don't have enough time to play with this problem. I think solution is with using suppress warnings. If I return to this problem and find solution I will post it here.

Link to maven2 checkstyle plugin is here.

Saturday, October 27, 2007

Checkstyle vs Nonsense code

Recently we have started to use checkstyle to verify our code. Although first intention of checkstyle is to check agreed code conventions, checkstyle is able to check for some known code structures that often mean wrong design of code. This is useful because you can catch common mistakes very early and therefore avoid bugs later.

Among others, one of our checks is that there should not be empty block of code, like catch block.
And then we had our first rule violation. For testing of course we are using JUnit. Standard design of JUnit 3.8 test to verify that certain part of code throws exception looks like this:

try {
performExceptionThrowingOperation(...)
fail();
} catch(ExpectedException excepiton) {
// expected exception
}


Well as you can see catch block is empty, there is no java code in it. And of course checkstyle complained about this.
I see two options here:
  • leave code as it is and do not fix checkstyle violation
  • add nonsense line of code like assertTrue(true)
At the end I have selected third option :), change checkstyle rule so it accepts in the block also plain text e.g. comment (not only java code) and therefore checkstyle violation disappeared.

But now I am getting to the point of my thoughts. As code grows, there will be more and more places where maybe it will be strange (e.g. adding assertTrue(true)) to fix checkstyle violation and not possible to solve it with simple additional configuration of checkstyle rule.

Question is: should we leave checkstyle violation as it is or add strange code of assertTrue(true) style?

My opinion is that we should add strange code and fix checkstyle violation. If we start to leave checkstyle violations open, number of opened checkstyle violations will grow with time and checkstyle violations warnings will loose their intention warning about bad code design. Because if there is long existing list of false checkstyle warnings we may miss the new, but important one. And then, instead of fixing the bug immediately, maybe we will fix it in later testing phases. From my point of view, it is better to have some strange but harmless code lines than to miss one important bug because of long list of existing checkstyle warnings.

If somebody will ever read this :) it would be nice to hear your opinion.

Friday, October 26, 2007

My new blog

So I have setup my blog. Actually, I am not big fan of blogs. I thought I will never have one. I read blogs occasionally and consider myself not being part of blog fever. But then, suddenly I have decided to create blog. I am still not sure what will be in my blog, but maybe I will place some ideas here. Maybe somebody will read my blog. Maybe this will be my first and last blog. Who knows :)
I will see what will come out of this.

Nice evening to everybody.