Introduction

If you're not already a member of the Scrum development group on Yahoo, you really should join. There's a fortune of information changing hands and you can learn so much from the interactions. Just recently there was a huge debate on the topic of technical stories.

The question

The underlying question the team debated was should technical stories appear on the backlog.
<!--break-->
If they are on the backlog, it means the technical stories are to be prioritized by the PO. This may not be such a good idea considering that PO's are generally going to be biased towards prioritizing features and functionality over technical stories. Examples given were "Installing Cruise Control", upgrading DB from MySQL to Oracle, Setting up VMWare etc. Most thought leaders on the forum argued that technical stories should not appear on the backlog, overwhelmingly so in fact. But some rightfully point out that all work requiring development resources should appear on the backlog.

What does Scrum say?

If you take a look at the definition from the Scrum Guide on Scrum.org, it states:

"The Product Backlog represents everything necessary to develop and launch a successful product. It is a list of all features, functions, technologies, enhancements, and bug fixes that constitute the changes that will be made to the product for future releases."

"The Sprint Backlog consists of the tasks the Team performs to turn Product Backlog items into a “done” increment. Many are developed during the Sprint Planning Meeting. It is all of the work that the Team identifies as necessary to meet the Sprint goal."

So what's the right answer?

Well, my answer is it depends and it depends on the context. For example, if your definition of done includes unit tests, automated tests etc then these work items don't need to be specific items on a backlog. This is stuff that gets done by the development team and there is no negotiation. Estimates need to include the time required to complete all of these elements of the definition of done.

But what about the type of story asked by one of the members "As a development team member, I want the existing unit tests to run under CruiseControl, so that I know if anything breaks". Where does this belong?

Well this is a perfectly written story but the story really has no ultimate value for the end user at least not directly. In this particular case I'd therefore suggest the following:

This type of story definitely doesn't belong on the product backlog but would be a perfect task that could exist on the Sprint backlog assuming you're tracking tasks. I am still in the Scrum camp on task breakdown as opposed to the XP folks who prefer to work just at the story level. If you're doing task level breakdown during the sprint planning meeting then this type of Story or work could exist on the Sprint Backlog as a task and the time associated with doing this can be tracked on the burndown. Most XP folks will say this is just micro-managing all over again.

To quote Ron Jefferies: "Technical stories have been found to be an inferior idea by many practitioners who have tried both ways. I don't know of a single one who would go back." Now Ron is a really smart guy and has ton's of experience and it's hard to argue against his opinions or any of the other smart folks on that forum.

In conclusion

I suggest you decide how to handle this as a team and do what you (the team) think is best. I will state however that the XP folks appear to be the most progressive in forging new ground in agile efficiencies and techniques so watch what they say carefully and even consider what they say.

Introduction

I may have blogged about this previously. I have written so many blogs, I can't recall any more. However questions regarding Sprint length surface on the forums regularly.

As per usual, the answers one must give always depends on the context and every context is different than the next. So let me start with the context - this is an excerpt of a post on the scrum development group on Yahoo. Incidentally, Yahoo groups is a good place to hang out. You learn a lot from all the questions and the different contexts facing teams around the world.

The Context

A team of 5 members currently working with 10-day sprints. They haven't managed in the previous 5 sprints to have 100% of the User Stories completed. It is typically around 60-70% completeness.

There is proposal to increase the sprint duration to 15 days "because doing review meetings and planning every 10 days is a lot of overhead" according to the team.

My thoughts

Let me start out by stating some facts ...

The official Scrum sprint length is 30 days. However I don't think (I don't have facts to back me up on this but it's the sense I get from all the communications on all the forums) there are many teams working to 30 days any more.

Much of the Agile community agrees that shorter Sprints are better. So 2 week Sprints and even 1 week Sprints are becoming more the norm.

Why are shorter Sprints better?

1. Well we have learned from the Lean folks that shorter Sprints means less work-in-progress which means shorter cycle times and overall less waste.

2. Additionally, shorter Sprints tends to stress your process, revealing any flaws. Like no automated build process, automated test harnesses our unit test frameworks. Fixing these flaws has a tendency to provide leaps in productivity gains for your organization.

So assuming you buy the argument that shorter Sprints are better. My initial quick answer to the question is don't try to lengthen the Sprint. Rather try to figure out why you're only hitting 60% - 70% of your originally committed goals.

By the way, 60% - 70% may not be that bad, after all you have a team that is currently demonstrating a consistent output Sprint after Sprint.

So that leads me to think that either the story point estimation is not consistent, or the team is just over-committing. So I would suggest that they do the following.

Try to really assess what is going on in the retrospective. Let team members speak freely about their thoughts on the matter.

I would definitely spend a little bit of time re-assessing the size of a few completed items i.e. if the story was 10 points originally, what would they estimate the size now, after the fact. Re-assessing the relative size may well fix the problem.

Some folks, most notably Ron Jefferies, would argue why do you need to get your estimation down pat. Well in my opinion for one, predictability goes a long way to help remove team stress. So its great for a team to say we can commit to say 100 points and deliver between 90 and 110 each Sprint. The business will love you for this.

Whats good about this problem in and of itself is that Scrum is doing what it's supposed to do; surface issues for the team to resolve. And if the team feels that going to 15 day Sprints is the right thing to do, so be it - it might well be. But I would try to first figure out why 2 weeks is not cutting it. Many teams make it work so it should be doable.

Hope this helps if you're in the same boat. If not at least if it provides food for thought!

Jack
agilebuddy

Introduction

Based on a recent post on yahoo forums, seems like there may still be confusion out there as to what the differences are between these two roles. Questions like, is there overlap? can the Product Manager take on the responsibilities of the Product Owner? what are the specific requirements for either role? pop up all the time.

There was a really good discussion on the Scrum Development Yahoo group on this topic and some really good points were made. So I'll try to distill this for you here and of course put my own twist on this.

I think that the founders of Scrum purposely chose a different title for a reason. They could have easily just kept the title the same. But I think there was good reason for this. And that is that the PO has a specific set of duties in the Scrum role.

How do their roles differ?

First and foremost is that the PO drives the priorities for the development team. This is done via the Product Backlog as a vehicle for communicating priorities. Essentially, a company has a certain available capacity to turn requirements into working functioning code. How that capacity is used up is completely in the hands of the PO.

In order for the PO to do this, the PO needs to understand the bigger picture. Whether or not the PO distills this information from other roles or whether he has to get this information himself depends on the company.

Based on his/her understanding of the bigger picture, there are some additional specific duties that the PO must perform (not necessarily defined anywhere):

1. Articulate the product vision to the team
2. Define the goals at the beginning of every sprint
3. Tell the story behind each user story so that the development team understands what is required. So the PO must understand the end user requirements.
4. Define or help define the user story acceptance criteria so the team knows when they are DONE
5. Be able to prioritize the stories and be able to negotiate/collaborate on priorities with the team. Negotiate priorities occurs when after taking the top priorities off the backlog; there may be some remaining capacity that the next highest priority story won't fit in to. So in those cases, a lower priority feature could be picked.
6. Must be available at all inspect and adapt points to answer questions and help guide the team empirically

Product Managers on the other hand must be able to do a whole bunch of other things, including but not limited to:

1. Defining the marketing strategies and outbound marketing communications
2. Pricing strategies
3. Understanding the positioning of the product in the market place
4. Competitive analysis

A couple quotes from the forum worth repeating here

"Scrum does not prescribe further responsibility beyond optimizing development to ensure business success" (Anonymous)

"The Product Owner role, on the other hand, is really about representing the business side and working with engineering to optimize the software (or technology) delivery part of the entire product solution." (Greg)

The best ..

"The product owner role is a genuinely new role and disruptive for most organisations, as it does not easily map onto existing roles and structures" (Roman)

In summary, depending on your situation, the PM and PO roles will be performed by the same person, a group of people or separate people - frankly I don't care. As long as there is a person dedicated to doing the PO duties and responsibilities as defined in the 6 bullets above, then all is good from a scrum perspective.

Introduction

I blogged about this some time ago and then posted the blog on various agile forums to judge peoples responses.

Most of the responses were well reasoned, however, one of the responses I received shocked me somewhat and so I feel that it's worth blogging about this particular situation once more.

The response I received was "You're not serious you're going to ignore the PO" and "You can't be a slave to the process"

In all fairness, there are many situations under which the need to switch stories arise. And the specifics were not really provided. For example:

How long are the sprints?
How far into the current sprint are you?
Are there stories that have yet to start that is of similar size that you can switch it out with?
Is this a critical issue that needs to be fixed ASAP as customers are complaining and may negatively impact revenues?

Those are some of the questions that need to be asked when making that decision.

In response to being a slave to the process...

Well you're either a slave to the process or the team is a slave to any chicken in the company who shouts the loudest. Lets go back to basics and why the Sprint is there in the first place. It's designed to provide stability for the team to get stuff done. Hopefully you're doing short sprints so it's not a lot of time before the team pops it's head up again and asks for more direction.

If the team listens to whatever the next flavor of the month is, then we're back to square one where there's just chaos and nothing gets done. Seriously, smart people figured out why we need to do it this way. There is strong evidence to support that this makes a difference, a really positive difference. So lets not willy nilly and go changing plans whenever someone in the organization feels like there's something more important to do.

Moreover, in this situation, I think it's important the team asks some serious questions as to why suddenly there's a story that's so super urgent that it calls for a change in plan. Lets say you're doing 2 weeks sprints and lets say you're midway. This means in reality that 5 days ago, nothing was more important (top priority items get selected to go into the sprint). Why all of sudden is their a need to change direction so soon after. Additionally can't it wait another 5 days?

Now I am sure there are times where such a situation arises and that's fine. I would never be so hard-line to suggest that the team doesn't collaborate over this and decide what are the best options. And in such a case, it's important that the team does this.

But lets be very careful how we deal with this. Because once you do this once, it's a slippery slope after that.

So what would I do. As already mentioned above, I would sit down with the team. Have the PO explain the dilemma. Thereafter, it's up to the team to decide if there is an easy swap out that doesn't impact the Sprint goals and productivity. Ultimately if it is possible, I am sure most teams would do it any ways. Level heads should prevail.

Introduction

Seems like there's lots going on in the agile world right now. Lots of talk about Lean and it's impact on Agile. Lots of attacks going on at the CSM certification. Kanban is all over the news these days. And just last week, I read about a new Agile methodology called Stride.

So how do we make sense of this all?

My opinion is that there is value in each of the methodologies (for the purposes of this blog I'll refer to them all as methodologies even though some of you might not think of them as such). It's real important to read about them all so that you are armed with enough knowledge to know what's out there. I see this as a toolset from which you can choose for your specific situation.

In order to illustrate ....

Scrum is a methodology and process that provides the mechanisms for teams to learn and adapt. Scrum however doesn't say much about the meaning of DONE and how to accomplish that. That's where XP comes in. XP has great practices around engineering discipline. It teaches us all about craftsmanship and producing quality work. I personally cannot see anyone practicing Scrum without at least some elements of the XP toolset. Be it pair programming, TDD, ruthless refactoring, emergent architectures etc.

Lean on the other hand is way more philosophical, but they have great teachings. For example, recognizing that work-in-progress is a liability is huge. If you start to think like this, you're going to minimize work-in-progress and as a result you will improve overall cycle time. With Lean, people come first. What effect does this have on your organization? Well happy teams make happy customers, better quality software, improved work culture.

And Kanban? Well Kanban helps teams with flow (i.e. cycle time, throughput etc) and almost eliminates the need for traditional sprints which I won't get into here (subject for another discussion). So many teams are using kanban boards for controlling the workflow of tasks or stories, or both.

Scrum has also been said to have problems with scalability and cross site development shops. Well Stride in it's infancy (not even sure you can call it an accepted methodology yet) has adapted Scrum to provide capabilities for better handling these sort of situations.

So what do you do?

Well in my opinion Scrum provides the best overall process or mechanism to manage agile project. It's a good base to start with and I would definitely start with Scrum. But you can't go it alone with Scrum. You have to pick and pack from other methodologies till you get what works for you.

I think Agile is evolving and most likely wont stop. And why should it. I want us to get better at it. And there's so many smart people thinking about how to make software development better. I can't wait to see what it will be like in 5 years from now.

Software engineering is still a very human endeavor. Its a complex process that requires our ability to create non-standard solutions for non-standard problems. But with this ability to creatively solve problems comes a tendency to introduce defects, one of the seven wastes lean production and lean programming try to eliminate. Jack Milunsky describes several ways to reduce the amount of defects in his article. I want to look at another idea from lean manufacturing for reducing the number of defects called Poka Yoke in Japanese and have a look at how we can apply this idea to software engineering.
<!--break-->
Poka yoke is a Japanese term that means something like fail-safing or as I like to call it fool-proofing. The idea is that instead of just telling people how to do things you design your process, your tools or your product in such a way that it becomes harder to make mistakes and easier to do things the right way.
USB cables are a good example from manufacturing. It's impossible to accidentally connect two USB printers together because of the way the plugs are designed. You can only connect a USB device to a USB host. Another well know example are microwave ovens that turn off the moment you open the door.

Poka yoke in software engineering takes a bit more creativity. Software is incredibly flexible so it's often harder to define right and wrong ways to fit parts together. First lets look at a couple of ways we already do poka yoke without knowing it to get some inspiration. Later we can look at how we can fit more of it in our process.
We have a lot of practices that make it more difficult to check in faulty code. Unit testing and regression testing together with a continuous integration system makes defects very visible very fast. A similar but older technique is coding by contract. Defining pre-, and post-conditions for every function in your code making it fail fast whenever it is called in the wrong context or with the wrong parameters.
Another concept in API design that is gaining popularity is 'convention over configuration'. Instead of letting consumers of your API configure every aspect before they can start to use it you give them a set of sensible defaults that 'just work'

Of course it's easy to go overboard with fool-proofing your process. We know that for every fool-proof system the universe just invents a bigger fool. The trick is to just fool proof the parts of your process that repeatedly cause defects. The best way to do this is to bring it up as part of your retrospective. Try to find the weak spots in your system and make them more robust by setting sensible defaults or adding tests. Sometimes even improving names in your code can improve things.

Introduction

When one looks at all the wastes, defects has to be the most obvious one. The cost and repercussions of finding defects varies depending on where in the cycle they're found. Defects found early on in the development life-cycle are way less costly to resolve than defects found later on in the cycle; the most expensive being when applications are already in-production.

Additionally, depending on when the defects are found, defects can and do trigger other wastes like task switching, relearning etc.

Defects can be very costly for an organization. So the trick with defects is that you need to 1) Prevent them from happening in the first place and 2) Find and fix them as early in the development life-cycle as possible.

So what can you do to prevent them from happening in the first place?

1. Keep your stories small, and unambiguous.
2. Ensure that each story has well defined acceptance test criteria (assisted by input from the customer).
3. Ensure that your code is well tested. Adopting good Test Driven Development habits will pay back in spades.
4. Pair programming and code reviews are excellent ways in which to prevent bugs from being created in the first place.
5. The more you automate the better. So automated functional testing is a good way to automate the functional test plan and get results fast and to ensure that new code changes aren't breaking other parts of the code.
6. Ensure staging environments mimic production as closely as possible.
7. Make sure that as you find new defects, you add unit tests to your test suite to keep your unit testing current.
8. Finally, document your code well.

An interesting quote from Mary and Tom Poppendieck's book called Implementing Lean Software Development,

"If software routinely enters final verification with defects, then it is being produced by a defective process"

.... which needs to be fixed. So...

Step 1. Prevent the bugs from happening in the first place by "mistake proofing" your code
Step 2. Do whatever you can to find the bugs early on in the process while the code is fresh in the developers minds - i.e. ensure you have good functional test plans and test tools.
Step 3. Ensure a proper staging (or test environment) that mimics production environment as closely as possible
Step 4. Get the software into the hands of customers as quickly as possible, so good Agile practices help here.
Step 5. If you find defects, ensure they never happen again by keeping your unit tests current.

Few days ago I was honored to speak about Agile Planning on Agile Eastern Europe conference that happened in Kiev on 18th and 19th of September. I promised many people to share the conference photos, so here is a slideshow. I also added some pictures of the Kiev and its people to help you feel the atmosphere and showcase my great photo talent If you prefer looking at photos one by one, you can find them in this flickr collection.

I hope you'll like the pictures.
Enjoy!
<!--break-->

Pair programming is one of the more controversial extreme programming practices. Having two people work on the same piece of code at the same time looks very unpractical and inefficient to someone not familiar with this practice. Pair programming proponents like me are usually quick to point out the benefits like improved quality, less rework, better communication and better knowledge sharing within teams but I think the biggest reason pair programming works is usually kept quiet.
<!--break-->
People work harder when there is someone looking over their shoulder.

I'm going to be completely honest with you here. When I work alone I spend a considerable amount of time surfing the web, reading email, twittering (you can follow me at @mendelt) getting coffee and talking to coworkers. This means I'm spending less time working but it also means I'm constantly interrupting myself during the time I am doing work making me even less productive.

Now we don't tell our managers this because it wouldn't do anyone much good, the problem is a bit more complex than just people slacking off. What I've observed is most people have a hard time pacing themselves. We do really focused work for five minutes and then take a break. Unfortunately breaks have a habit of taking more time than they should and productivity goes out the window.

To solve this problem we need a way to maintain a sustainable pace. Pair programming is a way to do just this. Forcing yourself to explain what you do to your pair is a great way to maintain a sustainable pace and work a bit harder.