Recently I’ve read a very interesting post by Anna Forss called “Stupidity of the Team”.  While Anna concludes, that it’s healthy to introduce diverse opinions and invite opposing minds to dissolve the like-mindedness of homogeneous teams, I think there’s one important nuance that shouldn’t be overlooked.

Let’s think:  teams exist for some purpose. To resolve some goals. If it’s a small product development company - then this team exists to develop a product. Permanent rebels are not welcome in any group - because what they do with their rebels, argues, drawing attention to themselves - they blur the focus of the whole purpose why team exists. Of course, a team will naturally outcast this person.  Next, if a team is bombarded by controversial opinions and judgments, they will spend all their time evaluating and thinking if this is right or wrong. They will get busy sticking tags on new opinions instead of focusing on their work - and they will inevitably lose their focus.

Life in a small development team can be compared to living in a sheltered reality, with it’s particular culture. An isolated sheltered reality will not last for long if it’s completely isolated, so emerging on the surface for a gulp of fresh air is really needed.  As a member of a small team - can you remember when the opposed rebels and opinions really did help? When they triggered something that the team would not have thought by themselves? Well, of course, if someone comes up and says - “your UI is bad” - then another person comes up and says - “your UI is bad” - then you start thinking that it’s indeed something wrong with it. You’ve got this signal from outside world. You work on it. Basically, you know what you should work on. The outsider’s opinion has accomplished it’s task - the outsider’s opinion can now go, because you’re not interested in hearing variations of one and the same opinion.  You get to work, and you work to develop a nice new UI.

There’s no need to focus on outsider’s opinions and pay too much attention to them. Outsider’s opinion is just a trigger to team’s actions - it’s not something that the team should busy their brains with all the time. In a way, diversity of opinions may be even harmful. I guess that’s why we’ve got leaders - authorities who tell the crowd “THIS is your Holy Grail”.

My conclusion is: healthy vaccination with opinions opposing a team’s culture is  good. But don’t overdo with them. Too many opinions will not increase collective intelligence for this team’s specific purpose, they will blur it.

Kai Gilb started a series of posts that challenge Agile Software Development. Part 2 is about creativity and “how well” focus in software development. While I agree with the problem, I strongly disagree with the solution. Kai recommends to write requirements (the format he uses doesn’t allow me to call it “user stories”) in a form that does not contain solution:

Developers Are Not Product Owners

Then developers should creatively find the ways to implement this requirement, to invent the solution and to solve the “how well” puzzle. We’ve got a huge problem here indeed. Most developers can’t invent solutions at this high level. They can create beautiful architecture and build the quality in. They can write tests upfront and use powerful tools to speed up development. They can do many things, but often they can’t invent a “how well” solution that will make customers happy. Why?

1. Most developers know nothing about User Experience (and don’t care).
2. Most developers can’t create usable design.
3. Many developers barely know a problem domain and in any case don’t want to dig into much details here.

In short, most developers are engineers (so far?), so what Kai proposes will not help. It is just a theoretical solution to “how well” problem. It’s the same like trying to grow 4 hands and 2 heads to double your productivity (well, sacrificing beauty maybe…)

In Scrum Product Owner is responsible for writing User Stories. It means he should invent solutions and care about “how well” thing.  I don’t see a significant problem here. Of course, it’s great if developers are interested in requirements handling and solutions investigation process, but if they are not, Product Owner can do that alone or with special people who care. Who they can be?

Those people are UX specialists. They know how to solve “how well” problems effectively. They know many things about design patterns and all new trends in interfaces. They like to explore problem domain and talk to customers. If you have such people among developers — you are SO lucky! But often this is not the case.

We, at TargetProcess, try to instill UX culture into development process. We started in August, now it’s February and I can’t say we’ve had much success. Sure there are significant improvements, but to be honest most developers are not so interested in UX staff, they like BDD, DDD, C# and SOA more. I expect it will be a long process that will take years. And our company is quite small. I can’t even imagine an effort for such a radical shift in a huge company.

Broad Requirements Are Not Testable

I wonder how you can write BDD scenarios for broad requirements as Kai suggests. How the hell can you test anything, if you don’t have any clue on how it should work? So broad requirements can’t replace user stories obviously. They can live in backlog, but developers should deal with concrete requirements that are testable right away.

I can agree that you can split backlog in 2 parts. The first small part contains detailed stories that can be put to development asap. The second large part contains broad requirements without solutions. That might be a good idea. But we should clearly separate UX phase and Development phase. During UX phase we should solve “how well/how cool” problems, and during development phase we should solve technical problems only (with some corrections in “how well” for sure if required).

P.S. Kai, I hate gray text on gray background in your blog. It is not user friendly.

Recently I’ve come across a post on harmfulness of analogies with martial arts. Indeed, there’s a handful of posts comparing software development, or agile adoption, or product development  with  martial arts  - Aikido, Karate, Judo etc.

If you make a direct analogy of software product development and mastering some martial art, it would not be exactly accurate. I guess  people revert to the analogies as they try to project their copies as romantic martial arts disciples into their usual lives as software developers, managers etc.  Perhaps, they’re making the image of their lives more mission-filled this way since there’s not too much space for showing primeval qualities of warrior in software development. But the need to exercise this attitude is still there, as it turns out.

We don’t have beasts to fight, bloody battles and mortal combats. This wrap-up for courage, strong will, persistence in achieving goals and readiness to fight has remained in the past largely. With no wrap-up, people forget that there’s lots of space to exercise these qualities in their usual life.

Let’s go back to analogies. Roger Federer is an unbeatable specimen of mastered elegant performance to me. Look at the way he plays. No waste. He knows, what to do, he knows when to do what. It’s a perfect flow,  the model for effective lean production with no waste and -  the model for perfect warrior  in our modern life. Elegant, no blood on his hands, but he fights, has pitfalls on the way,  stands up, recovers, marches on and wins gracefully.

But he’s just a guy in red T-Shirt. As many software developers

Another point is that it’s much harder to fight, win and achieve goals when there’s no immediate physical danger involved as in tennis.  Soo much room for elegant warriors, isn’t it ?

The point I’m trying to make is - let people compare their lives and their jobs to whatever they want. If it inspires and motivates them, makes them feel good about their lives and their work - and makes them feel like warriors, achievers, believers, fighters, winners.

P.S.  February 23 is a perfect day for such a blog post Wish you well, guys!

I read Agile development grows UP article written by Mark Lines. It has quite a strange taste… Is it an attempt to advertise new methodology? I hope it is, otherwise Mark is completely missing the point and agile development trends.

I want to review the article. Just can’t resist, sorry.

Many of us don’t have the benefit of co-location, where the entire team works in close proximity (ideally in the same room), with the user representative present at all times to answer questions

Sure we all know that a co-located team is better. It is not a surprise that consultants advise to co-locate team members. If you need to win the race, it is better to drive BMW M3 than Ford Focus. There are so many discussions around distributed agile this year. There are so many ideas and experience reports on how to implement agile remotely. Agile works in remote teams.

Pair programming i.e. 2 programmers sharing 1 keyboard is an expenditure most management will not endorse

It’s a very strange argument. Many managers will not support continuous integration, iterative development and BDD. It does not mean that these practices are bad. Context is everything. In some companies and on some projects pair programming will increase productivity. While on other projects there will be no benefits or even degradation. How many agile coaches insist on pair programming and say “You are not agile without PP”? Not many. It is wrong to spread this argument to the whole agile community.

Delivering a system without moderately detailed requirements (beyond User Stories) is not acceptable for testing, writing training materials, and production support purposes.

C’mon! User stories can be VERY detailed. For example, you may use BDD-style user stories format and write many cases that will describe functionality in some format, that can be directly converted to unit tests! It is an incredible thing, to be honest.

Fixing architecture as you go (i.e. refactoring) without some initial architectural design is not appropriate for large-scale enterprise systems

Most people in agile community believe that it is required to spend some time on architecture. Again, it is so context dependent. If you create a simple application, one day may be enough to nail down all important decisions. If you create a complex system, it may take several months. Common sense is a critical factor here. There is always a danger to over-architect things and working software is the best proof of concept.

Also something called “emergent architecture” is being developed. I like the idea and maybe it is a right direction.

Most projects are not completely independent and cannot be built in isolation. Dependencies and integrations with other systems are required and require some co-ordination.

Agile adoption is spreading rapidly and indeed there are no common/standard approaches to solve this problem. But I believe it will be resolved during agile enterprise adoption (and we already stepping into this phase).

Organizations usually have some governance practices in place (such as PMOs, ISO, CMMI) that necessitate a level of bureaucracy and documentation

Does it all mean that PMO, ISO and CMMI are good and we should give up and follow the rules? Agile community tries to fight bureaucracy and I personally love that. Documented process means NOTHING for success of software development project. In 90% of cases it holds you in the spider’s net and blocks creativity. There is NO software development without creativity. Wake up, Mark.

We live with these standards, but we should fight them.

Deploying software to users every 1 or 2 weeks is usually not practical in most organizations. Just migrating software through development, test, system test, user acceptance, production environments is a major undertaking. Having many projects trying to do this in parallel on a weekly basis simply isn’t feasible.

This argument is so common… and “not practical in most organizations” is a truly masterpiece. Any statistic data? Did Mark try to deploy anything outside the enterprise-scale Java mastodon applications world? Many SaaS services do painless weekly updates. Some services do painless updates on a Daily basis! It is unbelievable, but I like that. Customers like that!

Mark promotes OpenUP methodology (based on Unified Process). Only advantages in the post, no disadvantages or context at all. Folks, I think it is a so long-awaited silver bullet! Finally it is there! Let’s bury Scrum. Let’s bury XP. Let’s burn Kanban and all the other processes that spark agile movement. God bless the OpenUP silver bullet. Amen.

Cumulative Flow diagram is a very good starting point for stop-the-line or retrospective meeting. Here is a real example from TargetProcess development.

You can see in the chart that we had a bottleneck in the beginning of December. It was caused by a quite complex user story. In theory a single user story should not affect cycle time significantly and should not create bottlenecks, but if you violate some rules it might be the case.

The user story was about replacing jQuery javascript framework with ExtJS framework. It took 1 month to implement by 1 developer. During this month we’ve made several releases and all went smooth. Then this user story was verified by testers and all existing acceptance tests were passed. So we’ve made a decision to merge this story to the main code line.

Unfortunately, after the merge quite many bugs were found in the build during smoke testing. These bugs took more than a week to fix, and during this time we were unable to release anything, since the merge was already done and the rollback was quite complex as well.

The lesson learned is to put more effort into testing of user stories that are more complex in nature. This particular story affected many places in the application and usual smoke testing was not enough. So we are going to introduce a new class of service (let’s call it  “technically complex story” so far which would mean more in-depth testing and verification before the merge.

In general, Cumulative Flow chart is a valuable tool to analyze historical data, but for emergency bottleneck identification Kanban Board is even better. If you have limits on Kanban Board, you see problems immediately.

I’m digging into Lean Manufacturing and it’s so interesting to learn manufacturing history and its trends. There are so many parallels between manufacturing and software development. Of course they are not direct, but a curious mind can draw them quite easily. Let’s take Cellular Manufacturing as an example.

Cellular Manufacturing is a workspace organization technique. The main principle is to organize production of a single product in a small dedicated unit (cell) that consists of several people and several workstations. The benefits are surprisingly huge:

“The result is very fast throughput. Communication is easy since every operator is close to the others. This improves quality and coordination. Proximity and a common mission enhance teamwork.”

It is a significant improvement over the functional space configuration, when you have large queues and long transition time.

Now, if we think about software development, it is becoming clear why distributed teams have problems and why functional departments will not survive in the near future.

Cellular manufacturing applied to software development directly leads to cross-functional teams that work on a single project. Cell is a single team that has its own inventory and cross-trained people. Expanding it further, it means that developers should be able to do testers’ job, and vice versa.

From lean perspective, managers job is to setup cells configuration efficiently. It is not required to track individual work, but to track cell work instead. It brings process optimization to a higher level and powers system thinking.

It is really amazing how much we can extract from lean manufacturing and adopt this knowledge to software development…

Books by Edward Tufte are a piece of art. I’ve been savoring them to myself for a while, and now I decided to share some sketches and criticism inspired by Tufte’s high art visual designs.

Commonly, designers represent visual information by scarce means of 2D realm: screen and paper. Our universe is 3D (if not 5D, 6D or whatever more dimensions), but people got used to squeezing  images into 2D flatland.  Even rock paintings of pre-historic humans have their touch of 2-D abstraction and symbolism.

Our universe is not just 3D. It’s dynamic 3D. Paper is static (paper planes are exceptions). That’s another limitation of 2D.

Limitations are great. They motivate designers to find solutions. The more limitations - the harder it is to find a solution. Good designers love difficult tasks, since they view them as great opportunities to put their brains to use. Bad designers do not want to use their brains - they want to use templates.

The image below is a template solution for a weather map. View from above.  Let alone template thinking, the representation of this template is poor.

The appalling hint of white shade is a helpless attempt to compensate inadequate color selection for numbers.  What do you think of blue numbers on blue background? You hate that, to say the least of it. What’s the message of these pseudo-3D grey circles? Are they some grey moons? Or cavities  in the designer’s brain?

Now let’s take a look at the Euronews channel weather map.  One may think that this map represents the effects of global warming and Australia is completely hidden under water now. Also, what do those bold numbers show? Probably the depth of the ocean in this area. In meters. Or in miles? But the area is still lit by sunshine, which instills some hope.

As a contrast, here’s a weather map from a Japanese daily, beautiful in its simplicity.  This is the same Japan as on the first weather map above, only from the ocean perspective.  This map provides 0°C и 10°C  isotherms.  You see fine clouds on this map. The map shows sun movement.  OMG, it shows stratosphere! And it’s nothing more than just a weather map from a daily newspaper  - but created by a good visual designer.

Of course, Japan is well-suited for such a nice graphical representation. But you gotta have guts to catch and use this ocean perspective, instead of helplessly surrendering to boilerplate view-from-above weather maps imposed by paper sheet or screen limitations.

I got this insight on lean and agile techniques in military context when reading  “Ideas Made to Stick” book.  The workflow of the military was described there as an example of how important it is to make messages as concise as possible to accomplish tasks.

The evolution of the military strategy with dates and sources is not a subject of this blog
With no extra details, here’s the core of the point I’m trying to make:

First the Army’s approach was to make sure that every single action is planned down to smallest details. But, as they found out, “enemy bears no plans”. An unexpected move could destroy the whole set-up so  “all king’s horses and all king’s men” could not make the Army function again. Effectively, I mean.

So, they reverted to something very similar to agile technique of creating user stories - Commander’s Intent. Commander’s Intent is “the commander’s stated vision which defines the purpose of an operation, the end state with respect to the relationship among the force, the enemy and the terrain; it must enable subordinates to quickly grasp the successful end state and their part in achieving it”.

Do you see the resemblance? It’s a replica of lean production principles, only in military, not in civic, context. And it’s exactly like the agile principle of engaging people and encouraging their creativity to achieve one common goal.

I will not go into quoting sources on the web on Commanders’ Intent and military doctrines to find more analogies and similarities in military setup and lean/agile methodology.  I just outlined the idea.

There’s a misconception related to this topic: Some people think that agile works all OK for lazy folks. It’s all about freedom, about looseness, no obligations. Not at all. Agile is less conventional, it does not care about being in the office at 9 am sharp, or about wearing a tight business suit. But agile projects do have their Commanders’ Intent and they require genuine responsibility and engagement from the team - the soldiers.

Now, as  the Army follows agile principles, would you need any better proof on the effectiveness of agile and lean methodology?

You Kanbanized your development process and setup a perfect flow. You see the flow and feel its power. You even measure cycle time and want to reduce it. But it is not as easy as expected. Some problems in your development process are hard to find and are revealed on team level. Here I’ll describe an interesting technique that may help you.

The idea is quite simple. Take a single user story and visualize its states transition from start to finish, thus you see the whole flow of a single story. Draw all available states on Y-axis and days on X-axis. As a result, you will have a chart like that (click to enlarge).

It is a real user story from TargetProcess project. It was quite complex and its cycle time was 19 days. Let me provide some comments.

User story has been in Planned state during just 1 day, then developers took it and moved to In Progress state. So far so good. 4 days of development and user story is moved to Coded state, but it sits there for 2 days. Why? Why testers did not take it immediately? Clearly, we found 2 days of delay and the reason of the delay should be investigated, since it increases story cycle time.

Then testers took the user story and tested it for 2 days. Then the  story was moved to In Progress state again and developers worked on it for 2 more days (including Saturday). Why? It is a half of the initial development time. It is a clear sign of a waste, but we need more details. It appeared, that testers found 11 bugs. Half of the bugs were caused by unclear user story specification, other are subject to investigate. We should apply root-cause analysis here to find the real problems.

OK. Then testers have verified the user story for 2 days and moved it to Ready to Merge state. It was merged quite fast (in 1 day), but still there may be a waste to eliminate. Finally, the user story have lived in Merged state for 5 days and was released Jan-29.

In total (and as a minimum!) we discovered about 9 days of waste. It means we can reduce cycle time significantly by eliminating this wait time (waste). If you check several user stories using this technique, I believe you will find comparable results for simple and complex stories.

Here is another example for a quite simple user story. Still we clearly see wait times in our flow! We can reduce cycle time to 2 days instead of 6 by just eliminating waits in the flow. But it is not easy

This chart is a really powerful tool. It is easy to create and you could use it to find waste in the development flow.

Bringing together agile philosophy and software development outsourcing has been one of the hottest topics over the last decade.  Lots of blog posts, discussions in social networks — people try to figure out for themselves if it’s at all possible to align agile methodology with the existing reality of outsourced projects.

How can vendors who practice agile gain new customers? How can customers as newly-converted adepts paste their vision of agile thinking on their existing outsourcing partners?

First of all, let’s look back into when and why actually did it all start with IT outsourcing. Manufacturing outsourcing existed since long ago, but IT offshoring only started at the end of 80’s - beginning of 90’s. Companies jumped at the opportunity to save bucks and use cheap labor not only for producing goods, but for producing software. So, the main point is that from the very beginning outsourcing has all been about saving money. No other notable motivation - just saving money and using cheaper labor.

Next, along comes agile manifesto. People start seeing that the waterfall approach  they’ve been using with their outsourcing vendors is not that good after all.  Fixed price contracts do not guarantee real value (Scott Ambler writes very well on that in this article). Next, the more labor is outsourced to some country, the higher are the costs, so the main point for outsourcing which is cost savings makes no sense any more.

There’re other even deeper-lying consequences. On the one hand, the country which outsources - or businesses in this country not the country itself - they save bucks but lose in the long run as they do not grow their own engineering minds, let alone all the problems that you have working with remote teams - yes, we have all this telecom in place, but nothing ever will replace face-to-face communication. If you often go on business trips to the outsourced destination to talk to your team - again, it’s more costs.  On the other hand, outsourcing makes a dangerous long-term impact on the recipient economies as well.

But the point is not about how bad or good outsourcing is.  Vendors have written an array of blog posts on how to align agile and outsourcing - and that’s natural - they want to keep going, so they do everything to back up their point of view proving that agile goes well with outsourcing. This might work true in some cases.

The companies that outsource on the other hand - they have legacy outsourcing teams. They need to get going with them as well, to stand up to all the funds already invested in their outsourcing center/provider.

So with all this outsourcing in our hands - what do we do?

As an agile outsourcing vendor, you should be ready to invest lots of time and effort to educating your new customers on the value of agile and to building a solid relationship. This might be a very difficult task since some people just don’t want to get educated and prefer to stick to good old fixed price bids, logging and billing for gazillion of change requests, lack of communication and other “joys” of classical outsourced development.

As a company with established outsourcing facility, you’re better off. But perhaps you could be even better off if you started this relationship not with an overseas company, but with a guy next door at least.

Anyway,  you are where you are - so you would need to work with your outsourcing partner to practice the agile approach, since at the end of it all work with agile methodology brings real value, as opposed to counting short-term waterfall pennies and losing long-term gain.