Xanpan

An unfinished book

This book is currently a work in progress. I look forward to your comments: please e-mail them to the review list or directly to me at xanpan@allankelly.net.

Not all of the book is written: I am still debating whether to add chapters on requirements and management, or whether to put these into another volume. Please let me know where you would like this material.

I have also noticed that on some e-readers the images - figures - do not necessarily show up as expected. I’m still trying to understand what is happening and whether this is something I’m doing, LeanPub are doing or the reader software. Again, please let me know if you find problems, so I can try to fix them.

Twitter & other online media

Please Tweet!

Please help Allan Kelly spread the word about this book on Twitter.

Allan’s Twitter handle is @allankellynet, and the hashtag for this book and all things Xanpan is #xanpan.

Find out what others are saying about this book on Twitter with this link:

http://twitter.com/search/#xanpan

Websites

There are two websites to accompany this book. The first is the LeanPub page and the second the author’s own:

http://leanpub.com/xanpan

http://www.xanpan.org

And the author’s own website is:

http://www.allankelly.net

What people say about Xanpan….

“This book is an important contribution to the agile literature. It tackles the challenges that most teams face, head-on, by proposing alternatives that some might consider heretical.” Seb Rose

“Xanpan is a method based on the best from Scrum and Kanban. It reflects Allan Kelly’s very pragmatic approach to agile software development, based on extensive experience. This book will help you avoid waste and turning up the good.” Nicolas Umiastowski

“Allan Kelly’s Xanpan is now required reading for our team” Dom Davis Head of IT, Virgin Wines.

“A well written and insightful reference material, packs a punch throughout!!!” Sunish Chabba

“Finished @allankellynet ‘s book XanPan http://nud.gr/1fNSSxF if you’re doing agile, lean, Kanban, xp a mix of any or just starting read it” Kev McCabe @BigMadKev on Twitter

“Unhappy with the strictness of Scrum he crafted his own method called Xanpan, a combination of Kanban and Extreme Programming (XP). XP focus is on engineering practices to write good code and achieve a sustainable level of quality, while Kanban (or Scrum) covers the organisational aspects. Using the best of both makes Xanpan an interesting new player for organising software projects.” Johnny Graber

About the author

Allan Kelly has held just about every job in IT. London-based, he works for Software Strategy, where he provides training and consultancy in Agile practices. He specialises in working with software product companies, aligning company strategy with products and processes.

He wrote his first program at the age of 12 on a Sinclair ZX81, in Basic. He quickly progressed beyond the ZX81 and spent the mid-80’s programming the BBC Micro in BBC Basic, 6502 Assembler, Pascal and Forth. As well as appearing in several hobbyist magazines of the time, he was a regular on BBC Telesoftware, with programs such as Printer Dump Program (PDP and PDR), Eclipse, Snapshot, Echos, Fonts, FEMCOMS and, with David Halligan, Demon’s Tomb, and EMACS (Envelop Manipulation and Control System, nothing to do with its more famous namesake!).

The low point of this early career came in 1986 when Cambridge Examinations docked a mark from his GCE ‘O’ level Computer Science project for not using the GOTO statement in his code. The high point came five years later when he held an internship at Distributed Information Processing in Guildford, working on the Sharp PC-3000.

He believes his first Agile project was in 1997 - although it might have been 1994. His Agile journey began after the Railtrack Aplan ISO-Waterfall death march in 1996 and reading Jim McCarthy’s Dynamics of Software Development (1995). Since 2000 he has helped numerous companies - particularly in Cornwall - adopt Agile and Lean ideas.

In addition to numerous journal articles and conference presentations, he is the author of Business Patterns for Software Developers (2012) and Changing Software Development: Learning to be Agile (2008), both published by John Wiley & Sons. He is also the originator of Retrospective Dialogue Sheets (www.dialoguesheets.com).

More about Allan at http://www.allankelly.net and on Twitter as @allankellynet.

Business Patterns for Software DevelopersChanging Software Development: Learning to Be Agile

Introduction

Is it possible to write a book by accident? Maybe not, but I certainly seem to have started writing one by accident.

This book is several things: first it captures my personal experience, understanding and reflections on the thing we call ‘Agile’. Second, it describes a type of Agile working - an Agile method if you like - I call Xanpan. Xanpan is itself the result of my experience, understanding and reflections.

The world doesn’t need another Agile method, or indeed any other software development method for that matter. The real point of Xanpan is not that it is a development method. Rather, the significance of Xanpan is that it is a derived method, it is a hybrid method, it is an example of a method that anyone could create.

Xanpan is both a method and a philosophy - my philosophy on how software is, or should be, created, and how Agile works - or should work. While I have every reason to believe it is applicable in fields beside software development I have not had a chance to try it to date.

As the name suggests, it is a cross between Extreme Programming (XP) (Beck 2000) and Kanban (Anderson 2010). It is also an elaboration of the process first described as Blue-White-Red (Kelly 2007), which was itself an implementation of XP ideas. Xanpan also contains ideas, techniques and approaches from other methods and schools of thought, specifically product management.

Like all Agile processes, Xanpan rests on Lean thinking (Womack and Jones 2005; Womack, Jones, and Roos 1991; M. Poppendieck and Poppendieck 2003) for its underlying philosophy and principles. As such it may be described as ‘Agile’ or ‘Lean’.

At first sight Xanpan probably does not look vastly different from XP or Scrum. But by making several tweaks to the iteration the process flow is enhanced and emphasised: work can be carried between iterations, teams are not asked to commit personally and unplanned work is allowed for.

I think the best way to discuss Xanpan is to step through an example and point out where it differs from other methods - specifically Scrum and Kanban. So after a brief look at some principles behind Xanpan, the description moves to examples, interspersed with explanation and notes on practice.

Xanpan originates in software development, and the description here assumes it is used in a software development area. However it is expected that Xanpan, with modifications, may be used in other domains. At the time of writing there are too few examples to support a description of uses outside software.

More importantly, Xanpan demonstrates how ideas from Agile, Lean and elsewhere, brought together with one’s own experiences, can be the basis for a coherent method. Exploring the roots, principles and laws behind Xanpan thinking fulfils two roles: firstly it exposes what I consider to be the logic of software development, and secondly it shows how to build an approach with depth.

Xanpan wasn’t born fully formed, it wasn’t built bottom up from principles; while practice drove development top-down, there is a back story - there are principles which informed the development. Only when - if - this book is complete might Xanpan be considered anything near fully formed. Even then it will continue to grow and evolve.

I have long advocated that teams need to ‘roll their own’ development methods - this was a minor theme in Changing Software Development (Kelly 2008). Look at Scrum, yes steal from XP, certainly learn about Kanban and let it permeate your working. But above all, combine the parts of methods that work for you.

All development methods seem to say ‘tailor this method to your own needs’. The difference between traditional - including SSADM, PRINCE 2, RUP and others - and Agile methods - Scrum, XP, Kanban, etc. - is that the former start large and say ‘Tailor this method down’, while the latter start small and say (if only implicitly) ‘Tailor this method up’.

Indeed, Agile methods have been called ‘barely sufficient’ - they lack a lot of the details and contingency tools present in other methods. This makes them easy to tailor, but also makes tailoring essential.

Xanpan is an example of how I have rolled these ideas together and helped many teams adopt a similar style of working tailored to their own needs. A later chapter discusses the origins of Xanpan in more depth.

Depending on your point of view, Xanpan is a tailored version of Extreme Programming, or a tailored version of Kanban. The context of Xanpan tailoring is:

  • Teams initially adopting Agile development
  • Organisations that have generally stable teams
  • Software development groups in Europe, and in particular the United Kingdom

I hope this book still offers useful content to those outside this context, but I can’t guarantee it!

After saying all that, if changing the world does require another methodology, I am more than happy to offer Xanpan, and would be delighted to hear of more teams adopting Xanpan.

Why write another book about Agile?

There is no shortage of books about Agile available today, and specifically Agile software development. So why write another one?

Three reasons really, three reasons that also happen to define the audience - or is it the three audiences that define the reason?

  • The Experienced: think of this book as a post-Scrum book. The world needed XP and Scrum when they appeared in the late 1990s. Many teams have really benefitted from simply ‘doing Scrum by the book’. But the world has moved on, we have all learned.

This book captures one person’s learning, the author’s, and encourages teams to use their own experience and learning to create post-Scrum methods that work for them.

  • The Newcomers: when asked “What book would you recommend for newcomers?”, I’m never quite sure what to answer. In part this book answers that question - and conveniently gives the author something to give to students attending his introductory Agile training courses.
  • The Author: as all authors discover, writing forces one to clarify one’s ideas, put them in a communicative form and in doing so deepen one’s understanding. When I first started using the term ‘Xanpan’ I didn’t know what I was getting into. The more I used it, the more people wanted to know and the more I realised there was to it. This manuscript allows me to put my thinking into a coherent whole.

Conscience drives the author to apologise for writing for himself. But he does not. Writing a book, even compiling and editing existing material, is a long and lonely process. Business Patterns for Software Developers was to be the last book - at least for a while. Instead the author finds himself at work on another book.

This and subsequent books

At the moment this book is a detailed account of the Xanpan process. In my mind this is but one part of three - should I ever write them. One additional volume will look at requirements and another at the management of software development. Initially these will appear as individual volumes and later as one complete book, possibly.

Astute readers of this book through its various incarnations on LeanPub will notice a change in the title of this book. The early versions which appeared in 2013 were titled:

Xanpan: Personal reflections on Agile & Software Development

This book still summarises my personal reflections, or sense-making if you prefer, on Agile-type software development. However starting with the 2014 editions the title has been updated to:

Xanpan: Team Centric Agile Software Development

This reflects repeated observation from those who have read this book or seen my presentations on Xanpan. It is the team, not the project or product, which is the heart of Xanpan. This marks it out as subtlety different from other methods and I think more accurately reflects my own thinking on Agile development. Should I ever complete the management volume of Xanpan this aspect should be even clearer.

Saying Xanpan is team centric also helps reconcile a conflict within Xanpan. On the one hand Xanpan is a process a team could pick up and use. On the other hand Xanpan aims to say “create your own process, don’t follow someone else’s prescription.”

If the essence of Xanpan is the team then any team which step-up and actively create and agree their own process can be said to be following Xanpan.

Feedback

Although I increasing think this book is finished I continue to think of things I should, or could, change, or add. The nature of LeanPub books raises a question of: when is a book finished?

For the moment I think this book is mostly done. However as the later volumes are written there are bound to be changes to this one.

I would also like to encourage readers to send me their comments about the book and suggestions for improvement. Please send e-mail to xanpan@allankelly.net.

Coupons & freebies

Finally, as I produce new versions of this book and subsequent volumes I may decide to add some discounts and coupons for existing readers. I will place these at the very back of the book. Please check the last few pages of the book.

References

Anderson, D. 2010. Kanban. Blue Hole Press.

Beck, K. 2000. Extreme Programming Explained. Addison-Wesley.

Kelly, A. 2007. “Blue White Red - an example agile process.” ACCU Overload (81).

———. 2008. Changing Software Development: Learning to Become Agile. John Wiley & Sons.

Poppendieck, M., and T. Poppendieck. 2003. Lean Software Development. Agile Software Development Series. Addison-Wesley.

Womack, J. P., D. T. Jones, and D. Roos. 1991. The machine that changed the world. New York: HaperCollins.

Womack, J. P., and D. T. Jones. 2005. Lean Solutions. London: Simon & Schuster.

Prologue

Dear Customer: The Truth About IT Projects

Dear customer,

I think it’s time we in the IT industry came clean about how we charge you, why our bills are sometimes a bit higher than you might expect, and why so many IT projects result in disappointment. The truth is that when we start an IT project, we don’t know how much time and effort it will take to complete. Consequently, we don’t know how much it will cost. This may not be a message you want to hear, particularly since you are absolutely certain you know what you want.

Herein lies another truth, which I’ll try to put as politely as I can. You are, after all, a customer, and, really, I shouldn’t offend you. You know the saying “The customer is always right”? The thing is, you don’t know what you want. You may know in general terms, but the devil is in the detail - and the more detail you try to give us beforehand, the more likely your desires are to change. Each time you give us more detail, you are offering more hostages to fortune.

Software engineering expert Capers Jones believes the things you want (‘requirements’, as we like to call them) change 2% per month on average - thats close to 27% over a year once you compound changes. Personally, I’m surprised that number is so low.

Just to complicate matters, the world is uncertain. Things change, and companies go out of business. Remember Enron? Remember Lehman Brothers? Customer tastes change. Remember Cabbage Patch Kids? Fashion changes, governments change, and competitors do their best to make life hard. So, really, even if you do know absolutely what you want when you first speak to us, it is unlikely that it will stay the same for very long.

I’m afraid to say that there are people in the IT industry who will take advantage of this situation. They will smile and agree with you when you tell them what you want, right up to the point when you sign. From then on, it’s a different story; they know that changes are inevitable, and they plan to make a healthy profit from change requests and late additions at your expense.

While I’m being honest, it is true we sometimes gold-plate things. You might not need a data warehouse for your online retailer on day one. Yes, some of our engineers like to do more than what is needed, and yes, we have a vested interest in getting things added so that we can charge you more.

It is also true that you quite legitimately think of features and functionality you would like after we’ve begun. You naturally assume something is ‘in’ when we assume it is ‘out’. And, in the spirit of openness, can you honestly say that you’ve never tried to put one over on us? (Let’s not even talk about bugs right now: it just complicates everything.)

Frankly, given all this, it is touching that you have so much faith in technology to deliver. But when IT does deliver, does it deliver big. Look what it did for Bill Gates and Larry Page, or Amazon and FedEx. Isn’t it interesting that when the IT industry develops things for itself, we end up with multi-millionaires? When we develop for other people, they end up losing money.

How did we ever talk you into any of this? Well, we package this unsightly mess and try to sell it to you. To do this, we have to hide all this unpleasantness. We start with a ritual called ‘estimation’ - how much time we think the work will take. These ‘estimates’ are little better than guesses. Humans can’t estimate time. We’ve known this since at least the late ’70s, when Kahneman and Tversky described the ‘planning fallacy’ in 1979 and went on to win a Nobel Prize. Basically, humans consistently underestimate how long work will take and are overconfident in their estimates.

To make things worse, we have a bad habit we really should kick. Between estimating the work and doing the work, we usually change the team. The estimate may be made by the IT equivalent of Manchester United or the New York Yankees, but the team that actually does the work is more than likely a rag-tag bunch of coders, analysts and managers who’ve never met before.

Historical data - data about estimates, actuals, costs, etc - can help inform planning, but most companies don’t have their own data. For those that do have data, most of it is worse than useless. In fact, Capers Jones suggests that inaccurate historical data is a major cause of project failure. For example, software engineers rarely get paid overtime, so tracking systems often miss these extra hours. Indeed, some companies prohibit employees from logging more than their official hours in their systems.

So we make this guess (sorry, ‘estimate’) and double it - or we might even triple it. If the new number looks too high, we might reduce it. Once our engineers have finished massaging the number, we give it to the sales folk, who massage it some more. After all, we want you to say “yes” to the biggest sticker price we can get. That might sound awful, but remember: we could have guessed higher in the first place.

Please don’t shoot me: I’m only the messenger.

We don’t know which number is ‘right’, but to make it acceptable to you, we pretend it is certain and we take on the risk. We can only do this if the number is sufficiently padded (and, even then, we go wrong). If the risk pays off, we get a fat profit. If it doesn’t, we don’t get any profit and may take a loss. If it’s really bad, you don’t get anything and we end up in Court or bust.

The alternative is that you take on the risk - and the mess - and do it yourself. Unfortunately, another sad truth is that in-house IT is generally even worse than that provided by specialists. For a software company development is a core competency - such companies live or die by their ability to deliver software, and if they are bad, they cease to trade. Evolution weeds out the poor performers. Corporate IT on the other hand rarely destroys a business - although it may damage profits. Indeed, Capers Jones’ research also suggests specialist providers are generally better than corporate IT departments.

Sales folk might be absent, but the whole estimation process is open to gaming from many other sources and for many other reasons. The bottom line: if you decide to take on the risk, you may actually increase risk.

I know this sounds like a no-win scenario. You could just sit on the fence and wait for Microsoft or Google to solve your problems with a packaged solution, but will your competitors stand still while you do? Will you still be running a business when Google produces a free version?

Beware snake oil salesmen selling off-the-shelf applications. Once people start talking about ‘customisation’ or ‘configuration’, you head down a slippery slope. Configuring a large SAP installation is not a matter of selecting Tools, Options and then ticking a box. Configuring large packages is a major software development activity, no matter what you have been told. The people who undertake the configuration might be called ‘consultants’, but they are really specialist software developers, programmers by another name.

There really isn’t a nice, simple solution to any of this. We can’t solve this problem for you. We need you, but you have to work with us. As the customer, you have to be prepared to work with us, the supplier, again and again in order to reduce the risk. Addressing risks in a timely and cost-effective manner involves business-level decisions and trade-offs. If you aren’t there to help, we can either make the decision for you (adding the risk that you disagree), or spend your time and money to address it.

You need to be prepared to accept and share the risk with us. If you aren’t prepared to take on any risk, we will charge you a lot for all the risk we take on. Sharing the risk has the effect of reducing the risk, because once the risk is shared you, the customer, are motivated to reduce risk. One of the major risks on IT projects is a lack of customer involvement. You can help with that just by staying involved.

Ultimately all risk is your risk: you are the customer, you are paying for the project one way or another. If it fails to deliver value, it is your business that will suffer. When you share risks, when you are involved closely, risks can be addressed immediately rather than being allowed to fester and grow.

Finally, you may have grand ambitious, but we need to work in small chunks. I know this may not sound very sexy, but software creation works best when small. Economies of scale don’t exist. In fact, we have diseconomies of scale, so we need to work in tiny pieces again, again and again. If you are prepared to accept these suggestions, then let’s press ‘reset’ on our relationship and talk some more.

Yours sincerely,

The IT Industry

Originally published in the now defunct Agile Journal, March 2012

1. Xanpan Principles

Boiling Xanpan down to a core results in a few principles that guide thinking, and lead to a number of specific practices. Most of these principles and practices will look familiar to anyone who has worked with Agile and studied Scrum, Kanban or XP. However, Xanpan’s formulation is subtly different from all of these: it mixes these principles and practices differently.

Deeper still, underlying all of these are even deeper principles and laws. Some of these are specific to the software industry - I will elaborate these later - and some are broader still. The key, deepest, principles I refer to as ‘Philosophy’. These make up the “I think therefore I am” bedrock.

1.1 Work in iterations

The iteration, or sprint, is a well-established Agile practice. Working in regular cycles provides a rhythm to work, and also imposes a deadline - actually, a recurring deadline. As I have discussed elsewhere, humans are actually quite good at working to deadlines. Xanpan seeks to harness this effect by having teams work in fixed length iterations.

I like to use the train metaphor. A traditional project is like a train leaving London King’s Cross for Edinburgh Waverley station. Trains are infrequent, about every two hours if memory serves. And to get the cheapest tickets, you need to book in advance. As a result we check the timetable in advance, we book our ticket, and if there is any doubt about who needs to go on the train, or what we need to take, we take it. The risk of leaving something behind is too high.

I arrive at the station in plenty of time. If while I’m waiting a colleague phones and says “Hey, we’re having a beer in the Euston Flyer - want to join us for a quick half?”, then I look at my watch and say “Sorry, my train leaves in 15 minutes”. The risk to my schedule is too high, so I avoid changes. In the extreme, if one of my colleagues is late arriving for the train, I try to hold it back, I argue with the guard, I beg, I hold the door open in the hope my colleague makes the train.

When the train does leave I immediately call Edinburgh to say “I’m on my way”; when the train becomes late, I deny it or hope it will fix itself. Then I think “Well the taxi at the other end will only take 5 minutes, not the 20 I allowed”. Eventually the whole train is late; with other passengers, I argue with the guard to throw some people and packages off the train to make it go faster.

Eventually the train calls at Edinburgh Haymarket, and everyone gets off saying “That’s close enough”.

In contrast, working in iterations is the equivalent of getting the London Underground. It isn’t perfect, it has problems, but generally trains run, and they run regularly. I enter the Underground when I am ready; except for the first and last trains of the day, nobody checks the timetable.

If a train pulls in and it is over-crowded, I can decide: squeeze onto a packed train, or wait two minutes for the next. If I am lucky there is a sign that tells me how many minutes the next one will be. And if while I’m waiting someone calls me and says “We are in the St Pancras Champagne Bar, fancy a drink?” I have an option. I can decide: “Get the next train and get home when I planned, or enjoy a drink and get home 30 or 40 minutes later.” I have options. There will always be another train.

1.2 Team-centric: flow the work to the team

Xanpan is team-centric: the team is the production unit, need goes in, working - even valuable - software comes out. This is the machine, the goose that lays the Golden Egg.

Teams everywhere differ: some are large, some are small, some have testers, some don’t, some have requirements people, some don’t; some are collocated, some are distributed. And some work on one stream of work - call it a ‘project’ if you will - and some work on multiple streams. In fact, most teams of my experience - both as a team member and an observer - have to deal with multiple streams of work.

Multiple streams might be two projects at once, project A and project B. Or it might be working on a new product while maintaining an old one. Or it might be the team working on project A but individuals being pulled some days to work on something old. These streams might come from different sources, different business units. Or they might come from the same source.

Consequently some work can be known about and planned in advance, and some work just appears, unplanned. There seems to be some unspoken law which mandates that the later work appears, the more urgent it is.

Xanpan aims for stable teams that accept both planned and unplanned work on multiple streams.

1.3 Work to improve Flow

Hand in hand with team-centricity is flow. Work arrives at the team from somewhere, somehow. This is inbound flow. Ideally work is flowing into the developer (see Coplien and Harrison 2004 for the Work Flows Inward pattern). Then it has to flow out - perhaps via testers, operations, deployment, and finally to the customer. The actual time needed to do the work may form a very small part of the end-to-end elapsed time.

As with Kanban, Xanpan aims to reduce the overall end-to-end time by improving the flow, making it smoother, more regular, more predictable. Improving flow can also mean levelling flow - reducing the peaks and troughs in work patterns - and constraining work to deliver an overall improvement.

The key to improving flow in Xanpan is allowing work to span more than one iteration. This is heresy in some circles and some texts.

1.4 Quality is free (provided you invest in it)

Philip Crosby wasn’t writing about the software industry, but he could have been (Crosby 1980).

There are those in the software industry who believe there is a dial on the wall marked ‘Quality’. If you turn the dial down, quality falls and work happens faster. Turn it the other way, dial up quality, and work slows down. They are wrong. This might be true in some industries, but not software.

In software development the dial is wired in reverse: if you want to go fast you dial quality up; when you dial it down, you go slow. Capers Jones has devoted most of his career to studying software development quality and metrics, and he is unambiguous:

‘Projects with low defect potentials and high defect remove efficiency also have the shortest schedules, lowest costs, and best customer satisfaction levels.’ (Jones 2008)

To be clear, when I use the word ‘quality’ here I am not talking about features, polish, leather upholstery or just about any other external attribute. Indeed I’m not specifically talking about software design or architecture characteristics - although they are implied.

When I say ‘quality’ I am specifically referring to defects, bugs - or to put it another way: low quality implies rework fixes. High quality work does not need rework.

This is not an excuse to gold-plate and over-engineer systems; work may well need to change as we learn things, but we should not be delivering work which quickly requires fixing. All software needs rework. That is the nature of successful software. The question is, how easy is the rework? Low quality makes rework harder, and therefore slower. High quality makes rework easier, and therefore faster.

In the same book Jones says:

‘IBM was also the first company to discover that software quality and software productivity were directly coupled and that the project with the lowest defect counts by customers were those with the shortest schedules and the highest development productivity rates. This phenomenon, discovered by IBM in the early 1970s and put in the public domain in May 1975, is still not understood by many other software-producing enterprises that tend to think that quality and productivity are separate issues.’ (Jones 2008)

Of course rework occurs not just for defects: requirements changes and unforeseen changes can result in reworking existing code. And ones man’s bug is often another’s change request. I am perfectly happy to have these debates, but only when a team is approaching zero defects. Until then the team needs to work to improve quality and reduce rework.

Quality is important, because when quality is low there needs to be rework, and when there is significant rework other parts of the Agile jigsaw just don’t fit together. High quality is essential to Agile working:

  • Rework destroys flow: work must move backwards, work creates work because work creates rework.
  • The need for rework means stories and tasks can’t truly be considered ‘done’.
  • When stories aren’t truly ‘done’ iterations are an illusion, because hidden work is flowing between the iterations.
  • Metrics are destroyed because work that looks ‘done’ isn’t.
  • Developers, testers, managers and others spend inordinate amounts of time prioritising, reporting, managing and even doing rework rather than delivering value.
  • Organisations spend inordinate amounts of money on testing resources and cycles: the need for test demonstrates that quality has been sacrificed.

As far as I am concerned anyone who thinks reducing quality is a good way to speed up software development shouldn’t be working in the industry.

The need for high quality is why Xanpan embraces all the XP technical practices explicitly. These practices, specifically test-driven development, raise quality, thereby reducing defects and rework. (The software craftsmanship movement continues to advance this cause of quality, and I encourage all to follow the latest thinking here.)

Quality

Any mention of quality, especially high quality, demands a proper discussion of what is quality. Quality will be discussed in more depth in an appendix.

For now when I speak of quality I am essentially speaking of two attributes - qualities if you prefer - of a software product:

  • Defects: quality is inversely proportional to the number of defects seen in a system, i.e. lots of defects imply low quality; few (or no) observed defects do not by themselves imply high quality, but such a status is a precondition for high quality.
  • Maintainability - changeability and extendibility: successful software lives and needs to change over time. If software is not changeable, then it cannot change as it needs to during its lifetime, and it will be hard to remove any defects that are found.

There are other attributes that customers, users and the development team might like to include when talking about quality for a given product. Indeed I would encourage all teams to think about what constitutes quality for their product. However I believe that these two attributes of quality apply universally for any software product.

That said, this definition of quality is not a reason to over-engineer, over-design and gold-plate software. The Xanpan approach is to achieve these attributes, not through ‘big up-front design’, but through repeated design sessions, ‘little and often’, constant attention to technical excellence and delivery of working products in the short term.

Such an approach leads to a form of rework, but this is rework because things have changed, knowledge has improved, requirements have changed, new demands have emerged. This is not rework because fault has been found with existing work.

1.5 Visualise

There is surely no team sport in which every player on the field is not accurately aware of the score at any and every moment of play. Yet in software development it is not uncommon to find team members who do not know the next deadline, or what their colleagues are doing. Nor is it uncommon to find managers who have no insight into the work coming their way, or indeed what happens to work when it leaves them.

There are multiple ways in which teams can visualise their work: whiteboards, flip-charts, burn-down charts, cumulative flow diagrams, stickies, posters and many many more.

Software development is an extremely abstract activity. We do things that cause sub-atomic particles to move about in a machine. We work in obscure languages that might look mathematical to some, but look decidedly like natural language to pure mathematicians.

Visualising work helps people learn, it helps people improve flow; visualisation makes it difficult for problems to hide; it creates shared understanding and shared learning. If we cannot see, we cannot learn. If we cannot learn, we are doomed to failure.

1.6 References

Coplien, J. O., and N. B. Harrison. 2004. Organizational Patterns of Agile Software Development. Upper Saddle River, NJ: Pearson Prentice Hall.

Crosby, P. B. 1980. Quality is free : the art of making quality certain. New American Library.

Jones, C. 2008. Applied Software Measurement. McGraw Hill.

2. Board 1

In Xanpan all work is represented on a physical board, usually a magnetic white board, sometimes called a ‘Kanban board’. You might decide to use an electronic equivalent, but I always strongly advise teams to start with a physical one until they are familiar working in this way. Working physically is magical - the learning experience is so much stronger and faster. Even if an electronic system is adopted, the team should keep the electronic board publicly visible at all times.

There are many different board designs. Indeed, each team is encouraged to design their own board. Think of this like the light sabres used by Jedi Knights in the ‘Star Wars’ films: every Jedi must build their own light sabre, and every Xanpan team must design their own board.

Figure 1 - Basic board design
Figure 1 - Basic board design

Figure 1 shows perhaps the most basic board design, and will probably look familiar to many readers. This board is at a laboratory equipment maker in Falmouth (UK). The small - five person - development team creates desktop and embedded software used in products for IVF treatment. In addition they create some embedded software used in the company’s own product line facilities.

On the left of the board is the work to be done (the pending column), in the centre is the work in progress (current on this board, commonly known as ‘work in progress’ or WIP in Kanban), while on the right is the work completed. This photo was taken the day after the team reset the board in the planning meeting, so it is quite clear. Over the course of a sprint, cards move from left to right. The objective is to get as many cards as possible from the left-hand side to the right.

At the bottom left is the ‘product backlog’. As with Scrum, the product backlog contains all the work that might be done. In this case the product backlog is physical, and is attached to the board. Personally I like working with a physical backlog, but even I accept it has its limitations. Rows in an Excel spreadsheet can be used, while a GoogleDocs spreadsheet is even better for sharing. Plenty of electronic Agile and ‘requirements management’ tools are available, but there is value in simplicity.

From the product backlog a subset is selected to do in each sprint. In keeping with Scrum terminology, this is the ‘sprint backlog’, but may equally be called the ‘iteration backlog’. This team has two product backlogs for two ‘projects’: one embedded, one desktop. In the right-hand corner are graphs showing status. There are actually three burn-down charts here, so the team has three projects in flight. (The third backlog is not visible here for other reasons.)

Each sprint/iteration is two weeks long. They run back-to-back: the end of one sprint is the start of the next. Each sprint begins with an iteration planning meeting and ends with the next such meeting.

In each planning meeting the board is cleared of work done: cards counted, graphs updated, remaining work reviewed and removed if no longer needed, or left if still important. Then the team refill the left-hand side of the board to slightly more than its expected capacity.

This board also shows a blocked column. Whenever the team want to work on a card but cannot - they are blocked - the card is put in the blocked column. This is a signal that escalation may be needed. Or, as in this case, the team is waiting on the customer for additional information before work can continue.

The colour coding of cards is significant. While teams are free to follow their own colour convention, most follow the one laid out in ‘blue-white-red’:

  • Blue cards are development stories - often in user story format, but not always. Blue cards mean something to the business; each card should represent some item of functionality that is valuable to the business/customers.
  • White cards are tasks - developer tasks are the most common, but there may be test tasks, analysis tasks or any other type.
  • Red cards are bugs - if a problem is found in work done on a white or blue card before the end of the sprint, the card simply moves back to pending or current. If a bug escapes the sprint and a formal bug report is raised, it re-enters the board as a red card.
  • Yellow cards are unplanned work. Xanpan allows for both planned and unplanned work. More about these cards in a moment. Bugs requiring an urgent fix may well be unplanned work and should be on the board. Arguably they could be yellow, but red is more commonly used.

Xanpan teams generally follow the Scrum/XP iteration cycle: work is planned in a ‘planning meeting’ and the team accepts a reasonable amount of work. Unlike Scrum, the team is not asked to commit to performing all the work, neither is the cycle locked; work may continue to enter the sprint.

(The words ‘sprint’ and ‘iteration’ are taken to mean exactly the same thing, namely a fixed period of development activity. Scrum introduced the word ‘sprint’ for this, and Extreme Programming, ‘iteration’. In the Xanpan context there is no difference.)

Ideally the team would be able to plan all work up front: no work would be added or removed. However in many environments this is not possible. Bug reports - reds - are one way work may enter. In the case of this team yellows are largely IT support tasks the group needs to look at.

In this picture three IT support yellows are in pending. They have occurred since the start of the sprint, or perhaps they occurred in the last sprint and have been carried over into this one. (Also, unlike Scrum, work can be carried from sprint to sprint: more of this later.) One yellow is in the pending column: someone is actively working on this issue.

While every effort is made to limit work in progress - a default of one task per person - such limits are sometimes broken, and team members need to suspend one task to deal with something more urgent.

Close examination of this picture shows that the yellow in the pending column is on top of a white. One of the developers was working on the white, but has suspended work to deal with an urgent yellow. When this is complete the yellow will be moved to the ‘finished’ column and work will restart on the white.

Unlike whites, the yellows have no effort estimates. Unlike Kanban, tasks - and perhaps stories - are estimated, most probably using the standard planning poker technique. Yellows are retrospectively estimated by the person who completes the tasks when they move to the finished column.

Red, bugs, may or may not be estimated. They may be estimated in advance - as part of the planning meeting - or retrospectively by the person who undertakes the work. Whether and how they are estimated depends on the team’s approach to bugs. The aim is to have no reds - no bugs.

On this board the team has chosen to use Avatars to indicate who is working on which card. Many teams don’t bother indicating who is working on what - they remember or don’t need to know. Other teams use colour-coded magnets (usually the colour of the magnet is meaningless); one team used coloured star stickers, and another coloured shapes.

Regardless of how or whether the team indicates who is working on what, it is best to avoid allocating work until the last possible moment. This moment is when someone becomes available to work on a task. Ideally the tasks to be done are prioritised. The next time someone finishes a task and becomes available, they simply take the highest-priority task and work on it.

This is the ideal, and it is easy to imagine why this doesn’t always happen: someone may lack the skills to work on the next highest task, someone may be working on a related task and it would conflict for two to work on the same thing, and many more reasons.

Allocating tasks as late as possible ensures that priorities are worked on, and reduces bottlenecks around individuals. For this to work, team members need to have similar skill sets.

Most teams - and certainly the team illustrated here - frequently fail to work in strict priority order: however, it should be an aspiration for every team. The first step in this aspiration is to avoid deciding in the planning meeting who will work on each story or task. Instead, wait until at least the daily stand-up meeting, when it becomes clear which stories and tasks will be worked on during the day.

Finally, on this board no attempt is made to indicate how much effort is remaining on a card. The cards have their initial estimates, which are not reduced. It is possible to get an idea of how done a blue is by looking at how many of the associated whites are done, but even this is not very useful. Cards are either done or not done: in programming it is usually impossible to accurately tell how much work is remaining, you don’t know when you are about to hit a problem. Therefore we only consider ‘done’ and ‘not-done’.

Similarly there is no attempt to capture ‘actual effort’ or measure the difference between ‘actuals’ and estimates. For reasons discussed elsewhere, actuals are only retrospective estimates. When a yellow or red is undertaken without an initial estimate, then a retrospective estimate is put on the card; this is not perfect, but is good enough.

Figure 2 - Basic board five months later
Figure 2 - Basic board five months later

Figure 2 shows the same board five months later. There are fewer yellows in play because the team has hired an IT support engineer to handle most of these tasks. A few support issues or other unplanned work still persist. On the whole support tasks are managed on another board dedicated to the work of the support engineer.

The team has expanded in this picture by joining with a related team that has more engineers and projects (five backlogs now). The merging of the two teams was not entirely successful and was later reversed. The team has added another colour card to the convention: green cards represent process-improvement tasks, perhaps originating in a retrospective, or perhaps from general conversations.

The board has also acquired a new ‘review’ column. When a card is complete it is first moved here before progressing to the finished column. This serves two purposes: first, to demonstrate what items were completed yesterday when the stand-up meeting is held. Second, it allows the developer to ask for someone else to review the work before it is moved to the finished column, although this is not always necessary.

Key points:

  • Cards are colour coded: blue, white, red, yellow and green.
  • Iterations/sprints are two weeks in length, and start and end with a planning meeting.
  • Xanpan iterations contain both planned and unplanned work.
  • The board represent the state of the team and their work, not the state of a particular project.
  • The product backlog contains all the work that has been requested for a particular product.
  • The sprint backlog contains a subset of the product backlog that is currently in play.
  • Xanpan is team-centric, so the team may be working on more than one product or project at a time.
  • Estimates are made for work planned in the planning meeting; unplanned work is estimated in retrospect.