Lean Process and Kanban for Web Applications Development

Why Lean

These days, creating a dynamic website or a web application is a full-blown and non-trivial team-based endeavor that requires software development best practices to succeed. It's all about reusable tools and processes, tailored and in evolution within your organization, with the twin aims of finally defeating waterfall and waste, and guaranteeing delivered value. Whether the team is in-house or contracted for the duration of the project, the only deliverable that matters is the expected value to be provided: every project is a product, and we learn not only from agile and scrum, but from the power and pain found in the world of startups.

How can we acquire these tools and processes as part of a program of continuous improvement for our organization and our projects? Let's go over the following areas:

• Market, Problem and Product: Delivering Value is the essence of Lean
• Agile is the new Waterfall
• Kanban (not Scrum)

Market, Problem and Product: Delivering Value is The Essence of Lean

At the very heart of the Lean Process is the premise that we deliver real value, not just software. The fact that an application runs flawlessly, passing all unit, integration and end to end tests does not mean it is finished. There must be a period of maintenance iterations used to adjust or even pivot the product until it demonstrably provides value in the context of problem, market and product. It is important to ensure this is understood right from the start.

For the product (project) is itself simply the minimum viable platform for testing the hypotheses governing product market fit in the solution of a well-defined problem.

"A market is the group of people you think might want to buy your product.... A problem is the reason that those people are going to use your product... A product is simply the way that you’re going to solve the user’s problem. It’s the end result of what you’re building. It’s the thing that people, presumably in the target market, are going to pay you money for." (from Laura Klein, UX for Lean Startups, Chapter 1)

Validating the Market

"You’ll know that you’ve successfully validated your market when you can accurately predict that a particular type of person will have a specific problem and that the problem will be severe enough that that person is interested in purchasing a solution."

Validating the Problem

"You are going to discover a problem that exists within your target market that you are capable of solving. Remember, if there’s no problem, then there is no compelling reason for people to purchase your product.... You’ll know that you’ve validated a problem when you start to hear particular groups of people complaining about something specific."

The best way of expressing the problem is via a list of complaints, or pain points.

Validating the Product

"Just because you have discovered a real problem and have a group of people willing to pay you to solve their problem, that doesn’t necessarily mean that your product is the right solution.... You’ll know that you’ve validated your product when a large percentage of your target market offers to pay you money to solve their problem."

The list of pain points are elaborated in the form of a set of hypotheses, sub-hypotheses and sets of features that can be based on the following template:

We believe that
[doing this/building this feature/creating this experience]
for [these people/personas]
will achieve [this outcome].
We will know this is true when we see
[this market feedback, quantitative measure, or qualitative insight].

(taken from Gothelf, Jeff (2013-02-22). Lean UX: Applying Lean Principles to Improve User Experience

Agile is The New Waterfall

Agile, with its incremental and iterative approach to software development, is the process everyone says they are using, but Waterfall is the process that is most often used, by default. In the case of website and web app development, this only gets worse, as bad process leads to the creation of isolated discipline silos where deliverables are implemented and then handed off to others without there being anywhere near enough cross-collaboration or feedback. Agile is an improvement over Waterfall since the abyss separating requirements gathering from testing is shorter and since the iteration mini-cycles lend themselves to more testing and feedback taking place.

But as Desirée Sy's 2007 article Adapting Usability Investigations for Agile User-centered Design explains, Agile requires special forms of adaptation in order to really improve upon Waterfall in the case of Website and Web App software development projects.

A Better Agile Process for Web App Development

In the article Sy and Miller go on to describe in detail their idea of productive integration of Agile and user-centered design using a technique called Cycle 0 (sometimes called Sprint 0 or Staggered Sprints), in which design activity takes place one sprint ahead of development. Work is designed and validated during the “design sprint” and then passed off into the development stream to be implemented during the development sprint.

However, even though Sy and Miller always advocated strong collaboration between designers and developers, many teams still communicate by handoff, creating a kind of mini-waterfall process. But even when cross collaboration is encouraged, with staggered sprints it becomes very easy for the entire team to never be working on the same thing at the same time. You never realize the benefits of cross-functional collaboration because the different disciplines are most often focused on different things. Also, when two-week sprints are used in a Scrum context, designers do not get enough creative time.

An Even Better Process: Design + Dev Together

Gothelf takes into account Sy and Miller's findings and proposes a better process solution:

Many teams have misinterpreted this model. Sy and Miller always advocated strong collaboration between designers and developers during both the design and development sprints. Many teams have missed this critical point and have instead created workflows in which designers and developers communicate by handoff, creating a kind of mini-waterfall process.

From Gothelf, Jeff (2013-02-22). Lean UX: Applying Lean Principles to Improve User Experience (p. 97). O'Reilly Media. Kindle Edition

As a result it's necessary to go beyond staggered sprints.

First of all, themes (top level features) allow for several scrums to be united under the same umbrella, giving time for greater creativity on the part of graphic design and user experience team members, and "the entire team is working on the same thing at the same time".

Each theme, and indeed each sprint, is kicked off with a series of brainstorming and validation exercises with the participation of the whole team (light green). The resulting ideas and sketches are taken to the traditional sprint planning meeting, where again the whole team participates in a process of writing user stories, and then evaluating and prioritizing them to form the sprint backlog (purple).

Then, weekly user validation sessions are added in (blue-green), so that the ideas being implemented can be subject to testing for usability and value.

See also Beyond Staggered Sprints: How TheLadders.com Integrated UX into Agile.

The Best Process: Design + Dev + DevOps Together

The best process we know, and one that finally emancipates Agile, and us, from Waterfall, adds DevOps into the mix. Just as it is critical for cross-collaboration between design and development, so it is just as critical for there to be cross-collaboration between design, development and DevOps, both in provisioning so that the team can really work in parallel, as well as in building, feedback conversations, and testing. Enabling continuous integration and continuous delivery.

We can go a step further, optionally, adopting Kanban as a replacement for Scrum in our Agile approach.

Kanban (not Scrum)

The initial core team meets early on to plan out on the Kanban board the actual jobs of work needed so that the team can build the prototype build (MVP 1) as quickly and directly as possible and have everything ready for the Team Kickoff.

An Empty Kanban Board

We start with an empty Kanban board. There are four columns:

• New
• To do
• In Progress
• MVP 1

You can use the free online service Trello to create the board (Figure 1-1).

So why Kanban? And what is Kanban, exactly? Well, the original Kanban is a lean manufacturing process. It comes from the world of the nuts and bolts production chain, and is a system of logistical control. "Kanban was developed by Taiichi Ohno, at Toyota, as a system to improve and maintain a high level of production. Kanban is one method to achieve JIT". (JIT is the Just in Time production strategy).

Now, just as this production strategy was put into place with great discipline and sacrifice by Toyota and other companies in order to survive the booms and bubbles as well as the bursts (Lost Decade) inherent in the world economy, Kanban for software developers (and all knowledge workers) is a wonderful way to optimize energies and resources of all kinds through a just in time delivery flow in our own neck of the woods.

Above and beyond which framework or stack we might be using, one of the principal objectives here is to show how we can move from Kanban, Agile and Lean as mere buzzwords to success through actual best practices.

Listing and Prioritizing Jobs of Work

In line with keeping it simple, we go round robin around the team and stick no more than 20 items into the New column. We then remove what is not absolutely needed for the first MVP (for example, which hosting or cloud platform should be used for deployment, and the site map and menu navigation, all of which will definitely be subject to change after the kickoff session, so no sense in including throwaway jobs).

After some discussion, the Kanban board looks like this:

Each job of work occupying a card on the wall corresponds to a user story. Each value hypothesis is broken down into a series of sub-hypotheses, corresponding to features taking the form:

And each feature will be implemented by one or more user stories, each of which will appear as a job of work, a card on the Kanban. Each user story can be based on the following template Influenced by A framework for modern User Stories by @jonatisokon:

Classic format

As a [persona, user] I want to [perform action] so that [achievable goal].

User Validation Test Addendum

Given that I'm [performing action] ... 
When [point of observation]
Then [observable result]

The main thing to understand about Kanban is that it is not based on the pre-estimated "sprints" designed to deliver a complete mini-release one finds when using Scrum. Rather it is a continuous flow process. Core jobs of work seeking a particular outcome are implemented one at a time in cross-collaboration, and their associated cards pass through various states (represented by columns) in the process. At the start all issues are placed in the New column (state). Then these are prioritized and a number of them are placed in the limited backlog "To Do" column.

Later, when team members work on a job, it is placed in the "In Progress" or "Doing" column. This column is also limited in the number of issues it can contain. The reason for these constraints is key to why Kanban is different: it champions focus and prohibits people working on too many things at the same time. This is called a Work in Progress (WIP) limit, and it replaces the constraint of the sprint cycle length and time frame restrictions found in Scrum. In this way, Kanban frees us from waterfall-like mini-release "estimates". So while there are no sprints, only a continuous flow of jobs of work being delivered for outcome value testing, the WIP limit, seen in parenthesis in the To Do and In Progress column titles, seeks to optimize productivity without overload. The actual number is calculated by taking into account team size and velocity, and is adjusted for maximum productivity without overloading the team as the project matures. We limit the To Do column and the Doing column to 4 (number of separate disciplines being worked on with semi-paired programming).

Another benefit is that since the focus is on jobs of work and not abstract mini-releases, it is much easier to foster multi-discipline cross-collaboration.

As each card is placed in the In Progress column (two at any one time given the size of the team and the need for paired development and cross-collaboration on the job), conversation takes place among the team members assigned to the job, which is finally confirmed upon acceptance and moved to the MVP 1 column. And the Team Project Kickoff will make use of the MVP 1 (prototype) in order to carry out value proposition hypotheses creation.

That's Lean Process and Kanban for Web Application Development!

Bibliography

• Agile Wikipedia article
• Waterfall model Wikipedia article
• Desirée Sy's 2007 article Adapting Usability Investigations for Agile User-centered Design
• Book Gothelf, Jeff (2013-02-22). Lean UX: Applying Lean Principles to Improve User Experience
• Article Beyond Staggered Sprints: How TheLadders.com Integrated UX into Agile
• Book Klien, Laura (2013-05-16). UX for Lean Startups
• Article A framework for modern User Stories by @jonatisokon