The Skipper & First Mate: A Pattern for Continual Progress

How might a manager orchestrate the activities of developers such that progress is always being made? Software projects are fraught with interruptions, shifting priorities, and unforeseeable personnel events. Forward progress is often thwarted -- frequently in parts of a project, sometimes across whole projects -- when a developer in a critical chain becomes unavailable. Having a wizard developer is a wonderful thing; being in a situation where personal circumstances can lead to flow getting blocked is problematic, perhaps dangerous. In the spirit of Christopher Alexander, some observations of software development for over 20+ years are characterized here as the Skipper and First Mate pattern. It shifts attention to the feature being made and away from the people doing it.

Developing software is an exceptionally demanding intellectual task, one requiring exquisite attention to detail, crafting of architectures, inventing algorithms, and writing primo code (including effective tests). Whenever an individual developer is out of the workflow, a group's software development capability and capacity become yet more constrained. Each developer is unique. The knowledge and skills of each individual matters. The learning curve for taking on any new feature in mid-flight is daunting. The context switching is burdensome. And worst of all, some other important work would have to be shelved to free up someone who will have to drop what's underway and step in to pick up the task. Developers are not fungible, especially not in real time. The astute manager asks how such circumstances can be attenuated such that the work gets done and the critical chain remains unblocked. If addressed early, problematic situations won't spread too far or interrupt other work.

Sadly, management practice regarding software development is often blinded by an industrial-era mindset of now obsolete mass-production premises. Today, workers are no longer fungible. Developers aren't working to tend the machine as parts move through it; quite in contrast, the are creating "the machine" -- a very different thing altogether. Most of today's software managers are working in an environment where the technology, and the marketplace, are continually changing. Time is always of the essence -- every technology has a half-life, as does every market opportunity. Continual forward progress is a necessity of economic survival, a prerequisite of success. Having things delayed often means having things dropped, and the underlying investment becomes squandered. This starkly disappointing example of muda in product development is commonplace.

The cascade of illusion spawned by the mass production mindset is insidious and counter-productive when it comes to creating software. It is insidious in that it leads managers to break down work into "one person" tasks, often reflecting the tacit presumption that employees are fungible, and also separates key related elements (e.g., design and programming) into separate divisions of labor. It is counter-productive in that it doesn't recognize the risks and costs to the organization of reliance on individuals at critical chain nodes and the presumption that nothing will go wrong, and if it does, there's someone else at the ready to step in and get things done. If only it were that simple, but experience teaches us otherwise.

To the mass production mindset, the machine is the focus of human activity -- and humans are pretty much interchangeable in their roles. One person can do one job because should someone falter, another can step in a pick up where the other left off, nearly immediately. The critical chain is the machine, not the worker who is expected to execute known and often repetitive activities in a prescribed manner. With software, the critical chain goes through the human mind as the key factor in production. And that makes all the difference in the cosmos.

Things always go wrong -- people get sick, family members require attention, employees depart for greener pastures, and so on. An organization can only depend on individuals in critical chain processes when individuals are fungible -- one can quickly take over for another with no interruption of the work flow. Moreover, it implies that there's a pool of available developers, ready to step in, who themselves aren't engaged in other priority activities. Software isn't like that. Often, losing a developer, even for a while, means a feature or fix just got delayed or perhaps dropped. To the bean counters, and hopefully the managers, this means the entire investment has just been diluted, if not squandered. When such circumstances occur, expectations need to be reset -- uncomfortable conversations follow that never turn out well. This happens every day in organizations everywhere. There is an easy solution to this dilemma.

Many software managers practice, or have accepted the pressure, to assign single tasks to single individuals, and they leave it at that. Workplace rhetoric echoes with "directly responsible individuals" or of the importance that every task must have an "owner" if there's to be any hope of getting something done. Single individuals, of course, can complete even the most complex software tasks, but is that the best way for manager to ensure the feature gets done? Creating a feature, for instance, has diverse aspects (e.g., designing, coding, testing, etc.) any of which could be further decomposed into absurd detail. In the mass production mindset with its proliferation of organizational roles in the division of labor, it wouldn't be surprising to see each of these aspects handled by a different person in a different department. Role proliferation induces complex work flows, balkanizes knowledge, and creates numerous wait states and boundary transitions: all muda.

Accountability, of course, is a Wonderful Thing. But accountability doesn't go far enough when delivering a feature is the point of the activity . Having "one person responsible" shouldn't imply, as often seems to be the case, that only one individual gets to contribute to, and have responsibility for, getting something done. From a managerial standpoint, this raises a provocative question -- which is more important: having someone to blame or ensuring that the task gets done? To observe most organizations and most managers is to see that blame often trumps progress. Otherwise, managers would staff their projects to first and foremost ensure task completion. Too much conversation about responsibility drives out conversation about work and thereby diminishes the signal-noise level related to effective progress. A sticky web of human relations, violated expectations, and politics follows all of which detract from the objective of completing the work as originally intended.

The solution is actually simple: ensure that at least two people sign up for any feature (in a ranked manner described below). While two people share the tasks one is primarily responsible (a "skipper" so to speak) -- should things go awry, it becomes the other person's responsibility (the "first mate") to ensure the tasks get done. The focus here is on the completion of the feature. The pairing creates a circumstance of ensuring that knowledge is shared and that the work gets done. From a deployment standpoint, this means that at any one time, more people are assigned to fewer features, but those fewer features typically get done faster, with each avoiding the risk of total disruption due to factors arising from an individual's unforeseeable circumstances.

Yes, this means pairing, although not necessarily XP's keen insight of pair-programming. Just "pairing". It is an effective mechanism for enhancing team performance. In the simplest sense, pairing means two people working together -- not reflective of a pas-de-deux or duet, but more akin to a master-journeyman relationship or perhaps a bicinium. It means two people are responsible, one more so than the other, for finishing the work. Staff deployment for an iteration then has both a skipper and a first mate for each feature, aspect, or task being undertaken. With software teams, the skipper on one task also serves as the first mate on another, and vice versa. Given that every iteration has a prioritized activity list to be undertaken by the team (typically longer than can reasonably be completed) developers are deployed as skippers for the highest priority backlog items. Everyone is expected do serve in both roles.

Organizationally, the Skipper and First Mate is a powerfully productive pattern -- features get done at a reliable pace, personal circumstances of individuals don't thwart progress by disrupting a critical chain, and tacit knowledge diffuses throughout the team. The work gets done; progress continues apace.

Cooperation between the skipper and first mate takes many forms -- if the skipper is writing a feature, the first mate can be writing appropriate tests. If the skipper is authoring a design document, the first mate is available in a most timely manner for reading it closely for coherency and accuracy. If the skipper gets the flu, must return to the other side of Planet Earth, or is reorganized out of the workgroup, the first mate is obliged to step in and move the task forward until next iteration. The skipper and first mate allocations are revisited at every iteration planning session. This is a pattern of organizational adaptation to the changing circumstances inherent in having individual developers in a critical chain.

For every feature, developers are brought into on-going conversations with someone else about the design and code. They get a quick response for reviews, and get testing support right away. As a result of the interactions, more insights are generated, more risks are identified, and more solutions are surfaced. The key point here, however, is the feature typically gets done sooner than if an individual had done it alone, but more fundamentally, it's much more likely the feature will get done as a matter of pace in circumstances where the "directly responsible individual" becomes unavailable, distracted or reassigned. The critical chain is not disrupted and positive results are achieved.

The Skipper and First Mate is a pattern reflecting cooperation and readiness; it's about a shared objective and commitment. Someone with appropriate knowledge and skills is at the ready, but not waiting in reserve. No one sits idle, but the impact of any single individual's personal circumstances whether it be family emergency, temporary mind-block, whatever, doesn't thwart progress. The work gets done. In an environment of peer reviews, this pattern brings a shared sense of accomplishment, and respect for each other's contributions. The Organization benefits.

When bringing pairing to a team, the manager is faced with the delicate, but all so real, factor of personalities. Pairing is not a comfortable habitat for lone wolves. The extent that a manager caters to them (consciously or otherwise) does shape the nature of the team's activity and subsequent team performance. Inherent in this is the risk of diluting or squandering the to-date investment, and not meeting stakeholders' expectations.A manager needs to determine whether the work at hand requires a team or can be done by a set of individuals in critical chains. In a research group, for example, critical chains are few; in a product group they are common. Often, reliance on lone wolves brings about circumstances of critical chain disruption (and all the subsequent muda of adaptation and corrective action), although not necessarily intentionally. In an environment of peer reviews, this pattern brings the piercing eye of scrutiny. The Organization learns.

This pattern is offered in the spirit of Alexander's A Pattern Language, especially as brought to software engineering management through Coplien and Harrison's Organizational Patterns of Agile Software Development. Following the pattern definition of hillside.net, we have:

Name: Skipper and First Mate

Problem: An individual in the critical chain for completing a feature may at times becomes unavailable. The feature is necessary for an upcoming release and other teams are depending on it being ready for their own commitments. It is vital that the organization / department / team deliver the work in a timely manner.

Forces: Social pressure, economic understanding, and development habits lead managers to deploy staff in an ineffective manner. The work is on a critical chain for a project or feature needs to get done or is perhaps needed for an upcoming release or other teams are depending on it. Managers who have many responsibilities often believe that unless they assign one person per task they're not being efficient and this will reflect badly on them. These manager succumb to the mass production mindset of assigning one person per task. Problems can occur and the individual on the critical chain becomes unable to deliver in a timely manner. The work on the critical chain falters. Should the manager have placed two people on the task and the roles and expectations are unclear, efficiency is reduced as one person can remain idle and the situation of too much undone work becomes worse and the circumstances repeat themselves.

Solution: Sign-up two individuals, the "skipper" and the "first mate", with a clear order of responsibility and expectations of collaboration such as the skipper writing a feature while the first mate writes tests, the skipper writes a design doc whilst the first mate is at the ready to review it and so on. Should the skipper become unavailable; the first mate steps in to ensure the work is completed.

Resulting Context: The developer team becomes accustomed to working in pairs that are dynamic and often shift iteration by iteration. However, in organizations where the culture emphasizes single individual responsibility and individual accountability is more important than forward progress, a manager may be viewed negatively for implementing such a pattern leading the manager to return a single-task-per-person approach, thus reintroducing critical chain risks of reliance on single individuals and the cycle begins anew.

Rationale: Avoid depending on a single individual on a critical chain for development that is time critical.

Developers are bright folks. Many quickly recognize the personal advantages afforded by this pattern once they experience it. The skipper has someone to draw on, and the savvy ones will find advantage in that, especially for quick design, code reviews (or at least perusals), and testing. The first mate likewise knows that lending a helping had averts getting saddled with the work, and getting recognition for the contributions. Both recognize that influence of peer reviews on performance ratings. And the manager recognizes whether the feature got done. The organization continues progress apace.

The Skipper and First Mate pattern is broadly applicable. I've seen it work from advanced software development teams to university research projects, from the profit-focused corporations to non-profit service organizations. Most of all, people like to get things done, and many people like to work with others. This pattern is a simply way reducing risks along critical chains, ensuring necessary work gets done, and providing a convenient and comfortable opportunity for people to work together.

For more information on patterns, visit: c2.com, The Gang of Four, search Google books, or see pattern-related Amazon listings.