Back when I was on active duty in the U.S. Navy assigned to a systems command, I was assigned as project manager in the development of a methodology to incorporate technical performance within the realm of overall project performance measurement and control. I sat down before my boss, who was a brilliant engineer, and he asked me this question: “What is the value of failure?” Then, after a bit of discussion, he said: “That is your job to determine.”
Unlike with many of my contemporaries, failure was never really an issue for me, and I mean that with all honesty. Despite many notable successes that marked my career, I had also known failure in many of my pursuits and had learned how to overcome it. After a while I employed failure as a means of pushing the envelope, of exploring new ways of looking at things and approaching challenges, especially when the standard approaches seemed ineffective.
Needless to say, in returning to my project management assignment, my team and I failed at first in our mission. Senior officers and civil service personnel who were assigned as my superiors had to have a high tolerance for accepting interim failure as a means to success. But then we learned from those failures and—after a great deal of disagreement, healthy conflict, and applied learning—managed to come out with the first version of a workable approach. I’ve since revised and refined that approach, as have others who adopted it, but the fact of the matter is that to me failure is a means to something else.
So how does this work?
The first step is to learn to identify the difference between existential and transient failure. A rational person does not take uncalculated risks, especially if they can lead to death or complete failure of the organization or task. In the context in which I am addressing it then, failure is defined as “a process of learning in which intermediate steps accept setbacks as a means of achieving an ultimate goal through calculated risk.”
But there are barriers to this approach. Our society inculcates a cult of inerrancy—a belief in perfection—to the point where even the hint of setback or failure is seen as not only an individual character flaw, but also some kind of moral failure. These morality plays are destructive of calculated risk taking and, ultimately, innovation.
This standard is also unrealistic. People are not perfect. Anyone who thinks otherwise is either deluded or a narcissistic sociopath. In either case, it is a way of thinking to be avoided.
Learning from the Past
In mentoring junior officers and young professionals, I have encouraged them, in addition to developing their technical skills, to pursue a good liberal arts education. The reason for this is my strong belief, based on the documented results of a long line of educators and my direct observation, that a wide-ranging education is the best means for an individual to both realize their talents and avoid the pitfalls of solipsism and narcissism. By “wide-ranging” I am including all of the means of learning: direct experience, the experience of others, and formal education.
A lot of history has passed over the course of human civilization. As relatively short-lived beings we tend to place a great deal of emphasis on contemporary events, giving them an outsized importance in the overall scheme of things, failing to place them in their proper context. This distortion of perspective, while normal, can lead us to ruin or at least to counterproductive behaviors and pursuits.
Attempting to wrap one’s mind around any given concept of domain knowledge in the world is a humbling task in itself, especially for those of us in the information technology discipline. John von Neumann was wrong in one significant way in this regard: The mind is not a computer. Thus, we need the advantages of context and perspective to understand our world and avoid repeat mistakes.
There are numerous analogues that can be used. In understanding failure, the most effective one for me has been in looking at the life of Ulysses S. Grant and the conduct of the Civil War under Lincoln, especially after the president appointed Grant head of the Union armies. If you are so inclined, you may read about this example and my thoughts on it here at AITS.
Failure as Risk
For Grant, then as now tactical failure and setback were used to identify, assess, and address risk within the context of the strategic and operational plan, though they did not have the tools to view things in that way in 1864. Failure then, at the tactical level when applied to the more mundane tasks of project management, is really risk management. This goes beyond just an assessment of character in circumstances of adversity, though character does play a role.
This concept is best reflected in recalling a conversation in which I participated when I was on active duty. I was assigned as a business manager within the program management organization of a major systems command. During a program performance review we were looking at the normal summarized stoplight charts that define our discipline. Most of the stoplight chart for a particular program for both cost and schedule performance were marked yellow.
One of the deputy project managers looked at the yellow and asked: “How do we make this green?” After a good bit of back-and-forth on the validity of technical, cost, and schedule performance measurement the deputy clarified his statement. “No, what I’m asking is whether we can just make this green given that we’ve done everything that can be done.”
“I think you’re missing the point,” I responded. “If you want it green, let’s get a marker and mark it green, but it won’t solve the underlying issues and won’t reflect reality. Green is not the point.” My remark was met with mixed reaction, but I had discussed this concept at length with my boss, who was the senior manager of the organization. It was a concept that he had wanted to introduce, and he encouraged me to elaborate on my point.
“The emphasis on being ‘green’ is wrong. We should always be in the yellow. If you are pushing the envelope on technology you will always experience some failure and setback. It is important to record and learn from that failure, as we do in the fleet during training. If we paper this over with re-planning we will never understand how we got to where we are, and how we get to where want to go. This is also important for when we do this the next time. When you are in the yellow you know that you are managing risk. Anyone can do the easy stuff. That’s the effort that’s always green.”
This was a new perspective at the time because it dealt with project management data as a management indicator that directly reflected risk manifestation, rather than as a means of reporting success and failure. The stoplight chart was communicating a subliminal message that one’s success and future prospects for promotion hinged on “being green.” We decided to turn this perception on its head. Green was for slackers. If your project matters and the team is managing risk, then you should shoot for yellow.
This new paradigm became institutionalized but, alas, changed with new leadership, though the concept does persist in many places. The transition of staying in the yellow led to success and encouraged the deployment of new concepts and strategies for identifying and handling risk.
When we look at the development of project management risk, what we see is the assessment and handling of levels of failure so that we stay within acceptable tolerances that are indicative that the enterprise before us should be continued. Ad hoc risk without an assessment and realization of the consequences is reckless. Measured risk is good practice.
Failure as Learning
Though this is not a new idea, business has begun to incorporate the acceptance of failure in its literature. The book Little Bets, by Peter Sims, has been very influential of late, and Harvard Business Review dedicated an entire issue back in 2011 to this topic.
However, what I find most interesting is how these exceptions to the rule prove the rule that the literature of failure is replete with self-denial. Back in 2013 Business Insider published a survey that indicated that, while 50 to 70 percent of businesses fail within the first 18 months, 86% of entrepreneurs and business owners stated that they felt they could achieve anything they wanted to do (the follow-your-dreams meme), and 89% described themselves as leaders. Notably, 77% said they would rather learn from failure than not try their hand at business. It would be interesting to see if the ones who were in failing enterprises still held these opinions after their business failed.
But to return to the literature in both Sims’s book and the HBR articles, failure can be incorporated into our modus operandi as a learning tool and is intrinsic to it. This should not be a surprise given that formal education necessary to possess the basic foundation of knowledge takes twelve years of primary and secondary education, followed by numerous additional years for undergraduate and graduate education.
During each of these stages of education, trial and error is a necessary component of pedagogy. Given the emphasis on GPA and competition for “good” schools, many parents within our society have displayed both little acknowledgement or psychological maturity in accepting this natural process. Such pressure is usually counterproductive, serving as an impediment to learning and psychological health. This practice often translates to unrealistic expectations beyond education.
To use the example of refresher training onboard U.S. Navy ships, each member of the crew is designated a specialty: from supply specialists, to boatswain’s mates, to electronics and electricians specialists, to hull, mechanical, and other physical ratings. Their rank is a reflection of both their experience and expertise. The commissioned officers on the ship also have areas of expertise, whether it be surface line officers focused on shipboard operation, supply officers who focus on logistics, and aviation officers when aircraft are part of the ship’s complement.
At core however, everyone on board the ship is a mariner—a sailor—and must be aware of the intrinsic dangers of operating a vessel at sea and also of those particular to a U.S. Navy warship. Before a ship deploys the crew must demonstrate their expertise on many levels in operating the ship, and so a period of time is set aside for the crew to go to sea and simulate operating under a number of possibly life-threatening or existential situations.
Under these simulated scenarios, the crew is tasked with taking action in real time based on the information being provided, whether it includes going to general quarters, manning battle stations, fighting fires and flooding, or reacting to a conventional or Nuclear, Biological, or Chemical (NBC) attack. The performance of the crew is assessed by outside instructors during these situations, which are not preannounced. During these testing scenarios the ship is operating at sea so those systems related to operations must be maintained.
Needless to say, the stress level during these extended periods of training closely aligns with those that would be experienced in real time. The purpose of the training is to accept failure during the simulation so that the initial reactions necessary to unexpected and possibly catastrophic events become almost automatic and are performed at a very high level. The crew is expected to climb the learning curve. Variations during the exercise are imposed so that the crew learns to always understand the situation, constantly reconsiders the options and risks, and makes quick decisions with less than perfect information. In all cases the goal is to save the ship.
In real-life situations in business and project management, such operational simulations are not always possible. But taking measured risks, and handling risk by taking large efforts and breaking them down into their component parts and then implementing the appropriate risk-handling strategies in each element, can allow us to live another day to learn from the failure. It can also allow us to make necessary adjustments that allow us to derive value from our failures, which we make temporary and transient.
Failure as Value
My colleague Glen Alleman, both a well-respected scientist and project manager, has written at length over the years at his Herding Cats blog about risk and uncertainty. In a recent post, Glen notes the difference between aleatory uncertainty and epistemic uncertainty; in this case it is within the context of a critical analysis of the agile #NoEstimates position. But within the context of understanding failure it is also pertinent.
All undertakings have an element of uncertainty attached to them. As Glen notes, to deny uncertainty must lead one to determinism. I would put this a little differently. We live within a deterministic universe, but the individual components of this universe have a degree of uncertainty attached to them—what we call free will. The chances of an outcome are a matter of probabilities.
Thus, rather than living in a universe where perfection is possible, what we find is that we live in one that plays a game of failure, much as hitting a baseball is a game of failure—where hitting the ball safely 30% of the time is a measure of proficiency and hitting it safely 40% of the time is a unicorn. The high rate of project failure noted by various sources that I’ve written about in this blog series, and the high rate of business failure noted above, proves this characteristic about risky human undertakings.
But we do learn and are able to do things today that would have been impossible in the recent past, often with amazing regularity once the lessons have been incorporated into our day-to-day processes. Understanding this then allows us to see that there is value in failure—to answer the question that my senior engineering boss posed years ago.
When we fail a test or a development of a module or sprint, we must be able to define the elements of success and failure. An assessment then must be made of the probability of achieving success in the next iteration. The progress actually achieved, even if the end result was an intermediate failure, is the value gain to that point.
This is an important concept not only to those who apply such project management techniques as earned value, but for those who have to deal with the financialization of business, which is becoming widespread. Risk must be honestly assessed, but a setback in the development of a product is not necessarily an overriding, singular event. When broken into its constituent parts, the project may show that the effort is worth continuing despite the psychological and sociological effects that attend it.
For example, in the case of the federal website rollout for the Affordable Care Act (also referred to as Obamacare), I noted at the time how the program itself was a success while the transient issues of the website project were blown out of proportion due to the politics surrounding the program. The falling numbers of uninsured people around the country since the program’s rollout prove out this analysis.
The Characteristics of Turning Failure (Possibly) into Ultimate Success
Thus, here is a back-of-the-envelope list of characteristics in accepting and coping with failure to provide a basis for achieving the ultimate goal:
- Develop a strategic vision and always work toward its achievement.
- Identify the elements of the task at hand, and identify risks associated with them.
- Implement risk handling at each appropriate level. Only assume measured risk.
- Always have a Plan B.
- Initiate and encourage an honest situational assessment. Develop and assume intellectual honesty.
- Possess a willingness to adjust both the operational and tactical plans as necessary.
- Develop and assume psychological and emotional maturity, resilience, and doggedness.
- Develop and deploy systems to identify value from setbacks and failures, learn from them, and apply those lessons learned.
- Develop and deploy systems that identify the value intrinsic to those portions of the effort that succeeded in the face of setbacks and failures elsewhere. Build on those successes.
In the end, if the strategic vision becomes unattainable or the operational and tactical efforts are out of alignment with the vision, then the overall effort may need to be abandoned. If not completely abandoned, a new strategic vision based on the new reality may need to be developed. Ultimately, how we identify, assess, characterize, and cope with failure is intrinsic to how we think of project management and the systems we deploy in its support.
For more brilliant insights, check out Nick’s blog: Life, Project Management, and Everything