13 years, 4 months ago

When effort estimations feel like buying showroom kitchens

a modern showroom kitchen

A showroom kitchen sans sales staff

Purchasing a showroom kitchen can be an unpleasant experience, especially if you’re not sure of the value of what you are buying. Kitchen showrooms tend to have the atmosphere of a prestigious car dealership, perhaps because the price tag of a well-designed kitchen is similar to a mid-sized car. If you happen to linger at one of the displays for a while, it doesn’t take long to find a well-dressed salesperson by your side. Armed with copious amounts of charm dripping off in puddles, the salesperson has a constant smile that doesn’t waver for a moment. Not when you are informed of the whopping price tag, nor when a special “one-time offer” is worked out only for you. Of course, you have to accept the great offer at that very moment to buy the entire kitchen for a fraction of the original price. If you end up purchasing the displayed kitchen, you can never be sure if you really got a good deal because you’re not sure of its actual worth. You have a sneaking suspicion that there is no way reconstituted wood, stone, or metal could be worth so much, but find some small consolations. After all it bears the mark of an Italian designer’s signature line and it has a construction quality that will last you a lifetime. After a few months, the first hinge falls apart from one of the cupboards, and then an edge chips off the stone slab. The drawers come apart if you pull them too hard and some of the stands begin to feel wobbly. You may come to learn that the wood of your drawers were logged in Indonesia, the stone slabs were quarried at minimum wage in Western India, and it was all put together by workers in a sweatshop in China. After spending some time with the kitchen, you come to hate the whole thing. You hate the fact that you got suckered into buying something of which you never really understood the actual value, and that the entire experience would have been exactly the same if you went to any other shop. With customers rarely returning to the same shop for another purchase, the sales crew know that they are spending their lives selling things of dubious value to people who are not happy with their purchase afterwards. The entire process, from manufacturing to sales, and from purchase to product usage, is an unhappy experience for all concerned.

There are other comparable experiences where the correlation between the price and the value of the proposition is unclear, such as buying holiday timeshares or discount fashion brands, and this is an environment where vague estimates thrive. Given the intangible nature of software, estimates for realizing software and systems are often quite poor, reflected in the high rate of failure of software projects (as high as 68% apparently).

Effort estimations in “sheltered” environments

Modern Times

A scene from Charlie Chaplin's classic "Modern Times"

Large organizations have several competencies, and sometimes these competencies have to work together to realize common goals. These common goals are realized through systems, and the requirements for such a system can pass through several organizational layers in a sequential chain. In some environments, it feels like the requirements have been sequentially enriched by each layer, each party in the chain adding insight and value. In others it feels like the requirements have been sequentially passed through several digestive systems, each party in the chain extracting context, meaning, relevance, and urgency from the requirements, until a final set of sterile requirements remain, well removed from real needs. The entire work model is loosely based on a layman’s view of an assembly line, where each person in the assembly line is perceived to complete a few repetitive tasks. The development methodology used in these environments is typically waterfall, with most parties in the sequential chain being accountable for an aspect of the deliverable, but not the deliverable itself. In this assembly line, a disproportional number of people specify and plan things for a few developers.

Apollo 11 launch

Sometimes estimates feel comparable to a moon mission

The experience of obtaining effort estimations in these environments isn’t very far off from buying timeshare or kitchens. The individuals who often do the estimates aren’t usually the ones that do the actual work, and in large organizations, the estimates are often generously padded against imaginary scenarios with a very low probability of occurrence. Waterfall processes are usually the only way to structure work in supply-demand relationships, where the focus rests on accountability rather than delivery. It is a painful process for the estimator who is accountable for the timely delivery for an ambiguous unit of work. The estimator has to dig deep to imagine everything that could possibly go wrong in the delivery process and determine the time it would take to resolve every obscure issue, and these are all included in the estimate. For good measure the estimator will include some padding to cover the scenarios that they have not encountered before. Uncertainty and doubt must be quantified upfront, and the estimator retaliates by providing estimates comparable to the Apollo Moon Mission. These estimates don’t stand very well to close scrutiny, and closer examination can see them fluctuate, a bit like the prices of a showroom kitchen.

“I’ve got a macro that does my estimations”

Around a decade or so ago, I used to work in a situation where my client had outsourced their development to another organization and I was the Architect in the project. Development was done offshore, and a series of intermediaries were responsible for providing upfront estimates. We needed some extensions on an existing application, and we had done an upfront prototype in 6 days for demonstrating the functionality of the extension. We had functional coverage for around 80% of the critical functions, and the remainder was documented in a specification. We sent it off to the chain of intermediaries that formed the delivery process, and after a few days I received a development estimate for 6000 hours. A bit surprised, I went over to the principal intermediary to have a chat, and asked him how he came up with that number. “Well”, he explained, ” I have this excel macro which gets all the other figures provided by other people in the process, and then figures out the total development effort, adding a few contingencies here and there.” I pointed out to him that 6000 hours was the rough equivalent of 1 person working for 35 months at a stretch, and although the prototype was a mish-mash between mock-screens, embedded data, and inline scripts, it did take 6 days. After some contemplation, he said he would get back to me, probably with a number closer to 3000 hours. In the course of the next few days, the numbers went down even further, from 3000 to 1200, and finally ended up at 500 hours. It smacked of buying a showroom kitchen, but it really wasn’t the intermediary’s fault. He was a cog in a larger network of machines, and interestingly, he had tried to make the mundane part of his job, being another cog doing estimates, interesting by writing an excel macro.

Effort estimations in “unsheltered” environments

Trading Floor

Sales-driven environments tend to provide opportunistic estimates

In IT organizations that are sales-driven, such as web agencies or product start-ups, estimates are often provided by people closer to the sales process than the development process. The proliferation of IT service companies over the past 20 years has gone beyond saturation point in most places in the world, and the intense competition has led to a pressure to promise more things (extensive functional coverage) delivered faster (shorter time to market) for less money. Estimates for Software Projects in these environments are opportunistic rather realistic, and extensive functional coverage is promised within aggressive delivery time lines by individuals that are more committed to the sales process than the delivery process. Similarly, software product vendors that may have innovative proposition, but lack functional coverage, can promise things that they have never encountered before “in the next release”. The expectations of the stakeholders are often higher than what can be delivered, and these initiatives usually result in delivery teams resorting to heroics to deliver what was promised.

Existing without shelters or the feeling of being unsheltered

Cynefin framework

Dave Snowden's Cynefin framework for structuring problems, situations, and systems

There are more innovative means of obtaining estimates, and in some cases, even fun ways of obtaining estimates (SCRUM planning poker is a good example). However, a prerequisite that must exist is a collaborative relationship between the people doing the work and the people requesting the work. As long as there is a sharp line demarcating one bunch of people as suppliers and another bunch as demanders, the value of the deliverable will always take 2nd place to concerns of liability. Strong leadership is required to make this transition, and it must have support at the highest levels of authority. A good understanding of what can be quantified, and what cannot, also helps. Dave Snowden’s Cynefin model, which is used to describe problems, situations, and systems, can be used to identify simple, complicated, and complex factors in the delivery process. Dave Snowden’s framework is a sense-making framework, and it should only be considered as a means of providing guidance and not an empirical model.

Using the Cynefin framework, we can structure requirements as simple, complicated, and complex. Simple requirements are those that have well-known existing solutions. Complicated requirements are those that require investigation before a solution can be found. Formal methodologies to manage requirements, no matter how innovative, can only address these 2 types of requirements at the onset of a software project, delivery cycle, iteration, or sprint. Complex requirements can only be realized retrospectively, where analysts can look back at a situation that was handled unsatisfactorily or sub-optimally, and come up with a structured solution afterwards. The 4th quadrant, chaos (sometimes managed in software delivery contracts as “an act of god”) is not something that can be handled through any solution.

Simple and complicated requirements can be handled in projects by tackling the most complex problems first, reducing the risk in the project. There are established software development methodologies that have evolved over the past 40 years that can handle simple and complicated requirements. Complex and chaotic elements that will occur in the course of the project can either be managed through fear or acceptance. If the fear of the complex and chaotic is a stronger force, it will be handled by a mixture of generous estimate padding, lengthy legal contracts with vaguely-worded caveats (“…but not limited to…”), and elaborate contingency plans that will try and manage the unknown. The acceptance of the complex and chaotic, on the other hand, is handled through transparency and openness between all parties, the ability to handle complex and chaotic things through an agility that is embedded in practice and work, a culture of continuous improvement, constantly learning from others, and being open to unraveling the complex and chaotic through fearless analysis.

Fearing the unknown

Car through Roof

Some things cannot be anticipated

Fear, no matter how irrational, is a natural emotion when facing uncertainty. Some uncertainty cannot be conclusively removed at the onset of a project, especially complex and chaotic elements. The pursuit of a commercial opportunity can be jeopardized by sharing fears openly, especially if there is a clear demarcation between a supplier and a customer. In new relationships, trust is progressively earned by demonstrating the ability to deliver on promises. Being open about uncertainty in requires the strong confidence of all parties of being able to deliver collaboratively on simple and complicated requirements. Collaboration is the key value to distill out of these new relationships, where the parties agree to tackle uncertainty together and not resort to a blame game. If poorly managed, fear (along with its cousins uncertainty and doubt) is managed by suppliers by trying to get as much money (or commitment for money in the contract) from customers upfront. When complexity or chaos crystallize, effort is expended in conflict resolution, which can ensue in a blame game, rather than working collaborative to solve complexity or chaos. Time and effort are spent on stressful firefighting, rather than experiencing the creative energy realized from working together for achieving common goals.

Accepting the unknowable

Classic Toyota 2000GT

The Toyota Production System was documented by Taiichi Ohno's in his seminal work in the 1970s

If all of the participants in a project are open to accepting that complexity and chaos, when they occur, will be handled collaboratively, the results will be far more pleasant. Complexity or chaos patterns, if they are already known through experience, should be shared upfront. Parallels with other industries or similar initiatives should be used whenever possible to gauge where complexity or chaos could possibly arise. But as complexity and chaos cannot be gauged up front by their very nature, rather than spending time upfront on doing effort estimates, focus on the nature of collaboration within the team rather than on concrete task estimations. This nature of collaboration will be implicit in the team if it is already operating in this mode for realizing simple and complicated requirements. The practical philosophy of Kaizen, invented at Toyota and applied through the Toyota Production System, is the practice of continuous improvement. It is a daily process that humanizes the workplace, eliminates excessive hard work, and teaches people to perform progressive experiments at the workplace. The single focus is around the elimination of waste, and the collaborative work environment is focused on this single-minded goal. The roots of agile methodologies, like SCRUM, are based on the Toyota Production System, and Taiichi Ohno’s seminal work is a good start in understanding the culture of a collaborative work environment.

When things go wrong, blame the environment and not the individual

All Blacks

The All Blacks put on a display of fearlessness to their competition.

Consider this. When someone in the team fails to produce effective results in a Kaizen environment, the manager walks up to that individual and apologizes for not having created the conditions for them to work effectively. The principal task of a leader is to create a working environment where everyone can be productive, and progressively removing a blame culture (if it exists). The principal purpose of planning is to provide direction in outlining what’s coming up, so a person doing a task at the present understands how the nature of the work will change in the short and long term. Exact time and effort estimates of future effort are fiction, and the only thing a project manager can tackle with certainty is the ability of his or her team to handle the unknown fearlessly.

Perhaps your current working environment today isn’t exactly Kaizen. But don’t you wish it could be? The beauty of the perfect working environment is that it is a constant pursuit, and in its pursuit, everyone in the workplace finds themselves in a great environment.

I would like to thank my friends Tony (for the Showroom Kitchen analogy), Ewout (for the ideas of community-based governance), Andre (for the wonders of Kaizen in a SDLC), and Sergej (for educating me on value-oriented thinking) for providing me much of the ideas and material in this post.

© 2010-2011 Indranil Bhattacharya