Percent Complete is a Lie

I’m sure you’ve been there – you’re sitting in a project status meeting, and a manager is going around the room asking for updates on all the assigned tasks: “is this task complete?  No, then what % complete is it?”,  or “this task was at 50% complete last week, what % would you say it is today?”

No, just no.  Percent Complete is a Lie.  A complete lie. It gives everyone in the meeting a false sense of progress, masks potential risks, and ultimately leads to schedule slippages.  A task has been stalled at 80% for 3 weeks now, finally it’s starting to become a critical path issue – oh but we still have another week, that 20% can be completed in a week, right?  Well, no – it’s the Pareto Principle coming to bite you – turns out the reason it was stalled was that “20%” was the developers staring down an intractable problem with error handling, or were hopefully optimistic that some buggy library they were depending on would be fixed in the next release before they really had to use it.  Or they were banging their head on the desk over an IE 8 incompatibility issue.  You never know – the problems were all masked because you kept asking for “your % complete, please”

This problem is endemic to software development, but is likely an issue in other fields as well.  Software development is not baking and developers are not ovens …

In the late 1980s, I was in graduate school in the D.C. area. I was in the economics department, but had the good fortune of studying under Don Lavoie, who was a multi-disciplinary thinker. He started exploring organizational learning (and would later create a department called Program on Social and Organizational Learning), so in the summer we participated in a series of seminars co-hosted by MIT and Harvard.  Material that would later become Peter Senge’s book The Fifth Discipline was presented at this seminar, and us participants got free copies of Idea, a very attractive, Mac-based process modeling application. A user could build a process model with feedback loops and bottlenecks, using flow tools like conveyor belts, queues, and ovens.  An oven would allow a certain amount of inputs in, then bake for a specific time, then output.

With an oven, one can ask the question, “how much time has it cooked?”, and “how much time is left?”, and with a little math, “what is the percentage complete?”.

Software development is not an oven.  There are some grunt-work tasks in software construction, but over time most savvy developers build up, or acquire, tools and frameworks to reduce time spent on such mundane development.  So, the mundane, oven-like software tasks are simmering away on the back burner, while the bulk of software work, like creating new recipes or overcoming a collapsed soufflé, takes over all available kitchen space.  And, like other creative endeavors, is unpredictable.

Creative writing, technical writing, graphic illustrations, design – they’re not ovens. Since their creation processes have risk and unpredictability, it is inappropriate to ask what “percent complete” is their task.  This goes for any project task that isn’t already in the oven with the door close and the timer on.

So, you may ask, “how then do you measure project progress?”   Well, for software development, my process with my teams is to break down the tasks at hand into smaller, more predictable chunks, and then progress toward tangible milestones.  These “tangibles” are usually user-visible features – architectural framework doesn’t count, a business layer doesn’t count, neither does a database design or anything back-end.

If the team is developing a complex layered architecture, then get them to quickly develop a “hello world” app that calls through all the layers, even if only with “stub” method calls.  Then proceed from there, adding user features.  If the software engineer declares any single step within these layers will take more than a week, then have them break it down into smaller chunks.  Maybe the “chunks” is to separate function definitions from business object declarations, for example; or if the developer is tasked with the creation of a series of functions, or a series of database tables, then break them down into multiple series.

When I’m doing rough-cut estimating, I’ll break down the functional requirements into all their component pieces and layers – “ok, this will require a search screen, then a screen of search results – that’s 2 screens; search happens via client-side call to a server API, so that’s 1 client function, 1 server function; the server function does a database call, that’s another function or layer, the database itself will require development of an index table and a documents table, that’s 2, then there’s the search result “object”, maybe it’s a JSON dictionary, let’s define that and call it another task, etc., etc.

Then, for each identified piece, I give it a “trivial”, “easy”, or “complex” rating – trivial, 15 minutes or so, we don’t even count.  “Easy”, I typically break into 4 hour chunks, or if being conservative a day. “Complex”, well, that’s an eyebrow-raiser, but for estimating purposes I’ll give it a week, then when it comes down to actual development, I’ll focus on that as a risk item and ask for it to be prototyped rapidly, so the developers can discover the risks, or come up with alternative plans in case their chosen approach proves intractable.

None of this process involves % completes.  Instead, the project manager now has smaller tasks to track, asking, simply, are they complete or not.  Tasks that keep slipping off the schedule, that becomes your immediate alert that the project may be at risk, versus some task that has been at 50% for a couple months, only later to be revealed that it really can’t be done as designed.

So, throw away the whole “% complete” lingo, and insist on a more transparent, risk-apparent approach to your projects.

Indonesia … and a “million dollars of problems”

(see also, Jakarta Contrasts)

Many folks have asked me, “how did you get to work in Indonesia?”  Well, it was a small chain of events.  I was doing Lotus Notes development work at the time, and Lotus Consulting partnered with my small firm (they were very keen on their partner network) to do several small applications.   One of their managers asked if I could work on a project in New York.  Turns out it was a “troubled project”, a highly visible project that wasn’t going well, and they were at risk of losing the whole contract.  The client was Ernst & Young.

As one of the Big Six professional services firm, E&Y was a high stress environment.  This project was for some internal knowledge tools,  development was mostly complete, but E&Y was not happy with the results.  When I arrived, tension was clearly in the air, but I ignored it.  I simply concentrated on what that they weren’t happy about.  As a newcomer, I could ignore the internal politics, the past conflicts, the personality clashes, whatever – I just asked, “so, specifically why are you not happy?”.  Unsatisfied with vague complaints and accusations,  I pressed to be specific in their grievances, with a focus on the software application we were delivering to them.

Continue reading

About Us

Agoric Source, LLC is a Houston-based boutique software development firm.  We are somewhat on hiatus, as the principal, Kevin Lacobie, is exploring a career in the corporate world.

Agoric Source was created in 2002, and is the 3rd firm co-founded by Kevin Lacobie, the first being Agoric Enterprises, Inc., 1993, in Fairfax, Virginia, and Agorics, Inc., 1995, in Los Altos, California.  Kevin Lacobie has 30+ years experience in the software and IT industry, and with an advanced degree in Economics (MA, George Mason, 1991), brings a unique perspective to the software engineering discipline.

Coders like to code, code, code; software designers like to design elegantly and cleanly; software engineers like to weigh the broader economic tradeoffs between different technical solutions and their use in practical business environments. Software engineering explores tradeoffs between current code and future, maintainable code, and between automation and human processes.  This blog will share insights about all these issues, from Kevin’s perspective of theory and much industry experience.