Characterisation of projects: part 4 - implications of the framework

I’ve previously described a characterisation framework for projects, in which I differentiated projects based upon the level of uncertainty involved, and the potential consequences of that uncertainty. I’ve also referred to some prior frameworks, and discussed how some project management process models fitted the framework.

In this post, I talk about what use this framework (which I’m now calling the UC Framework, with capital letters even) might be for project managers.

In its simplest application, placing a project on the UC Framework allows it to be compared with the comfort zones of various process models, assisting in the selection of a process model. For example, an NPD project involving a small team and known markets and technology ( a “line extension” project) would typically have low scores on both U and C axes, suggesting that plan-driven, Agile, and possibly even ad-hoc approaches would be effective. Selection of a process model then comes down to individual project requirements. In this example the characteristics of the product might point away from Agile techniques and a compromise between plan-driven and ad-hoc – effectively a bare-bones implementation of a plan-driven model – might be used.

The model can also be used to identify leverage points for adjusting the project’s parameters to provide a better fit to a preferred process model. For example in major infrastructure projects, the major driver of project uncertainty is complexity, and this can be sufficient as to raise project uncertainty outside the plan-driven model’s comfort zone [75]. If a plan-driven process model is to be used, reducing the uncertainty effects of this complexity by improving the team’s understanding of it could improve project management effectiveness [76].

Given a system for quantifying the axes, different projects could also be plotted against each other on the UC Framework, allowing a comparison of the relative risk (UxC value, by analogy to other risk management techniques) of significantly different projects to be made. The framework also highlights opportunities to trade off different factors against each other to minimise project risk. For example does the increase in CU gained by shortening the project duration and thus improving environmental stability? Alternatively is the criticality of the project such that the extra costs of collocating the team are justified? caused by using a large geographically dispersed team outweigh the reduction in

Such assessments are necessarily only snapshots, and projects change over time. Over the course of a project, the level of uncertainty associated with it will typically drop, as information is acquired and decisions made - indeed, a project can be viewed as an information search [77]. Similarly, over a project’s course the consequences will typically rise: team size and distribution is more likely to increase than decrease during the project, “sunk costs” increase throughout, and commitments which raise the stakes are typically made in the later stages. This can lead to the need for adjustment of the process model, and even the need to switch models entirely – this is particularly true for projects which commence in a problem-structuring mode, as this mode is likely to be less helpful in the latter stages of a project unless implementation proves to be trivial. Such changes over time can be highlighted by periodic reassessment of the project.

The left and upwards trend is broadly similar to the shape of a “constant risk” curve. This is consistent with the tendency of people to adjust their behaviour to maintain constant “target risk” levels (Risk Homeostasis Theory [78]) – project managers and team members will tend where possible to defer the more critical decisions until they are made more “comfortable” by the availability of better information. Real options theory [79] suggests that this is economically sensible behaviour, and that committing early to important (C increasing) decisions may be counterproductive. However the UC Framework shows that the risk associated with such decisions may be reduced by minimising other C-increasing factors, or by pulling forward U-reducing activities in proportion.

Fundamental principles: Show me the money

“An engineer is a man who can do for a dime what any fool can do for a dollar” - Anonymous

The “a man” bit’s inaccurate nowadays, but the rest of it hits the nail right on the head.  At its basic level, the profession is all about tradeoffs, and the biggest one of them is performance vs cost.  Maximising bang for buck is what it’s all about.  Awareness of the short term and long term cost implications of everything you do and every decision you make is one of the core abilities of a top-level engineer.

This is one of an occasional series of posts on the fundamental principles of engineering.

ZED Forum

I attended AUT’s ZED Innovation and New Product Development forum yesterday. As with most such things, patchy but some of it was pretty good. Oddly enough, Chris Quinn from Gen-i, who spoke mainly about maintaining a culture of innovation in a large organization, was the standout for me in terms of food for thought. Both Hans Van der Voorn of Australo and Paul Adams of EveridgeIP had interesting stuff to say - Hans about the need for speed and product usability in startup companies, and Paul about leveraging unused IP.  Australo are doing cool stuff - this sort of company that finds a tiny, high-margin niche and aims to own it strikes me as the way of the future for NZ. Bill Day of Seaworks and Geoff Ross of 42 Below were, as expected, highly entertaining speakers though Bill’s theme, how entrepreneurs think, was some distance off topic for the forum - enterpreneurialism and innovation aren’t the same beast, though there’s a strong trend within forums of this type in NZ to conflate them.

Couple of interesting quotes:

From Chris Quinn:  “Culture is the sum of the conversations of the senior people“.  So true - leadership can’t kid themselves that people won’t notice if they talk the talk but don’t walk the walk.

From Mark Cowsill (MD of Frucor):  “Every company has a process, they’re all pretty similar, and I don’t believe they’re a competitive advantage at all“.  An interesting perspective in this context, particularly given that the general theme of his presentation seemed to be “we’re really good at this stuff, but buggered if I know why”.  Having said that, I tend to agree with him about the competitive advantage bit - the only sustainable competitive advantages are culture, recruitment, and ability to learn (which collectively add up to getting, keeping, and leveraging the best people).  A screwed up process can be a pretty powerful competitive disadvantage though.

Learning from Making

I did a short (20×20) presentation to the Designers Institute of New Zealand a couple of days ago. I can really recommend this presentation format - it’s a pig to prepare for, but it’s incredibly effective for making you really think about what you need to say. The presentation (heavily academicized and with some stuff on the end about what I’m hoping to do about it) is going to form the core of my preliminary review, which is due in a couple of months time - I figure now I’ve got it condensed down to almost nothing, expanding it to 30 pages or so should be a breeze.

The text and slides are below. A note on the slides: most of the images are either mine or came off flickr or wikimedia and are either GNU or Creative Commons licensed, but a few are taken from I can’t remember where on the web. I’m fairly comfortable that my use of high-res copies in the in the presentation and of thumbnails here is within “fair use” or “fair dealing” provisions, but if any of the imagery is yours and it offends you, let me know and I’ll pull it down.

The presentation is on the subject of how design can learn from other disciplines - primarily software and manufacturing - about managing their process. Since I come from the “hard” rather than “soft” end of design (i.e. engineering) I started by sticking a few stakes in the ground as to what I’m talking about when I say design - this probably doesn’t overlap much with the viewpoint of a lot of the audience, since many of them were from fashion or graphic design industries.

————————

The first point I want to make about design is it has a purpose. To me, all of these are designed objects. And all of them are designed for something. The purpose might be functionality, aesthetics, image, communication, or whatever. But what they have in common is the for. And the for’s not realized till the job’s complete. Completion is important.

The next thing about design is that it’s a process. It’s not enough to realize that what the world needs is a chilli banana chesecake – the job’s not done until the happy customers leave the restaurant, and slaving over the typeface of the menu is just as much a part of the process as the fun stuff in the kitchen developing the recipe.

The third thing is it’s an uncertain process. Novelty, by definition, means uncertainty. You can’t innovate in a perfectly controlled environment. And lets face it, in the small business environments in which most of us work, you can’t control the environment anyway. But the current innovation management methods try to do exactly that. And they fail.

They fail even in the big corporates they were designed for. In small, fast-growing tech businesses – the kind NZ needs – they’re exactly the wrong tools. Which is why nobody uses them. There’s plenty of research to show that most small businesses run design and product development by the seat of their pants. But winging it can only take us so far.

We need a way to keep a degree of control while letting creativity flourish. And oddly enough, we can find models for that where we might least expect them. The mass production paradigm has changed the world in the past century. But even in this controlled environment, the rise of lean manufacturing over the past 20 years has shown that flexibility and adaptability outweigh efficiency and control.

And in the IT world, the past decade has seen the waterfall software development model fighting a losing battle against the wide-eyed idealists of the Agile movement. What lean and agile share is an emphasis on people over process, flexibility over compliance to plan. But unlike the all-or-nothing approach we tend to have in the design world, these processes manage to combine flexibility with method.

It’d be nice to think we could just plug them straight into our design process. Unfortunately it doesn’t work like that. Agile straight out of the box relies heavily on the special characteristics of software. Lean has been translated into design management – not least by Toyota - but read a book like this one, and you quickly realise that plugging the Toyota system straight in to a small business would be both impossible and a disaster.

So we need to customize the tools. And that’s where my research lies – I’m going back to first principles and asking what these approaches have in common that can work for us. I don’t have any answers for you today, but I want to highlight some of the areas where there are lessons that could turn into tools.

I see the common thread coming from another unlikely area: industrial process control. Control theory tells us that to control an undefined process, you need to use frequent small adjustments. That means shortening the feedback loop as much as possible, and improving the system’s ability to respond to feedback. Both lean and agile focus on ways to do exactly that.

Lets look first at feedback. In design, feedback comes from generating information, learning from it, and adapting to it. That means getting the right info to the right people as early as possible and ensuring they can act on it. It means putting as much effort into surveys, tests and experiments as you do into the product: information gathering is a critical design activity and deserves respect.

It means working on cycle costs – the cheaper and faster it is to spit out a daft or prototype and get feedback on it, the more often you can do it and the faster and better you’ll learn. It means keeping the customer focussed on value not contracts: there’s no point in learning things if you can’t change course based on the new knowledge.

And it means keeping your options open. Once a decision is made it’s hard to unmake it. The right time to make irreversible decisions is as late as possible – at the last responsible moment. Planning needs to take this into account – plan like hell for the stuff you can predict, but don’t trap yourself into letting your plan make your decisions before you’re ready to make them.

Now lets talk about response. We’ve all been pushed to do stuff faster. Personally, I don’t believe design can be fundamentally sped up without compromises. Thinking takes as long as it takes, and if you try to do it faster you just get less of it. Maybe you’re lucky enough to cut out the low-value stuff, but mostly you just cut down quality. There’s a lot of evidence to show that the correlation between accelerated product development and profit is dodgy at best.

But there are argument for speed. Information is our stock in trade, and it gets lost, forgotten, and obsoleted. The photo here is the inside of a two year old Formula One engine. I took it – in public - about the same time Maclaren were being fined a hundred million bucks million for stealing one year old data. A hundred million to worthless in 1 year. F1 is an extreme case, but information depreciates fast.

What lean and agile show us is that to go faster, we shouldn’t focus on what we’re doing, but what we’re not.The leverage is in the gaps. There’s more time to be gained by keeping the work moving than by doing tasks faster. And the economics of information depreciation means it’s worth spending money on that. A job sitting waiting is bleeding money. We need to organize our processes around workflow.

A classic example of that is avoiding batch processing. Anyone been to the Coromandel? Remember the Kopu bridge? More importantly, anyone remember the two or three intersections after the Kopu bridge? Batches hold up the workflow – not just at the batchpoint, but downstream. They’re a convenient way to work – build it up till you’ve got enough to be worth doing – but they’re a bad way to work.

Another example is timeboxing. Keeping the job rolling to a rhythm lets feedback be incorprated smoothly and lets the players be ready when it comes their way. So instead of breaking it into chunks of work which take as long as they take, break it into chunks of time that achieve as much as they achieve. As long as you prioritize the important stuff within each time box, the 80/20 rule will work in your favour.

Another idea which I think has real value is extremism. Don’t do things by halves – take it to the limit one way or the other. In IT, finding bugs sooner is better. What’s the extreme of that? Finding them as they’re written. So Agilists put 2 people at one computer and do the dubugging simultaneously with the coding.What’s important in your process? And how do you turn the knobs on that up to 10?

Conversely, what can you turn right down? Lean gurus believe “if it’s not contributing value it’s waste”, and the agilists mantra is YAGNI – You Ain’t Gonna Need It. Elegance in design is the art of the minimum necessary. ISO9000 and suchlike encourage us to design processes like motorway bridges – we have to ask ourselves “how big a bridge do we REALLY need?”

I want to conclude by suggesting that design isn’t a machine, and we shouldn’t treat it as such. But neither is it a craft, and we shouldn’t treat it as that either. The design process is something that could benefit from some design, and, just as in the design of things, if we don’t take inspiration from outside our field, we’re letting ourselves down.

Cheap/fast/good

Ben Casnocha in his book My Startup Life makes the very good point that you can only have two out of three of the above, and it’s a good idea to pick which ones are important before you start.

In NPD, it’s complicated by the fact that cheap and good for the project aren’t necessarily the same for the product. So we have what I’m hereby dubbing the NPD hourglass. Pick two for each triangle but if you pick “fast” you need to pick it for both. A cheap fast project can give you a cheap product or a good product, not both. On the other hand a cheap good project can give you a cheap good product, but it won’t be fast.

My chain of thought so far expressed in 4 books

In order of how the logic develops:

Managing the Design Factory, by Don Reinertsen

The Toyota Product Development System, by Morgan & Liker

Lean Software Development, by Mary & Tom Poppendieck

Flexible Product Development, by Preston Smith

Zotero

This sort of thing is why open source is so cool.  Blows EndNote out of the water.

Design patterns

Semi-formed thought for the day:  are Design Patterns a useful variation on the Lean theme of ‘low level standardization for high-level freedom”  like Toyota’s use of design standards and tradeoff curves?

Characterisation of projects: part 3 - fitting methodologies to the framework

A while back I wrote about a characterization framework for projects, in which I differentiate projects based upon the level of uncertainty involved, and the potential consequences of that uncertainty. I’m hardly the first person to try and differentiate projects in some systematic way - I highlighted a few other efforts here.

In this post, I look at how some of the major project management methodologies fit on this framework. I use “project management” here in a much broader sense than conventional usage: any method for organizing and undertaking project tasks carries implications ofr how that process can or should be managed, so even methods which are normally thought of as being about execution rather than managing, are in a wider sense PM methods.

In a plan-driven methodology such as those advocated by PMI IPMA etc, project goals are identified and the necessary steps to achieve this goal determined. The steps are organized into an optimal sequence given resource and other constraints to form a project plan. This plan is then followed, with the objective of management being to administer the activities and to deal with deviations from the plan.

Considering this methodology on our framework, its effectiveness can be seen in the ideal case to be independent of the consequences axis. As the objective of planning is to anticipate and avoid surprises, their consequences are irrelevant. However the effectiveness of the plan driven methodology will vary along the uncertainty axis. The plan is inevitably based upon assumptions about the goal, methods, required effort and resource constraints among other things. As uncertainty increases these assumptions will be less valid. This can be compensated for to some extent by increased planning effort, however in the limiting case developing the plan itself becomes a dominant project task. The upshot is that as uncertainty increases, deviations from plan will increase, and the project will become less plan driven, and more driven by ad-hoc responses to surprises.

The plan-driven methodology is an example of an uncertainty-limited methodology, as shown in Figure 2.

Another such example is that of the problem structuring methodology. Problem structuring methods such as the Soft Systems Method (SSM) focus upon the definition of the objectives and environment of the project. Their effectiveness is again independent of the consequences axis, as implementation – and therefore the opportunity for consequences to occur - is assumed to be trivial. But effectiveness again varies along the uncertainty axis. As noted above, the relative importance of implementation and validating assumptions – or problem definition - changes. At low uncertainty levels, where the situation is relatively clear, “the process of inquiry into the problematic situation” [67 p S50] becomes a trivial one, and problem structuring methods become irrelevant. The problem structuring methodology is thus also uncertainty limited, but the limit is at the lower bound.

Not all methodologies are limited by uncertainty. The simplest project methodology is the ad hoc one, also known as “winging it” or “hacking”. In this methodology, no planning or preparation is undertaken, and the project team simply follow their noses and deal with problems as they arise. This methodology’s effectiveness is in the ideal case independent of uncertainty, as the lack of anticipation means that everything comes as a surprise – uncertainty is effectively maximum in all cases. However its effectiveness is highly dependent upon consequences: lack of any control system means that any event which the team is unable to handle is likely to quickly run away to disaster, and that it will not necessarily be known whether the event has been handled satisfactorily until too late. This methodology is thus restricted to situations where the teams ability to handle unexpected events is high (meaning extremely small and expert teams), and “disasters” cannot be overly serious: the methodology can be considered to be consequence limited.

A similar argument can be applied to the Agile methodology. This is similar to the ad hoc methodology in that uncertainty is dealt with by adaptation rather than anticipation. The technique is again limited in the consequence dimension by the team’s ability to cope with the unexpected (which is dictated by factors such team size, distribution, empowerment and competence), and by the magnitude of the worst case disaster. However techniques within the Agile methodology are largely aimed at providing control systems to mitigate these limitations, by maximizing team responsiveness and providing rapid feedback as to the effectiveness of responses. The upper consequence limit under which Agile techniques are effective is thus higher than for the ad hoc method, again as shown in Figure 2.

The limiting values shown in Figure 2 are arbitrary, and in fact a tapering off of effectiveness rather than a hard limit would be more realistic. However the graph shows clearly the respective “comfort zones” of the methodologies discussed, and areas of overlap and of lack of coverage.

In areas of overlap, the issue for methodology selection is not so much one of effectiveness as efficiency. For example in the area of relatively low consequences and uncertainty, both Agile and plan driven methodologies are effective. However the cost drivers of each are different. Plan driven methods impose significant overhead costs in documentation and control, although these an be minimized by “lightweight” implementations (see for example [68]).  On the other hand, as a “learning” approach Agile has a lower ideal efficiency than the “prescriptive” approach [69, 70], imposing risks of wasted work and suboptimal sequencing.  Which cost dominates will depend on the particular circumstances of the project in question.

Although projects with significant uncertainty and high consequences have been managed successfully, the graph shows a gap in this area between the methodologies considered. The high incidence of failure in large capital projects, particularly in the IT and construction fields [71-74] suggests that this is a combination which is not easily manageable using existing project design methodologies.

Ask the impossible….

I was recently approached by a previous employer to go back to them. All very flattering, though I’m not sure that it’s for me. But it got me thinking about why I left. Like most job changes I suspect, in loose (or polite) terms this one was driven by “philosophical differences with management”.

What that came down to in this instance was goal setting. Goals at this company were set very aggressively - during the time I was there I don’t think we ever hit all the goals of a project, and reading between the lines of annual reports since I’ve left, it looks like that record has been maintained. There are a lot of reasons why that might happen, but in the case of this company I think it’s because the CEO is a believer in the maxim “ask the impossible to achieve the best possible”.

That’s not something I buy into. To me, overly aggressive goal-setting is counterproductive.  It leads to rushed work, poor decision-making, and an inevitable descent into firefighting and resource shortages.  Products get launched half developed, which leads to poor sales growth, reputation damage, and still more rework. If developers don’t buy in to the possibility of management’s goals, they ignore them and start to set their own, which makes it that much harder for management to maintain staff morale (and to maintain their own credibility:  from personal experience, trying to persuade people that something that both you and they know is a dumb idea is actually a good one is no fun) .  Stress and the repeated sense of failure when goals aren’t met (even if the project turns out to be a success from a long-term business perspective) kill motivation, and lead to burnout and staff turnover, thus exacerbating the resource shortage problem.  Resource shortages lead to the low-priority infrastructure and forward research jobs never getting done, which in time reduces the organization’s innovative capacity enough that even just keeping up with the competition is a  stretch, never mind getting ahead.

You can get away with setting big hairy audacious goals every so often (though it’s worth noting that the original BHAG was supposed to be a vision statement - something so long term that the question of actually achieving it never really arises), and goal setting needs to be aggressive enough to avoid gold-plating and the student syndrome, but going for the big win as a matter of routine doesn’t work.