Learning from Making

I did a short (20×20) presentation to the Designers Institute of New Zealand a couple of days ago. I can really recommend this presentation format – it’s a pig to prepare for, but it’s incredibly effective for making you really think about what you need to say. The presentation (heavily academicized and with some stuff on the end about what I’m hoping to do about it) is going to form the core of my preliminary review, which is due in a couple of months time – I figure now I’ve got it condensed down to almost nothing, expanding it to 30 pages or so should be a breeze.

The text and slides are below. A note on the slides: most of the images are either mine or came off flickr or wikimedia and are either GNU or Creative Commons licensed, but a few are taken from I can’t remember where on the web. I’m fairly comfortable that my use of high-res copies in the in the presentation and of thumbnails here is within “fair use” or “fair dealing” provisions, but if any of the imagery is yours and it offends you, let me know and I’ll pull it down.

The presentation is on the subject of how design can learn from other disciplines – primarily software and manufacturing – about managing their process. Since I come from the “hard” rather than “soft” end of design (i.e. engineering) I started by sticking a few stakes in the ground as to what I’m talking about when I say design – this probably doesn’t overlap much with the viewpoint of a lot of the audience, since many of them were from fashion or graphic design industries.

————————

The first point I want to make about design is it has a purpose. To me, all of these are designed objects. And all of them are designed for something. The purpose might be functionality, aesthetics, image, communication, or whatever. But what they have in common is the for. And the for’s not realized till the job’s complete. Completion is important.

.

The next thing about design is that it’s a process. It’s not enough to realize that what the world needs is a chilli banana chesecake – the job’s not done until the happy customers leave the restaurant, and slaving over the typeface of the menu is just as much a part of the process as the fun stuff in the kitchen developing the recipe.

.

The third thing is it’s an uncertain process. Novelty, by definition, means uncertainty. You can’t innovate in a perfectly controlled environment. And lets face it, in the small business environments in which most of us work, you can’t control the environment anyway. But the current innovation management methods try to do exactly that. And they fail.

.

They fail even in the big corporates they were designed for. In small, fast-growing tech businesses – the kind NZ needs – they’re exactly the wrong tools. Which is why nobody uses them. There’s plenty of research to show that most small businesses run design and product development by the seat of their pants. But winging it can only take us so far.

.

.

We need a way to keep a degree of control while letting creativity flourish. And oddly enough, we can find models for that where we might least expect them. The mass production paradigm has changed the world in the past century. But even in this controlled environment, the rise of lean manufacturing over the past 20 years has shown that flexibility and adaptability outweigh efficiency and control.

.

And in the IT world, the past decade has seen the waterfall software development model fighting a losing battle against the wide-eyed idealists of the Agile movement. What lean and agile share is an emphasis on people over process, flexibility over compliance to plan. But unlike the all-or-nothing approach we tend to have in the design world, these processes manage to combine flexibility with method.

.

It’d be nice to think we could just plug them straight into our design process. Unfortunately it doesn’t work like that. Agile straight out of the box relies heavily on the special characteristics of software. Lean has been translated into design management – not least by Toyota – but read a book like this one, and you quickly realise that plugging the Toyota system straight in to a small business would be both impossible and a disaster.

.

So we need to customize the tools. And that’s where my research lies – I’m going back to first principles and asking what these approaches have in common that can work for us. I don’t have any answers for you today, but I want to highlight some of the areas where there are lessons that could turn into tools.

.

.

I see the common thread coming from another unlikely area: industrial process control. Control theory tells us that to control an undefined process, you need to use frequent small adjustments. That means shortening the feedback loop as much as possible, and improving the system’s ability to respond to feedback. Both lean and agile focus on ways to do exactly that.

.

Lets look first at feedback. In design, feedback comes from generating information, learning from it, and adapting to it. That means getting the right info to the right people as early as possible and ensuring they can act on it. It means putting as much effort into surveys, tests and experiments as you do into the product: information gathering is a critical design activity and deserves respect.

.

It means working on cycle costs – the cheaper and faster it is to spit out a daft or prototype and get feedback on it, the more often you can do it and the faster and better you’ll learn. It means keeping the customer focussed on value not contracts: there’s no point in learning things if you can’t change course based on the new knowledge.

.

And it means keeping your options open. Once a decision is made it’s hard to unmake it. The right time to make irreversible decisions is as late as possible – at the last responsible moment. Planning needs to take this into account – plan like hell for the stuff you can predict, but don’t trap yourself into letting your plan make your decisions before you’re ready to make them.

.

.

Now lets talk about response. We’ve all been pushed to do stuff faster. Personally, I don’t believe design can be fundamentally sped up without compromises. Thinking takes as long as it takes, and if you try to do it faster you just get less of it. Maybe you’re lucky enough to cut out the low-value stuff, but mostly you just cut down quality. There’s a lot of evidence to show that the correlation between accelerated product development and profit is dodgy at best.

.

But there are argument for speed. Information is our stock in trade, and it gets lost, forgotten, and obsoleted. The photo here is the inside of a two year old Formula One engine. I took it – in public – about the same time Maclaren were being fined a hundred million bucks million for stealing one year old data. A hundred million to worthless in 1 year. F1 is an extreme case, but information depreciates fast.

.

.

What lean and agile show us is that to go faster, we shouldn’t focus on what we’re doing, but what we’re not.The leverage is in the gaps. There’s more time to be gained by keeping the work moving than by doing tasks faster. And the economics of information depreciation means it’s worth spending money on that. A job sitting waiting is bleeding money. We need to organize our processes around workflow.

.

.

A classic example of that is avoiding batch processing. Anyone been to the Coromandel? Remember the Kopu bridge? More importantly, anyone remember the two or three intersections after the Kopu bridge? Batches hold up the workflow – not just at the batchpoint, but downstream. They’re a convenient way to work – build it up till you’ve got enough to be worth doing – but they’re a bad way to work.

.

.

Another example is timeboxing. Keeping the job rolling to a rhythm lets feedback be incorprated smoothly and lets the players be ready when it comes their way. So instead of breaking it into chunks of work which take as long as they take, break it into chunks of time that achieve as much as they achieve. As long as you prioritize the important stuff within each time box, the 80/20 rule will work in your favour.

.

.

Another idea which I think has real value is extremism. Don’t do things by halves – take it to the limit one way or the other. In IT, finding bugs sooner is better. What’s the extreme of that? Finding them as they’re written. So Agilists put 2 people at one computer and do the dubugging simultaneously with the coding.What’s important in your process? And how do you turn the knobs on that up to 10?

.

.

Conversely, what can you turn right down? Lean gurus believe “if it’s not contributing value it’s waste”, and the agilists mantra is YAGNI – You Ain’t Gonna Need It. Elegance in design is the art of the minimum necessary. ISO9000 and suchlike encourage us to design processes like motorway bridges – we have to ask ourselves “how big a bridge do we REALLY need?”

.

.

I want to conclude by suggesting that design isn’t a machine, and we shouldn’t treat it as such. But neither is it a craft, and we shouldn’t treat it as that either. The design process is something that could benefit from some design, and, just as in the design of things, if we don’t take inspiration from outside our field, we’re letting ourselves down.

Cheap/fast/good

Ben Casnocha in his book My Startup Life makes the very good point that you can only have two out of three of the above, and it’s a good idea to pick which ones are important before you start.

In NPD, it’s complicated by the fact that cheap and good for the project aren’t necessarily the same for the product. So we have what I’m hereby dubbing the NPD hourglass. Pick two for each triangle but if you pick “fast” you need to pick it for both. A cheap fast project can give you a cheap product or a good product, not both. On the other hand a cheap good project can give you a cheap good product, but it won’t be fast.

My chain of thought so far expressed in 4 books

In order of how the logic develops:

Managing the Design Factory, by Don Reinertsen

The Toyota Product Development System, by Morgan & Liker

Lean Software Development, by Mary & Tom Poppendieck

Flexible Product Development, by Preston Smith

Zotero

This sort of thing is why open source is so cool.  Blows EndNote out of the water.

Design patterns

Semi-formed thought for the day:  are Design Patterns a useful variation on the Lean theme of ‘low level standardization for high-level freedom”  like Toyota’s use of design standards and tradeoff curves?

Characterisation of projects: part 3 – fitting methodologies to the framework

A while back I wrote about a characterization framework for projects, in which I differentiate projects based upon the level of uncertainty involved, and the potential consequences of that uncertainty. I’m hardly the first person to try and differentiate projects in some systematic way – I highlighted a few other efforts here.

In this post, I look at how some of the major project management methodologies fit on this framework. I use “project management” here in a much broader sense than conventional usage: any method for organizing and undertaking project tasks carries implications ofr how that process can or should be managed, so even methods which are normally thought of as being about execution rather than managing, are in a wider sense PM methods.

In a plan-driven methodology such as those advocated by PMI IPMA etc, project goals are identified and the necessary steps to achieve this goal determined. The steps are organized into an optimal sequence given resource and other constraints to form a project plan. This plan is then followed, with the objective of management being to administer the activities and to deal with deviations from the plan.

Considering this methodology on our framework, its effectiveness can be seen in the ideal case to be independent of the consequences axis. As the objective of planning is to anticipate and avoid surprises, their consequences are irrelevant. However the effectiveness of the plan driven methodology will vary along the uncertainty axis. The plan is inevitably based upon assumptions about the goal, methods, required effort and resource constraints among other things. As uncertainty increases these assumptions will be less valid. This can be compensated for to some extent by increased planning effort, however in the limiting case developing the plan itself becomes a dominant project task. The upshot is that as uncertainty increases, deviations from plan will increase, and the project will become less plan driven, and more driven by ad-hoc responses to surprises.

The plan-driven methodology is an example of an uncertainty-limited methodology, as shown in Figure 2.

Another such example is that of the problem structuring methodology. Problem structuring methods such as the Soft Systems Method (SSM) focus upon the definition of the objectives and environment of the project. Their effectiveness is again independent of the consequences axis, as implementation – and therefore the opportunity for consequences to occur – is assumed to be trivial. But effectiveness again varies along the uncertainty axis. As noted above, the relative importance of implementation and validating assumptions – or problem definition – changes. At low uncertainty levels, where the situation is relatively clear, “the process of inquiry into the problematic situation” [67 p S50] becomes a trivial one, and problem structuring methods become irrelevant. The problem structuring methodology is thus also uncertainty limited, but the limit is at the lower bound.

Not all methodologies are limited by uncertainty. The simplest project methodology is the ad hoc one, also known as “winging it” or “hacking”. In this methodology, no planning or preparation is undertaken, and the project team simply follow their noses and deal with problems as they arise. This methodology’s effectiveness is in the ideal case independent of uncertainty, as the lack of anticipation means that everything comes as a surprise – uncertainty is effectively maximum in all cases. However its effectiveness is highly dependent upon consequences: lack of any control system means that any event which the team is unable to handle is likely to quickly run away to disaster, and that it will not necessarily be known whether the event has been handled satisfactorily until too late. This methodology is thus restricted to situations where the teams ability to handle unexpected events is high (meaning extremely small and expert teams), and “disasters” cannot be overly serious: the methodology can be considered to be consequence limited.

A similar argument can be applied to the Agile methodology. This is similar to the ad hoc methodology in that uncertainty is dealt with by adaptation rather than anticipation. The technique is again limited in the consequence dimension by the team’s ability to cope with the unexpected (which is dictated by factors such team size, distribution, empowerment and competence), and by the magnitude of the worst case disaster. However techniques within the Agile methodology are largely aimed at providing control systems to mitigate these limitations, by maximizing team responsiveness and providing rapid feedback as to the effectiveness of responses. The upper consequence limit under which Agile techniques are effective is thus higher than for the ad hoc method, again as shown in Figure 2.

The limiting values shown in Figure 2 are arbitrary, and in fact a tapering off of effectiveness rather than a hard limit would be more realistic. However the graph shows clearly the respective “comfort zones” of the methodologies discussed, and areas of overlap and of lack of coverage.

In areas of overlap, the issue for methodology selection is not so much one of effectiveness as efficiency. For example in the area of relatively low consequences and uncertainty, both Agile and plan driven methodologies are effective. However the cost drivers of each are different. Plan driven methods impose significant overhead costs in documentation and control, although these an be minimized by “lightweight” implementations (see for example [68]).  On the other hand, as a “learning” approach Agile has a lower ideal efficiency than the “prescriptive” approach [69, 70], imposing risks of wasted work and suboptimal sequencing.  Which cost dominates will depend on the particular circumstances of the project in question.

Although projects with significant uncertainty and high consequences have been managed successfully, the graph shows a gap in this area between the methodologies considered. The high incidence of failure in large capital projects, particularly in the IT and construction fields [71-74] suggests that this is a combination which is not easily manageable using existing project design methodologies.

Ask the impossible….

I was recently approached by a previous employer to go back to them. All very flattering, though I’m not sure that it’s for me. But it got me thinking about why I left. Like most job changes I suspect, in loose (or polite) terms this one was driven by “philosophical differences with management”.

What that came down to in this instance was goal setting. Goals at this company were set very aggressively – during the time I was there I don’t think we ever hit all the goals of a project, and reading between the lines of annual reports since I’ve left, it looks like that record has been maintained. There are a lot of reasons why that might happen, but in the case of this company I think it’s because the CEO is a believer in the maxim “ask the impossible to achieve the best possible”.

That’s not something I buy into. To me, overly aggressive goal-setting is counterproductive.  It leads to rushed work, poor decision-making, and an inevitable descent into firefighting and resource shortages.  Products get launched half developed, which leads to poor sales growth, reputation damage, and still more rework. If developers don’t buy in to the possibility of management’s goals, they ignore them and start to set their own, which makes it that much harder for management to maintain staff morale (and to maintain their own credibility:  from personal experience, trying to persuade people that something that both you and they know is a dumb idea is actually a good one is no fun) .  Stress and the repeated sense of failure when goals aren’t met (even if the project turns out to be a success from a long-term business perspective) kill motivation, and lead to burnout and staff turnover, thus exacerbating the resource shortage problem.  Resource shortages lead to the low-priority infrastructure and forward research jobs never getting done, which in time reduces the organization’s innovative capacity enough that even just keeping up with the competition is a  stretch, never mind getting ahead.

You can get away with setting big hairy audacious goals every so often (though it’s worth noting that the original BHAG was supposed to be a vision statement – something so long term that the question of actually achieving it never really arises), and goal setting needs to be aggressive enough to avoid gold-plating and the student syndrome, but going for the big win as a matter of routine doesn’t work.

Good words: Value

This post – and the last one - are an attempt at making more sense of a somewhat incoherent comment I made on Glen Alleman’s Herding Cats blog. 

I gave the background in my last post.  The second thought Glen’s post raised was re definitions of value.  Value is one of those words that has a different definition every time you look at it.  Glen seems to be basically arguing that the value (as determined by Earned Value or equivalent) delivered by incremental milestones under DoD processes is equivalent to the value delivered by iterative releases under Agile ones, in that they both provide “business value”. 

My take on that is that it’s true as far as it goes, but it assumes that there is no utility value in the intermediate deliverables.  In a lot of cases that’s true – it’s hard to see any partial implementation of an aircraft  flight control system being of much commercial use, for example.  But it’s not always the case.  For a lot of desktop applications a partial system can be useful once it reaches the stage of “better than what we’ve got now”, and that can happen pretty early in the piece, depending how crappy the existing system is.  If a business can be using the new one while you’re still developing it, they’re capturing extra value that they wouldn’t have got under a methdology that didn’t require early releases, whether you’re using EV or not.

I think where Glen was coming from though was information value - the value of confidence.  And that value is solid financial value – I think the Agile community is overly aggressive if they dismiss it completely.  Options have value, and more information gives you more options earlier.  Both EV milestones and Agile iterative releases support this by only counting deliverables when you can definitely say “yes, that’s done”, thus giving a more concrete indication to the customer of how close they are getting to having what they want, relative to how much they’ve spent on it.  Which neither a conventional but non-EV project or a highly-iterative project without intermediate releases achieves to the same extent. 

So yes, EV type approaches give business value incrementally,  but only if you’re prepared to define value differently from how agile does (which I am.  I prefer the lean approach – you don’t get to decide what constitutes value, your customer does).  And, of course, if you’re prepared to assume the plan is solid enough that it’ll withstand something coming out of left field – without that, progress against plan is meaningless, however well measured.  That one, I’m less prepared to take as a given.

Bad words: BDUF

This post – and the next one – are an attempt at making more sense of a somewhat incoherent comment I made on Glen Alleman’s Herding Cats blog.

Glen, who comes at the agile vs conventional discussion from a large-scale DoD/aerospace perspective, was arguing that modern DoD processes have incremental maturity milestones, which via earned value methods give early delivery of ”business value” equivalent to the agile concept of rapid cycles of “releasable” software.

All of which is good stuff – I don’t have any problem at all with phased processes for big projects, and for something that size you’d damn well better have some way of confirming along the way that you really are where you think you are – EV is a good approach to this. 

His post gave rise to a couple of thoughts which seemed worth sharing though, and the first of them is:

Don’t let the other side own the vocabulary

As soon as the other side of an argument gets to define the words, you’ve lost.  If X = Bad, where X is anything that looks even vaguely like what you’re advocating, then you’re reduced to saying “but my system isn’t X”.  Which is pretty much how Glen opens his post.  And that puts you on the defensive, and devalues whatever point it is you’re actually trying to make.

 And in this case, it’s unnecessary.  Firstly, BDUF=Bad is false.  There’s nothing inherently wrong with a BDUF approach.  The single plan-do loop has its place – in sufficiently predictable circumstances, it’s a highly efficient approach.  The problem has always been that “sufficiently predictable circumstances” are a rare beast – particularly in the software industry – and BDUF is applied in way too many circumstances.  But selecting an appropriate methodology is a management function.  If the wrong one is used, that’s a management failure, not a methodology one – even back in the 1970s, alternative approaches were available.  BDUF=Bad in this context is just a way of saying “Bad management decisons = Bad”, which isn’t something that needs a great deal of debate.

 Secondly, BDUF as the idealised agile bogeyman of absolute adherence to a linear plan isn’t even a good implementation of BDUF.  If you’re allowed to use variations on “if…then” and “repeat…until” in writing software, you’re allowed to use them in planning it.  Allow the planning part of a single-loop plan-do approach to have some risk management, contingency buffers, and iteration - all of which have been “conventional” project management staples since way back when - and you have an efficient and robust process which is effective across a reasonable range of projects.  Not all of them, by any means, but enough to make it a useful methodology.  So in this context, BDUF=Bad reduces to “Bad BDUF=Bad”, which again is true but not very useful.

Unfortunately, there are lots of bad high-level managers, and bad project managers out there.  So both “bad management=bad” and “bad BDUF=bad” provide plenty of examples to reinforce the “BDUF=Bad” argument.  But that doesn’t mean the conventional school needs to buy into it.  And if you don’t buy into BDUF=Bad, the argument then stops being about ‘is this approach BDUF” and can be purely about “does this approach work in its context, and why?  And more importantly, can any of it work in MY context“.  Which in my view is where it should be.

More Zen: another go at the “is testing waste” question

Having commented not 5 minutes ago that discussions like this weren’t very useful, I’m going to continue this one.  That’s what happens when your mind is going 20 to the dozen at 3am for no good reason.

I think the is testing waste question, at least for whatever variation of testing you’re thinking of right now, can be answered by this simple question:  What is your instinctive gut reaction to testing?

1. We should do more of it
2. We should do less of it
3. We should do the minimum necessary to serve the purpose.

If (1), it’s value added, if (2) it’s type 2 waste, if (3) it’s type 1 waste.

An alternative viewpoint is from the perspective of “what type of testing are we talking about?”.  As I see it there are basically 3 types of testing:

1. Testing as exploration – “What will happen if I try this?” Or, more commonly “I just tried this.  Lets find out if it worked”
2. Testing as verification – “I did the right thing.  Did I do it right?”
3. Testing  that is vanishingly unlikely to yield useful information – for example testing so someone higher up the food chain can tell someone even higher up the food chain “yes, we did a million miles testing on that”.  Or marketing can tell it to the customer, or whatever.   What we used to call in the auto industry “Miles for Smiles”.

(1) is value added, provided the “this” you just tried is value added.  A known good solution is worth more than a hypothesis.  (2) is type 1 waste (as long as there is a significant possibility that the answer is “no”, otherwise it’s the same as (3)).  You are not increasing customer value, you’re just avoiding reducing it.  (3) is type 2 waste.  Stop doing it.  Or at least be explicitly aware that it’s a marketing exercise (even if it’s your boss selling to his boss) not an engineering one.  Interestingly, the more you do of (1), the more (2) becomes (3).

 Or, like I said in the first place, “It depends”.