14 days ago

Documenting the Not-known

Link: http://weblog.tetradian.com/2018/05/06/documenting-the-not-known/

In our work on simplifying the ‘inventory’ of tools and techniques I’d developed for whole-enterprise architecture and the like, one of the challenges we keep hitting up against is how to capture ideas, information and insights, as they appear during the necessary ‘explorations into the Not-known‘.

The problem itself is quite well described in my old 2011 post ‘Models as decision-records (Enterprise Canvas)‘. Conventionally, we collect information on a text-based form, either on paper or via some kind of app or screen. (The ludicrously-tiny text-windows of one of the more infamous ‘enterprise’-architecture toolsets come to mind at this point…) But there’s no real guidance, and no easy way to handle the sheer messiness of any real-world exploration, with ideas and insights spinning off in all different directions, and nowhere on the form that actually covers what we need to record. But on the other side, if there’s no way to structure anything, what we end up with is at best a set of bullet-points, but more likely a swathe of whiteboard-scrawls that make no sense to anyone even a week or two later, or maybe even at that time itself. So what do we do to help people make sense over time?

Here’s our challenge. We’re building a frame and method in which we split the overall exploration-process across five distinct domains – the Five Elements, shown, in the visual metaphor that we’re using, as five rooms or departments arranged around a central collation/curation-space:

In each ‘room’ we use various tools as ‘visual checklists’, to guide exploration – much as described in the ‘Models as decision-records’ post, and in more depth in the post ‘Tools for change: Back to the basics‘. But how do we capture the ideas, insights and information that arise during that process? The ‘obvious’ answer is some kind of form – or a ‘canvas’, maybe. But that brings us straight back to the problems outlined earlier above: there’s a tradeoff between not-enough-structure and too-much-structure. So how do we find the Goldilocks-point of just-enough-structure – enough structure to help, but without getting in the way? There doesn’t seem to be any easy answer – not yet, anyway.

The catch is that the ‘just-enough’ seems to vary according to where we are in the exploration-process, even within each individual ‘room’. In the Five Elements model, we’d tried to simplify it right down to having just one form for basic capture, and anything else would be brought in from the plug-in tools. But what we’ve found is that it doesn’t seem to work: a setup that gives us exactly the right ‘just-enough-structure’ at one part of the process is completely wrong for later on.

The catch seems to be our good old friend/enemy ‘the Squiggle‘, by Damien Newman:

The ‘Research’ phase in the Squiggle is full of uncertainty; whereas the ‘Design’ phase needs clarity and focus. Straight away there’s an inherent clash, right there: the ‘just-enough-structure’ tradeoff is going to be different between those two spaces, pretty much by definition.

But if we think the Squiggle is a one-off – we do it once, and then it’s done – then it might seem that it kinda doesn’t matter that much what we do with that information that we’d capture in the ‘Research’ phase, or even the Concepts/Prototype’ phase. It’s over, it’s done, we don’t need it any more. All we need is the final output, suitably structured, that we can pass on to the next stage of the work – presumably Deployment and Production or Operations, in most cases. That’s how it might seem, anyway.

Yet the catch there is that, in any iterative development-process, the Squiggle is actually a loop:

Which means that we do need to keep those previous Research-phase materials. Because if we don’t, we’ll have to do the whole new Research-phase from scratch, with nothing to compare it to, other than what little we have from the previous Design-phase – and that for a previous ‘solution’ which, if it’s dealing with a typical ‘wicked-problem‘, may well be significantly different from what we face right now.

Which in turn means that we do need to work more explicitly with that ‘just-enough-structure’ tradeoff – because if we don’t, we won’t have the right structure for the materials for information-capture in the Research-phase.

Or, for that matter, for the very start of the Research-phase, which is bringing in the structured-information needed to trigger off the Research-phase, which will typically draw on an end-of-iteration review after the previous Design-phase was complete. And that start-of-Research-phase document (think of a project-start form, for example), itself has its own distinct ‘just-enough-structure’ tradeoff – and will be different from either that in the maelstrom of the Research-phase, or the more structured needs for near the end of the Design-phase.

Which means we’ll need not just one document-type, but at least two, and more probably three, to cover those distinct phases of the Squiggle. And the Research-phase again is likely to have a multitude of different document-types, which, although they’ll share a similar ‘just-enough-structure’ tradeoff, will all be different in terms of content. (The ‘inventory’ should make that point clear – or, perhaps even better, see Andi Roberts’ excellent collection of ‘canvas’ visual-templates for business-explorations and the like.)

So to bring it back to our current work on Five Elements, we’d originally thought that we could get away with just one information-capture form for each ‘room’ of phase in that cycle. But it’s become clear that, as per the above, we’re going to need at least two different types of form, and possibly three, each with different answers to that tradeoff of ‘just-enough-structure’:

  • a tightly-structured ‘phase-start’ document, filled out in the Atrium and brought into a room (may be optional, if it’s essentially a copy of the ‘phase-end’ form from the previous phase)
  • a loosely-structured ‘mid-phase’ document – some kind of ‘canvas’ or similar visual-checklist that will trigger ideas and insights (there may be any number of different types of these, but the overall principle is much the same for each)
  • a tightly-structured ‘phase-end’ document, that summarises the information found, in a form that guides the review prior to starting the next Five Elements phase

Which also should remind us about the limitations of ‘canvas’-type forms: in most cases they’re great for the ‘messy’ work of the Squiggle’s Research-phase, but not so good for the more structured needs at the end of the Squiggle. And the problem is that too many of them try to tackle both requirements – and end up satisfying neither, because they get the ‘just-enough-structure’ tradeoff wrong for either of those two types of need. Which doesn’t help…

So how do you address this challenge? What do you do to capture ideas, insights and information from the Not-known? And how do you bring it down to a form that is actually structured-enough for others to use, in design and production?

Over to you for your comments on this, if you would?