Blueprints 2

Reposting my comment to Nick Malik's post on Blueprints, following my previous post on Blueprints. Nick asked me to spell out the implications of my previous comment - whether I meant that we don't need accuracy, or that we should start measuring - and challenged the strong association I implied between accuracy and measurement.

Firstly, let me affirm that I think measurement (in the broadest sense) is a good idea.

I am not sure I know what accuracy means except in terms of measurement. How can we reason about things like "structural integrity" and "quality" without some form of measurement? In engineering, we don't generally expect perfect integrity or perfect quality (which is usually either physically impossible or economically non-viable); we look for arguments of the form "X produces greater integrity/quality than Y" - where X/Y is some choice of material or technique or pattern or whatever. So there are implicit metrics running through all branches of engineering. Software engineering just isn't very good (and should be much better) at managing these metrics and making them explicit. As a result, we don't always see software engineers delivering the maximum integrity and quality for the minimum cost and effort.

So when I'm talking about measurement, I'm certainly not only interested in cost estimation and other project management stuff. I think architects should be thinking about things like the amount of complexity, the degree of coupling, the scale of integration, and you certainly can't read these quantities straight from a UML diagram.

Of course a building blueprint doesn't tell you everything you need either. If you are designing an airport, the blueprint will show how much flooring you need, but will not show whether there is enough space for the passengers to queue for passport control. If you are designing a tower block, you have to have some way of working out how many lifts to put in. In software engineering this kind of stuff is dismissed as non-functional requirements.

All engineering involves estimation. "is this bridge going to fall down" is an estimate.

In a traditional waterfall development, many people thought it was a good idea to address the functional requirements first (logical design), and then tweak the design (physical design) until you satisfied the non-functional requirements as well. But when you are composing solutions from prebuilt enterprise services, this approach just doesn't wash. Indeed, it may now be the other way around: if a service assembly fails some functional requirement, you may be able to plug/mash in some additional service to fill the gap; but if it fails the non-functional requirements you may have to throw the whole thing away and start again.

Finally, I don't say only big projects need accuracy. If a government builds a tiny service to be used by the entire population, a small project might have a massive impact. A garden shed may not need a professional architect: that's not because a garden shed doesn't need accuracy, but because an amateur builder can work out the quantities accurately enough herself.

Blueprints

Nick Malik has prompted a great discussion on the difference between accuracy in architecture and IT. He asks why IT architects don't produce blueprints that are as accurate as those produced by architects in the traditional world of physical construction.

The kind of accuracy Nick describes in traditional architecture is about quantity. The costs of a building are largely determined by the physical dimensions. (The cost of the carpet depends on the floor area.) So the first person who looks at the blueprint is not the builder but the quantity surveyor. The blueprint has to be good enough to enable reasonably accurate cost estimation.

We don't usually do that in IT. There is no How-Many/How-Much column in the Zachman framework. You can't work out quantities from a UML diagram. In a pre-SOA world, we thought cost estimation was largely about counting the number of components to be constructed (simple, medium, complex) and putting them into a time/effort formula. But this approach to cost-estimation is increasingly irrelevant to SOA.

If you are only building a garden shed then you possibly don't need a professional architect or surveyor. If you are building a tower block then you certainly do. The people who are doing serious architecture in an SOA world are those operating at internet scale - for example redesigning Skype so that it doesn't fall over on Patch Tuesday (see Skype Skuppered).

Squidoo Lens: Service Engineering

Scale and Self-Organization

Werner Vogels, CTO of Amazon, knows a thing or two about implementing large-scale distributed SOA. So it's worth taking seriously when he tells us that

"For any truly scalable agile environment, self-organization is essential."

In his post Reading References, he goes on to recommend some papers and books on scalability and self-organization.

• Richard Conniff, The Limits of the Alpha Male
• Steve Grand, Creation: Life and How to Make It
  

My own favourite book on self-organization remains Kevin Kelly's book Out of Control, now available online on KK's website. See also the entry on Self-Organization in Principia Cybernetica.

If we accept that self-organization can be effective for addressing some classes of complex problem, next step is to develop practical methods for engineering self-organizing systems. There are some promising research projects in this area, and there's a conference in Leibzig in September (SOAS 2007). I probably won't have time to attend the conference myself, but I shall look out for the proceedings. I wonder whether Amazon will be represented?

Squidoo Lens: Service Engineering, Self-Organization