Scale and Self-Organization

Werner Vogels, CTO of Amazon, knows a thing or two about implementing large-scale distributed SOA. So it's worth taking seriously when he tells us that

"For any truly scalable agile environment, self-organization is essential."

In his post Reading References, he goes on to recommend some papers and books on scalability and self-organization.

• Richard Conniff, The Limits of the Alpha Male
• Steve Grand, Creation: Life and How to Make It
  

My own favourite book on self-organization remains Kevin Kelly's book Out of Control, now available online on KK's website. See also the entry on Self-Organization in Principia Cybernetica.

If we accept that self-organization can be effective for addressing some classes of complex problem, next step is to develop practical methods for engineering self-organizing systems. There are some promising research projects in this area, and there's a conference in Leibzig in September (SOAS 2007). I probably won't have time to attend the conference myself, but I shall look out for the proceedings. I wonder whether Amazon will be represented?

Squidoo Lens: Service Engineering, Self-Organization

Clouds and Clocks 2

Pat Helland regrets ...

Pat Helland has resumed blogging, having recently returned to Microsoft from a sojourn at Amazon. In his latest post SOA and Newton's Universe, he renounces the quasi-Newtonian paradigm of distributed systems to which he adhered for most of his 30-year career, and outlines an alternative paradigm with some resemblance to the Special Theory of Relativity.

The Newtonian paradigm of distributed systems is that we are trying to make many systems appear as the One System. This paradigm may be linked with the idea of the Global Schema or Universal Ontology. Helland contrasts this with an alternative paradigm of distributed systems, in which the systems are entirely dependent on the point of view of the observer, and there is no Universal Ontology. (I'd have wanted to use the term relativistic semantics here, but it has already been bagsied for academic linguistics - see for example Catfood and Buses.)

Helland sees this in terms of a relaxation of consistency. I disagree. Distributed systems-of-systems (including SOA) must follow a consistent logic - but not necessarily a traditional two-valued logic. Flexibility comes from being underdetermined (clouds) rather than overdetermined (clocks).

See my earlier posts Beyond Binary Logic and On Clouds and Clocks. See also Philip Boxer on Modelling Structure-Determining Processes.