Ecosystem Advantage

Mark O'Neill (Vordel) discusses a couple of SOA projects with an interesting goal - causing your customers to use less of your products. This is not competitive advantage, at least not directly, it is an advantage for the ecosystem as a whole, which becomes more efficient, less wasteful.

Mark's examples are in the utility sector

• Gridwise - a US consortium of energy companies and technology companies
  
  • Pacific Northwest National Laboratory aims to empower utility customers (IBM April 2007)
    
  • How SOA could change the way you buy electricity (Network World, Oct 2007)
    

• EPAL - the Portuguese water board (one of Mark's own clients)
  

Similar thinking applies to the "pay-as-you-drive" concept in insurance - which I discussed here - encouraging drivers to use (and pay for) less risk.

These schemes are potentially very attractive. There are three main challenges here.

• Identification - analyzing an ecosystem systematically to discover opportunities for creating or releasing additional value

• Mobilization - aligning the ecosystem to the new improved distribution of value - easier where there is a single powerful player or industry regulator that can drive and enforce (and perhaps fund) the initiative until it becomes self-sustaining, otherwise more difficult

• Ecosystem side-effects - working out satisfactory multilateral solutions to complications such as privacy and security
  

Squidoo Lens: Service Engineering

Outside-In Architecture

Among many other interesting topics in his latest post (Unanswered Questions ...), John Hagel distinguishes between "inside-out" and "outside-in" architectures.

For Hagel, inside-out architectures are those that have evolved in many large organizations over the past couple of decades: starting from centralized IT architectures, gradually embracing distributed computing (client-server) and SOA, and very cautiously moving towards B2B. Such architectures are generally transactional, and controlled ("held captive" says Hagel) by IT architects.

A key challenge for inside-out architectures is the ability to connect and coordinate activities across large numbers of independent business partners, and to scale up to large and highly complex service-based ecosystems. Hagel believes that outside-in architectures will provide better support for complex collaborative ecosystems. He calls this relational - not using this term in the traditional IT sense ("relational databases"), but focused on supporting business relationships (and long-term business transactions) rather than short-term transactions.

The term "outside-in" seems to have a range of overlapping meanings, including:

• Looking at your business from the customer's perspective. There are several consultancies that claim to offer this as a service. For example, here's a curious page on the Outside-In company - with painfully bad graphic design and a revealing spelling mistake ("ourside in").

• Business-driven systems. For example, Zapthink uses the term to refer to "flexible business processes that respond to the way humans work, rather than processes that constrain humans to work the way the systems want them to work".

• Strategic Outsourcing. See for example What India and China can teach us about Innovation by Hagel & Seely-Brown.
  

• Outside-in SOA. This term seems to have been coined by Dave Linthicum, and refers mainly to interaction with external services (SaaS). See also Chuck Allen.
  

These are all very interesting trends to be sure, but is there a deeper pattern underlying some or all of them?

• Some of them are largely about extending the scope of the solution - finding new (more cost-efficient, more flexible) ways of satisfying the same old requirements. I think strategic outsourcing and SaaS come under this heading.

• Some of them are about extending the scope and perspective of the requirements - trying to add value to the customer's process, not just your own internal processes.
  

• And some of them hint at the need to reframe architecture itself - looking at the positive space between systems and organizations, not just the systems themselves.
  

Hagel avers that outside-in architectures must be "designed from the outset to support sustained collaboration". I agree that these opportunities call for a new kind of architectural thinking, and I agree that it may not always be easy to make the transition from existing inside-out architectures; but I think the evolutionary path is possible, given sufficient business motivation.

Squidoo Lens: Service Engineering

Service Ecosystem and Market Forces

One of the problems with a network of services is that the responsibilities and costs and risks are often in the wrong place.

In this post I'm going to explain what I mean by this statement, outline some of the difficulties, and then make some modest proposals.

The statement is based on a notion of the efficiency of an ecosystem. If there is one service provider and a thousand service consumers, it may be more efficient for the ecosystem as a whole if the service provider includes some particular capability or responsibility within the service, instead of each service consumer having to do this. In addition to the economics of scale, there may be economics of governance - for example, increased costs of managing the service relationship, especially if the service provider doesn't provide a complete service (in some sense).

One important application of this idea is in security, risk and liability. There is a very good discussion of this in the recent British House of Lords Science and Technology Committee Report into “Personal Internet Security", who specifically address the question whether ISPs and banks should take greater responsibility for the online security of their customers.

"A lot of people, notably the ISPs and the Government, dumped a lot of the responsibility onto individuals, which neatly avoided them having to shoulder very much themselves. But individuals are just not well-informed enough to understand the security implications of their actions, and although it’s desirable that they aren’t encouraged to do dumb things, most of the time they’re not in a position to know if an action is dumb or not." [via LightBlueTouchpaper]

In other words, the responsibility should be placed with the player who has (or may reasonably be expected to have) the greatest knowledge and power to do something about it. In many cases, this is the service provider. Some of us have been arguing this point for a long time - see for example my post on the Finance Industry View of Security (June 2004).

Similar arguments may apply to self-service. When self-service is done well, it provides huge benefits of flexibility and availability. When self-service is done poorly, it merely imposes additional effort and complexity. (Typical example via Telepocalypse). Some service providers seem to regard self-service primarily as a way of reducing their own costs, and do not seem much concerned about the amount of frustration experienced by users. (And this kind of thing doesn't just apply to end-consumers - similar considerations often apply between business partners.)

But it's all very well saying that the service provider ought to do X and the service consumer ought to do Y. What if there is no immediate incentive for the service provider to adopt this analysis? There are two likely responses.

1. "We don't agree with your analysis. Our analysis shows that the service consumer ought to do X."
2. "We agree it might be better if service providers always did X. But our competitors aren't doing X, and we don't want to put ourselves at a disadvantage."
   

More fundamentally, there may be a challenge to the possibility of making any prescriptive judgements about what ought to happen in a complex service ecoystem. This challenge is based on the assertion that such judgements are always relative to some scope and perspective, and can easily be disputed by anyone who scopes the problem differently, or takes a different stakeholder position.

Another fundamental challenge is based on the assertion that in an open competitive market, the market is always right. So if some arrangement is economically inefficient, it will sooner or later be replaced by some other arrangement that is economically superior. On this view, regulation can only really achieve two things: speed this process up, or slow it down.

But does this mean we have to give up architecture in despair - simply let market forces take their course? One of the essential characteristics of an open distributed world is that there is no central design architectural authority. Each organization within the ecosystem may have people trying to exercise some architectural judgement, but the overall outcome is the result of complex interplay between them.

How this interplay works, whether it is primarily driven by economics or by politics, is a question of governance. We need to spell out a (federated?) process for resolving architectural questions in an efficient, agile and equitable manner. This is where IT governance looks more than ever like town planning.

Notes

The House of Lords Science and Technology Committee Report into “Personal Internet Security" was published on August 10th 2007 (html, pdf). Richard Clayton , who was a specialist adviser to the committee, provides a good summary on his blog. Further comments by Bruce Schneier and Chris Walsh.

Squidoo Lens: Service Engineering

Business Geometry

A business or value chain is composed in a geometric structure. (In the past we have often called this structure an architecture, but this word has lots of different and tangential associations for different people.)

In SOA, we design a business or value chain as a network of services. This is a powerful geometrical pattern. But there may be many possible network geometries satisfying a given business requirement, all of which count as satisfying the principles of SOA. For example: hub/spoke or peer/peer.

Business Stack

Another common geometric pattern for SOA is stratification. A business process is composed of services from a set of lower-level services, presented as a platform. A good example of a business platform is the set of retail services offered by Amazon and eBay. Other service providers have built further retail/logistical services on top of the Amazon/eBay platforms.

Each platform is in turn built upon even lower services. At the lower levels, there may be collections of IT-based services, known as ESB. There may also be sociotechnical service platforms, such as call centres.

Thus we have a stratified geometry, in which a person tackling a problem at a given level is presented with a collection of available services, formed into a virtual platform. This can be thought of as a business stack, with one plaform stacked on top of another. Some of the lower layers of the stack may appear to be purely technical services; however a more complete picture should reveal the existence of an IT organization maintaining the platform, in other words a human platform of administrators and programmers supporting the technical platform.

Variable Geometry, Variable Granularity

While the SOA principles may provide some geometrical guidance, and mandate certain geometrical patterns, there is still a design job to determine the geometry. This design job may be easy when the requirement is trivial, but gets harder as the complexity increases.
In many situations, the demand side has more variation than a human designer (or design team) can accommodate. (We characterize this as an asymmetry of demand, which calls for a process of asymmetric design.) We need to start thinking about variable geometry solutions, where the geometry itself can be adapted on-demand.

more on asymmetry
asymmetry@technorati

For example, in the past we have assumed that granularity has to be fixed at design time. But we can conceive of a web service platform that detects patterns of demand-side composition, defines new composite services automatically, describes and publishes these new services in real time, and notifies likely users of the new service, complete with an incentive to switch. We can conceive of a web service platform that analyses the message content of a certain service, and produces a substitute service with a smaller footprint that would satisfy most of the uses in a more elegant way.

Value Landscape

I use the term value landscape to refer to the distribution of cost, benefit and risk across a complex market ecosystem, such as the insurance industry. Technology (including SOA) influences business geometry, not least because it affects transaction costs. The shape of the value landscape changes (has already started to change) as the result of B2B and B2C and BPO. Companies that once occupied safe market positions (the high ground) may find their commercial advantage slipping into the sea, or they may find themselves cut off from their natural markets or supply chains.

See Microsoft Blog Insurance Value Chain

Let's suppose an insurance company has the following strategic aims:

1. Profitability, short-term viability. To deliver the maximum service value as cost-effectively as possible, using available input services and technologies as efficiently as possible, with minimum costs/risks of change.
2. Adaptability, medium-term viability. To understand and respond to changing demand for insurance services, and to trends in cost and risk, both internally and across the industry. To develop and deploy new services to exploit new business opportunities and avoid emerging business threats.
3. Survival, long-term viability. Making sure the core business proposition remains valid, and doesn't get eroded by more agile players. Taking strategic action in relation to structural changes in the insurance industry.

If we are doing business geometry for an insurance company, we need to think about the insurance industry as an ecosystem. We need an AS-IS model of the present ecosystem (largely based on pre-SOA technologies) and a TO-BE model of an idealized ecosystem (based on SOA). We can expect the pre-SOA ecosystem to evolve into some form of SOA ecosystem, although we may not have much idea which of the possible changes is going to happen first. To satisfy all three strategic aims, an insurance company needs to exploit the pre-SOA ecosystem, and prepare for the SOA ecosystem.

Note that this situation may force the insurance company to implement a variable geometry, both in the organizational platform and in the IT platform. Otherwise, it will either have to operate suboptimally for an extended period, or incur significant organization costs and IT costs every time the industry takes another step towards SOA.

Amazon and Ebay

Amazon, eBay and PayPal are creating platforms for eCommerce.

• Amazon Web Services
  
• eBay Developers Programme
• PayPal Merchant Tools
  

• Reprogramming Amazon (Business Week, December 2003)
• Amazon CTO: 'We've Just Scratched The Surface' (Information Week, July 26, 2004)

At CBDI Forum, we have been tracking these developments from time to time. See news commentaries: July 2004, February 2004, February 2004, July 2003. See also Jeff Bezos and Ecosystem Thinking.

---

Jeffrey McManus of eBay claimed recently that 'we make inefficient markets efficient'. (Source: Aaron Johnson blog.)

In his blog, Jonathan Schwartz (Sun Microsystems) talks about selling computers on eBay, as a web services experiment. Schwartz expects this experiment to provide "unvarnished" pricing information, as well as providing practical experience with a service-oriented supply chain. This seems to support the McManus claim.

However, efficiency often conflicts with adaptability. How wise is it for Sun and other firms to link to the eBay platform? What is the likely ecosystem effect of allowing eCommerce services to be defined and controlled by a small number of like-minded firms?

I've just had a look at the eBay technical documentation. Developers must follow detailed procedures for design, testing and certification, before they are permitted to plug into the live eBay environment. Rightly so no doubt, but this raises questions about service life cycle management and technical change.

In response to the possible difficulties of raw eBay, a number of service providers are now offering DropOff services that provide a simple eBay front-end. These include AuctionDrop, AuctionWagon, QuikDrop and Picture It Sold! and 1StopAuctions. The last of these makes strong and explicit claims about the difficulty of using eBay. (Success here means not just completing the transaction but getting a good price.)

"To successfully sell an item on eBay requires marketing and selling strategies unique to eBay – and learning those strategies can take years to master. 1Stopauctions.com has that unique knowledge to successfully sell your items on eBay. We have been buying and selling on eBay since 1998. Over the years we have found the secrets of launching successful listings to obtain the highest possible price ... and so on ..."

For me, the appearance of these secondary service providers simply confirms that the big issue is business hegemony (power) - the value ladder now belongs to Amazon and eBay, they control the terms on which everyone else is trading.

Trusting Services

If there is only one implementation of a service then trusting the service entails trusting the implementation entails trusting the supplier. Furthermore, if there is only one channel to access this implementation, then this also entails trusting all intermediaries.

Typically this is done by having a large trusted supplier with a reputable brand. IBM or EDS. Large suppliers have a valuable brand image to maintain, and therefore may be supposed to have too much at stake to allow any service to fail.

We can break into this loop in two places. Firstly, we may be able to decouple the implementation from the supplier, through some kind of guaranteed escrow. If the supplier cannot deliver the service with the specified level of quality, then we or someone else will immediately take over the implementation (together with all resources required, such as persistent data). We should expect the web service platforms to support this.

More importantly, we should be able to decouple the service from a single implementation. It might be stupid to rely on a business critical service if there is only one small supplier. But if there are lots of interchangeable suppliers offering equivalent services, the disappearance of
one supplier doesn't matter very much.

Thus web services allows us to shift from trusting a single source to trusting multiple sources (market or ecosystem). Without this step, the service-based business contains many dependencies, and the business process may seem critically vulnerable.

The process implications of this are twofold. Firstly, the SOA process should include some consideration of the appropriate number/diversity of implementations of a given process - and should certainly not assume that the design task is always to select a single best-possible implementation. Secondly, the enterprise model of an SOA solution should allow some visibility and management of the true dependencies and consequent risk involved, since multiple independent implementations of a given service will generally be expected to increase the overall availability and reduce the overall risk.

Jeff Bezos and Ecosystem Thinking

Lots of commentators and blogs have picked up on a recent Business Week story about Amazon (December 22, 2003).

link corrected - sorry for previous error

But Amazon has been working towards this for a long time indeed. When I searched the web for "Bezos" and "ecosystem", the first page of hits included a story from June 2000, in which a senior manager at HP acknowledged that he had gotten ecosystem thinking from talking to Jeff Bezos.

http://www.crn.com/sections/BreakingNews/dailyarchives.asp?ArticleID=17530

Recent developments by Amazon and eBay (as described by David Sprott at the CBDi Forum) establish a business service stack for the eCommerce sector.

But this isn't just a clever tactical move. It follows logically from a strategic direction that Amazon has been pursuing for a considerable time. During the dot-Com boom, there was lots of superficial admiration of Amazon, and lots of companies trying to emulate it. But few of them mastered ecosystem thinking, and few of them survived the downturn.

For more on ecosystem thinking, see my 2001 book on Component-Based Business. Available from Amazon (of course).

More on Amazon and Ebay.
More on Jeff Bezos. Extract from the Book of Bezos.