Virtual Endpoints: A Loose Definition for Loose Coupling?

I listened to a podcast this week by Miko Matsumura, Deputy CTO of Software AG. His comments on Governance were interesting especially the concept of hot and cold parts of the architecture. Some things like policies should remain "cool" and not change often. Other parts such as how you combine your services into applications should be relatively "warm" and change more often.

The point that jumped out at me was his very concise descript of loose coupling "Loose Coupling has virtualized endpoints". No further definition was given.

This must be a bit of a Software AG thing. John Michelsen, also of Software AG says

…virtual endpoints are a great idea as you think about the scalability and the loose coupling we want in a SOA.

This is better explained in John's posting "An Intro to SOA and Virtualization" but all this virtualisation seems to be is getting rid of hard coded URLs. This does not seem terribly revolutionary and is certainly not synonymous with the concept of loose coupling that is, according to some writers, the whole reason behind the benefits of SOA.

Before I discount this as an overly narrow vendor view I would like to explore the concept a bit more deeply. If I look down my ever-increasing list coupling types I can see a number of properties of service interfaces. I define and endpoint as being a service interface then we can see an endpoint has having the following properties and we can see some corresponding coupling types:

Endpoint Property	Coupling Type (From previous posting)
URL	Registry Coupling
Platform	Platform Coupling
Interface Time	Queue Coupling
Processor Location	Distributed Coupling
Data Field Order & Presence	XML Message Payload Coupling

The only property that may need some explanation is "Interface Time". This is the time that a message leaves from or arrives at a service interface. If no queuing is involved then the only delay between the message leaving from a sending service and arriving at the receiving service is some network latency. If a queue is involved then the delay uncouples the sending time from the receiving time.

The concept of an endpoint does not have much meaning except where messaging is involved. There are some types of coupling in the messaging world that are not covered by the concept of virtualised endpoints. Coupling based on process state (including synchronous coupling) and coupling introduced to manage process control, such as transaction control is not easily expressed in terms of endpoint properties. Logical coupling where service behaviour changes significantly based Boolean or enumerated field values passed in the message also is not covered as an endpoint property.

The "Virtual Endpoint" concept turns out to be a fairly useful. We have some types of coupling that are only dependant on endpoints (or service interfaces) and others that depend more deeply on the implementation of the services or the interpretation of the messages being passed. Virtual endpoints are not the whole story when it comes to loose coupling but are part of it.

Subroutine Call Coupling

It has been more than a month since I did my last posting on coupling. Amazon has delivered my book on loose coupling by Doug Kaye and I am finding it valuable. Once I have finished that I hope to revise my original list of types of coupling.

In this posting I would like to say something about subroutine call semantics versus message passing and the coupling associated with this. I have discussed before that the synchronous approach which is a requirement of the subroutine call semantics is more tightly coupled than an asynchronous approach. There are two other differences between subroutine calling and message passing. These are

• the management of state of the calling module and the called module; and
• the management of the called instance itself.

When you invoke a subroutine it starts executing from its beginning. The invoked routine will usually have no state and complete before the invoking routine resumes again. Another form of routine call is the coroutine where the state in the called module is maintained from one call to another. The issue of state coupling was covered in my earlier posting and labelled as 'Process State Coupling'. This type coupling is intended to cover coroutine behaviour. A coroutine is even more tightly coupled than a normal subroutine call because of the dependency on state.

In a normal subroutine call state is not preserved in the called module from one call to the next however state is preserved in the calling module. It preserves its state by the simple expedient of suspending execution until the called module returns control.

Asynchronous message can still involve process state coupling. More loosely coupled messaging has enough information in the message to process without retaining state. This is, in general, better practice but is not always practical.

There is one other aspect of subroutine call behaviour that is different from message passing behaviour. When a subroutine is invoked it is in effect brought to life at that point. If you have to create an instance of a service in order to use it then this is another form of dependency. A service should be an entity that always "exists" that is ready to serve. In effect messages can wake up receiving services but it should not be the concern of the sending service to create or dispose of an instance of the service. I therefore intend to add 'Creation Coupling' to the coupling list when I next publish one. Creation coupling occurs in Java classes when a new object is created using the 'New' statement. Methods of that object are then called but the creation of new object (and sometimes its destruction) is often the role of the calling object.

It is the system's responsibility to effect and manage the invocation of the service, not that of the calling application. This allows two critical characteristics to be realized: 1) The services are truly independent, and 2) they can be managed. (Anagol-Subbarao)

In the paper "Web Servces Are Not Distributed Objects" by Werner Vogels he draws this distinction clearly. Loosely coupled web services do not have the same behaviour as objects.

An important notion at the core of distributed object technology is the object life cycle:
· Upon request, a factory instantiates the objects.
· The consumer who requested the instantiation performs various operations on the object instance.
· Sometime later, either the consumer releases the instance or the runtime system garbage-collects it.

It is this point that Werner distinguishes between services and objects and is also this point that makes distributed objects more tightly coupled than web services that are coupled using messages.

In this posting we have discussed subroutine semantics and discovered that subroutine a tightly coupled to their calling modules because of:

• Synchronous Coupling
• Process State Couping, and
• Creation Coupling

The use of asynchronous messages is much more loosely coupled as a result.

References:
Anagol-Subbarao, Anjali, J2EE Web Services on BEA WebLogic, Prentice Hall, October, 2004

Vogels, Werner, "Web Services are not Distributed Objects: Common Misconceptions about Service Oriented Architectures" , IEEE Internet Computing, vol. 7, no. 6, pp. 59–66, 2003

This is also available from Werner's blog site at
http://www.allthingsdistributed.com/historical/archives/000343.html

Marlin, Christopher D. Coroutines, Lecture notes in computer science, vol. 95, Springer-Verlag, Berlin, Heidelberg, 1980. ISBN 3 540 10256 6 and ISBN 0 387 10256 6.

Kaye, Doug, Loosely Coupled: The Missing Pieces of Web Services; RDS Press 2003, ISBN 1881378241

Code Abstractions and Babushka Dolls

There are various ways of encapsulating program code. We encapsulate code so that it can be invoked from some other program code. The encapsulated code can be used in its abbreviated form. I will also refer to it as an abstraction in this posting.

There are some ways of encapsulating code that encapsulate a small amount of code and are fine grained. There are some ways of encapsulating a large amount of code or other abstractions of code and are coarse grained. I would like to be able to list all the forms of encapsulation from smallest to largest and arrange them like Russian Babushka dolls with a clear scheme for which goes inside which.

At the finest level of code abstraction I will start with functions, procedures and methods. They are pretty simple. The name 'function' has a mathematical significance the pre-dates computers. Procedures, subroutines and methods are all variations on the theme of a group of instructions.

The next level of abstraction is the 'class' or the instance of that class the 'object'. Object is an unimaginative name. We may as well have used the synonymous term 'thing' but that would have sounded like we just did not know what to call this abstraction of code. "Let's build a thing to access our database" sounds a lot less specific than "Let's build an object to access our database".

Then we have services. These should be reasonably coarse grained. They may contain classes which may contain methods. I skip java beans in this abstraction discussion that would seem to inhabit the same level in the hierarchy as services.

My metaphor of the Babushka dolls is already looking a bit shaky. There are methods that can contain methods, classes that contain classes. Recursion does not work very well with Babushka dolls. Also there are abstractions like code libraries and include files that cut-across the above abstractions, but I will press on anyway.

The level of abstraction I would like to look at is the component as described by the Service Component Architecture (SCA). Component is another synonym for 'thing'. Instead of 'object' and 'component' we may as well have used 'thing 1' and 'thing 2' and apologized to Dr Seuss for using his naming convention. According to the SCA we should then consider encapsulating our components in 'composites' and our 'composites' in 'domains'.

'composites' and 'domains' at least are getting way from being synonyms of 'things'. A composite is just a group of things and a domain is somewhere you find things in, a bit like a container. Of course a 'container' is typically seen as more a runtime environment than a way of collecting together code or abstractions.

'Domain' is a rather overused term in IT at present. Do not try to draw any parallel between an SCA domain, a Domain Name Service (DNS) or Domain Driven Design. There is very little connection.

I would like to bring the concept of Roger Sessions' 'Software Fortress' in at this point. At least Roger has come up with an imaginative name for his abstraction. His explanation of the various roles that software plays in relation to the Fortress is also imaginative as these are illustrated by various goblins, fairies and ogres in his documentation.

Definition: Software Fortress
A software fortress is a conglomerate of software systems consisting of potentially many processes running on potentially many machines spread out over a potentially wide geographic area that:
I. Serve a common purpose
II. Are owned by a cohesive group of individuals
III. Work together in a trust relationship.
IV. Are built on a common technology base.

Roger could improve his definition by indicating whether "or" or "and" separated the different lines of his definition. Definition line IV suggests the Fortress is quite similar to the Domain concept from SCA. If we include the other three definition lines we may find that a Fortress has finer granularity than the domain and may be closer to a composite or component.

The recommendation provided in a couple of presentations I have seen from Glenn Smyth is that web services should be built if and only if they are required on the border of a software fortress. The essence for the SCA is that a service appears on the border of all components. Therefore the finer grained interpretation of 'Software Fortress' that incorporates an 'and' between the last lines of the definition would seem more consistent with this view.

One difference between ways of encapsulating program logic is the stage in the software development life cycle. At runtime then objects, services and containers encapsulate running logic. At development time classes, include files and functions are relevant. At requirements definition time the domains in domain driven design have relevance and the "common purpose" aspect of a fortress helps define the encapsulation. We therefore would more correctly have three sets of Babushka dolls.

SCA is all about runtime and development time. There is nothing that suggests what part of the design a component represents. Domain driven design is all about business requirements. There is nothing in this process that advises how you are to encapsulate your code to represent the domain. SOA attempts to span all three stages of the development life cycle both by suggesting business services and then keeping to this granularity in the developed code and the runtime.

The encapsulation and abstraction of logic is a complicated area. I probably have not made it any simpler with this posting. It is increasingly vital in software. There is not much excitement as there used to be around programming languages but how we bundle together program instructions is still an area of active interest. The point to take away from this is that we have to define our terms clearly and use them in the context of the stage of the software development lifecycle. A consistent approach from design through to runtime would be nice, but at least your software development methodology should identify the mapping between different ways of abstracting business logic thus linking your three or more sets of Babushka dolls.

References

David Chappell, Introducing SCA,

http://www.davidchappell.com/articles/Introducing_SCA.pdf

Eric Evans, Domain Driven Design,(WebCast)
http://www.infoq.com/presentations/model-to-work-evans

Eric Evans, Domain Drive/Model Driven
http://domaindrivendesign.org/discussion/blog/evans_eric_ddd_and_mdd.html

Roger Sessions has book on Software Fortresses which I confess I have not read yet but the following article incorporates and overview of this concept and the newsletter below explains some of its origins.
Roger Sessions, Modeling Software Architectures
http://www.objectwatch.com/PaperAndSpreadsheetVersion1-0.zip

Roger Sessions, The Software Fortress Model; A Next Generation Model for Describing Enterprise Software Architectures, ObjectWatch Newsletter # 36, November, 2001

http://www.objectwatch.com/newsletters/issue_36.htm

Glen Smyth's presentation to the ACS is now available at
http://sa.acs.org.au/it_arch/index.php/Presentations_Past%2C_Present_and_Future

Business Requirements: More than a Pretty Face

In the his opinion piece for ComputerWorld, Michael Hugos tries to give us a quick-fix for the complicated world of software development. A couple of good rebuttals of this "CIO at Large" have already appeared on the internet. In particular Bran Selic of the UML task force exposes the lack of Hugos's lack of UML knowledge.

Like Moliere's Monsieur Jourdain, who spoke prose all his life without knowing it, Hugos may be surprised to discover that four of his five diagram types are either valid UML diagrams or can be mapped directly to corresponding UML diagrams, with one important difference: UML diagrams are composed of elements that have a standard semantics associated with them.

"Five Diagrams Beat a Victorian Novel" is a catchy title and we can all relate to a good diagram being easier to absorb than thousands of words. We should encourage meaningful illustratons but there are many parts of our templates that are not amenable to the simple diagram. We need to talk about objectives, risks, non-functional requirements, costs etc. – these are easier to express in words.

Hugos is advocating a warterfall development methodology and not advocating anything new with his 5 diagrams.

The first diagram is a data flow diagram which was developed by Edward Yourdon and then improved by Gane & Sarson in the 1980s. Gane & Sarson added sources and sinks which evolved to be the actors in use-case diagrams and swim-lanes in swim-lane diagrams. The swim-lane diagram is a special case of the UML Activity Diagram. I prefer to see the swim-lane diagrams to the old Yourdon data flow diagram. The old DFDs certainly still have value but I believe the new diagrams are more expressive and better accepted.

The logical data model is good sense and developers have been using these for years. I would hope that any serious business analysis involved a logical data model and that this was based on an enterprise information model.

Hugos writes about a "System Technical Architecture" which is similar to a UML Deployment Diagram. This is not usually the role of the BA and sometimes not even the role of the software developer. The systems architect will often be responsible for what part of the application runs on what hardware and therefore this deployment diagram.

Hugos write about an Object Model which is the equivalent of a UML Class Diagram. It is the role of the programmer or systems analyst to develop these. Again I have no problem with this diagram being included in a solution design.

Storyboarding is a technique that has been around for a while and has been adopted by the XP (extreme programming) agile programming technique. Use cases attempt to portray the same information.

The use of screen mock-ups helps show the user some tangible picture of what the application might look like and is an alternative to prototyping the user interface. There is a good IBM article on this.

There is little question that use cases have a bigger impact on software engineering than storyboards. Use cases are the origin and basis for a well-defined object model that not only manages the functionality and business rules of a system, but also serves more than one HCI [Human Computer Interface]

The Business Analyst(BA) develops requirements for two audiences. It is the BA's call whether to do screen mock-ups or storyboards. If the BA wants to develop these to convey to the user what the solution might look like then this should be commended. However, a BA still needs to develop the use cases for the developers to understand the logic behind the user interface.

I got a difficult email this week. It went something like "I have just found something new. Apparently five diagrams can replace our long business requirement documents. Please investigate and see if we can get rid of RUP". This is what prompted me to prepare this posting. In general I believe diagrams are goodness but we need words to be precise. Hugos would like to regress to a simpler time. UML is a standard and we use well-accepted standards to make our life simpler. We should not be slaves to our methodologies. Develop screen-shots or storyboards or prototypes if they will help the users understand their requirements. We need to remember that the developer will benefit by seeing a use-case in a format that he or she is comfortable with and expresses what happens behind the user interface.

References:

Two responses to the Hugos article:

Letter: UML Shifts Specs to User's Point of View
Bob Jones, October 15, 2007 (Computerworld)
http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=304960

Opinion: Five Diagrams Redux
Bran Selic, an IBM distinguished engineer at IBM Canada, is co-chairman of the UML 2 task force in the OMG
http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9043002&source=rss_topic73

The IBM article on Storyboarding

Form feeds function: The role of storyboards in requirements elicitation
http://www.ibm.com/developerworks/rational/library/dec05/krebs/index.html

SOA Governance

I listened to a good Podcast on SOA governance today. It was Phil Windley and Scott Lemon interviewing Tod Biske and Ed Vasquez of Momentum SI. It is clear to me that SOA governance is important in managing SOA and SOA metadata. This podcast provides some ideas on what shape this governance can take and how products may or may not assist with this.

SOA has often been compared with Object Oriented development or even structured programming. The question was asked in the Podcast about why Governance was such an issue for SOA in comparison to these earlier approches. The answer is that the implication of an SOA decision goes beyond the IT group and single applications. The users of the service are significant stakeholders in an SOA and are directly affected by the policies. Organisation partners or organisation divisions sharing services need to agree on a service approach and comply with service interfaces in order to provide effective reuse.

Todd Biske emphasised three key aspects of governance in relation to policies that need to in place. These three aspects can be assisted by automation. They were

• Enforcement of policies
• Creation of policies
• Storage of policies

Later in the discussion two more aspects of governance were provided:

• Communication of polices
• Feedback on policies (policies evolve)

A strong theme of the podcast was that you cannot buy governance. It has to be appropriate for the organisation and a product is not yet out there that provides integrated assistance over all the aspects of governance. The process of building governance occurs by assessing the current situation, working out what processes are involved and then providing the appropriate level of manual operations and automation to support these processes.

Storage of policies is provided by registry and repository products. Registry products are moving toward providing more repository functions. The term "Regository" was coined in this podcast. A parallel can be drawn with LDAP. The registry is becoming much more of a managed data set which is configurable and provides information to humans not just automated processes.

Feedback on policies and services is very important. Without proper feedback the organisation can become a lame duck. Portfolio management is important for services as well as for applications and projects. The discussion reminded me of getting feedback on this statistics for my blog site (using http://www.mybloglog.com/). This tells me where my links are coming from and also gives me a warm feeling that people actually see this stuff. The same thing is true of getting feedback on a website with a product like Webtrends. Services are no different. Metrics about who is using a service, when they are using it and how often, provide valuable feedback.

This feedback and the process of understanding the customers of your services gets the IT person closer to the business. It was suggested in the podcast that in general SOA involves IT doing more tasks previously understood to be business tasks. If you are not experiencing the pressure to do more work related to the business then you are not really doing SOA.

There can be hundreds of policies associated with SOA. Design time policies will include technology choices, product choices and methodologies. Run-time policies include your service contracts and service level agreements (SLAs). These SLAs may apply to each producer-consumer pair and include security policies, operational policies, compliance policies and integration policies.

SOA policy should not be a police state. The approach to implementing policy should be that "The path of following policy is the path of least resistance". An organisation should not reach a gridlock because there are so many policies that the developer cannot move. Policies are something that involves discussion with service users and should not be confined to the IT group.

Effective policy agreements will save time by avoiding the need to build using more than one technology (eg. SOAP, REST and JMS). The value of an agreement is that it saves the cost of building with multiple standards.

Greater governance enables the services in SOA to be more loosely coupled. Loose coupling is about reducing dependencies between services and making those dependencies explicit. The fact that the service contracts are written outside service code and are clearly defined is a key determinate of loose coupling.

Todd's two most important points were

1. Use a service or application portfolio technique to identify what services are good, bad, expensive or good value.
2. To get started on a project ask
   - What services dose this solution expose or use?
   - What business events does this solution publish or subscribe to?
   - What business processes do this solution support?

There were some good insights in this podcast. Governance in SOA is important. It may stop your SOA becoming JBOWS (Just a Bunch of Old Web Services). It makes sure that your organisation gets the best of reuse and does not get lost in mire of standards, products and methodologies.

Footnote:
JBOWS (or JaBOWS) was a term I heard this week in a talk from Glenn Smyth. There is a good post from Joe McKendrick on this acronym. Another reference can by found in an article by Rich Seeley which in turn refers to another artlicle by Brent Carlson which uses the acronym ABOS (A Bunch of Services). It would have been more accurate to write about ABOS in my posting but JBOWS has a nicer ring to it and seems more likely to catch on.

The Low-Down on Bottoms-Up

This is a completely trivial rant on the use of the phrase "bottoms-up" when I the correct term should be "bottom up". We all know what we mean when software is developed using a "bottom up" approach or that ideas in an organisation come from the "bottom up". "Bottoms-up" on the other hand is a friendly remark made before drinking an alcoholic beverage. It also can be a call to get out of your seat an do some hard work although the origin of the "Head down tail up" description of hard work is more apt for digging and cleaning than it is for desk work.

I agree with Jonathan Betz when he says

What's bugging me these days is that a lot of people are trying to describe Google culture as "bottoms up," when they actually mean "bottom up" Bottoms up means a whole different thing. So, to all you Googlers throwing around the phrase "bottoms up" like it's a good thing: please stop. It hurts my ears when you say it.

I also agree with Jonathan Nightingale

I thought it was a rather humourous typo - obviously someone had confused the drinking cheer with the development methodology on a hungover Monday morning and typed the wrong thing.

I first wrote about this in an earlier posting but I have heard it since on many occasions. I list a number of references below who fall into the "Bottoms Up" abyss. Amongst them Bill Gates is quoted a number of times using this phrase. I like to think that this was probably a journalistic mistake rather than a slip-up by the world's richest computer nerd. Bill Inmon, one of the founders of data warehousing also has a couple of bloggers accusing him of advocating a "Bottoms Up" approach but there seems enough who use the correct term to suggest that Mr Inmon himself did not advocate this phrase.

A "Bottoms Up" approach could be when the software developers go out for a drink and come back in to write some code. This would be a new approach to software quality. The phrase could refer to when everyone is tail up and is very busy but I do not believe this is the intent in most cases. Congratulations though to Chris Keyser who has his own definition for a "Bottoms Up" person suggesting some justification for using this phrase:

But I have definitely been there working with a few bottoms up people. That's the person that when I arrive at some moderately elegant system architecture diagram with some pretty pictures in PowerPoint and Visio catches the few key leaps of faith I made when I am doing my hand waving around my vague idea of implementation, and has saved my rear-end more than once.

Still more worrying is when "Bottoms Up" is used with the phrase "Tops Down". Now that does conjure up some interesting images. The best idea I found on this subject comes from a posting response:

Perhaps they have actually installed a free beer cooler somewhere at IBM, and
they really are encouraging bottoms-up development, and anticipate that enough
of this could lead to tops-down development?

References:

The Bill Gates Quotes
http://www.crn.com/software/183701538
Microsoft wants to make collaboration "simple and bottoms up" but also give users visibility into their important business information, Microsoft Chairman Bill Gates told attendees of the company's second annual Microsoft Office System Developers Conference in Redmond, Wash.
"We are adding new capabilities to make collaboration simple and bottoms-up, but also gives [users] visibility into business information. There are new capabilities in Office aimed at filling the skills gap," Gates said.
http://www.microsoft-watch.com/content/operating_systems/gates_pushes_power_to_the_people_message.html

The productivity benefits accrued by companies that aren't afraid to back "bottoms-up empowerment" was a key theme for Gates during his hour-long address, attended by CEOs from Barnes & Noble, Berkshire-Hathaway, Dell, Delta, Fanny Mae, Hewlett-Packard, Home Depot and other Fortune 1000 firms from around the world.

(and the next one contains "Tops-Down")
http://www.microsoft.com/Presspass/exec/billg/speeches/2007/02-20BillGatesITACtranscript.mspx

The consumer Web brought in, with products like SharePoint, into an internal company environment, so you have bottoms-up creativity but no discontinuity as you're taking that bottoms-up creativity and connecting it with sort of tops-down financial systems because the kind of parts and information out of those systems can easily be imported into any of these engines or portals. We just pick a template and go. We don't have to call IT or think about it, and yet the audit trails and the rights management capabilities built into these things are what's appropriate now for this very digital environment.

The Bill Inmon Attributions
http://www.netcrucible.com/blog/2004/04/09/death-of-the-data-warehouse/

In fact, I remember Bill Inmon (the father of Data Warehousing) fighting against the spread of a particular concept of ?data mart?, since he felt it would give people the impression that data warehousing could be done in a bottoms-up fashion.

http://searchdatamanagement.techtarget.com/expert/KnowledgebaseAnswer/0,289625,sid91_gci1173583,00.html
You'll have two basic decisions to make and numerous follow-on decisions:
Enterprise vs. Data Mart-Oriented Architecture
Enterprise vs. Bottoms-up Oriented Methodology

Other Miscellaneous Quotes
http://www.ebizq.net/blogs/firstlook/2007/06/scalable_wikis_made_easy_by_mi.php
"This is a bottoms-up form of communication as opposed to a much more traditional centralized expert driven-down sort of communication," he says.

http://lxer.com/module/newswire/view/84664/index.html
Raising Linux Awareness: A "Bottoms-Up" Approach

http://duckdown.blogspot.com/2007/07/enterprise-architecture-top-down-or.html
(This is a rare blemish for James whose blog on Enterprise Architecture is well worth reading).

One of the more interesting things is that a person debating top-down vs bottoms-up needs to take into consideration is historically which one has had better results. Of course, top-down has the advantage that even mediocre delivery will be declared success while bottoms-up mediocrity will probably result in throwing daggers. If we ignore the whole perception is reality crowd for a moment and truly think about success, then I think our conclusions would be different.

http://www.uxcomm.com/corporate_news_ar_2007-01-09a.asp
In contrast, the "bottoms up" approach is characterized by the use of management software that vendors typically ship with their devices, or which is put in play when an Operating System (OS) is loaded onto a device.

http://www.worcestergroupinc.com/blog/index.php/2006/11/24/bottoms-up-or-top-down-the-eternal-business-intelligence-question/
[ # ] Bottoms Up or Top Down? The Eternal Business Intelligence Question

http://news.zdnet.com/2100-3513_22-6087566.html
Significantly, IBM chose not to define and then build a large-scale, sophisticated knowledge management system. Instead, the company is taking a bottoms-up approach, allowing contributors to have a more active hand in how collaborative work is organized.

http://marxsoftware.blogspot.com/2007/10/highly-useful-jax-ws-introductions.html
I have also followed with interest the debate about web services development with JAX-WS with a bottoms-up approach (annotation Java classes and generating appropriate artifacts) versus with a top-down approach (starting with WSDL).

http://weblog.halmacomber.com/2003_05_04_archive.html
Not only is LISP suitable to this style of development, I'd dare say the language anticipates a bottoms-up collaborative approach. Not all software development is collaborative and bottoms-up, just like not all architecture and engineering is collaborative and bottoms-up. The language helps.

Does SOA Rate a Mention in the Strategic Plan?

I was asked to draft a contribution to the IT Strategic Plan last week. My organization is a big public organization and the IT Strategic Plan is a glossy public document that has to be more of a marketing document than a technical plan. I wanted to put something in about SOA but I could not rely on reaching a technical readership or a readership that new much about the organization's IT at all.

This raised my curiosity. How many organizations actually put something about SOA in their IT strategic plan, let alone a corporate strategic plan?

Many commentators suggest SOA is a strategic imperative. For instance David Linthicum suggests "when thinking about SOA a long term strategic plan is best". Christopher Koch goes further and suggests that IT will assume responsibility for business innovation.

The ongoing shift to service-oriented architecture (SOA) will improve IT’s
ability to innovate because it requires that IT understand how the business does
its work. SOA represents the highest order of business process innovation

Surely SOA is of such central importance that it is going to be central to most IT Strategic Plans.

If I look to Forrester Research for some help with the IT Strategic plan I am advised that

IT's strategic plan is an essential tool to run IT like a business. The strategic plan for today's IT is different from the strategic plans that IT may have developed five years ago. It is purpose-driven and a complement to IT governance structures and processes.

Unfortunately SOA does not rate a menton in this article. Enterprise Architecture and Project Portfolio Management however bother get a Guernsey.

I spent some time with Google and found no strategic plans containing SOA and so I decided to look at some IT Strategic plans. Some of the findings are detailed at the end of this posting.

I found four IT strategic plans that put SOA as fairly a central goal or support for the plan. I found four where there was a mention of SOA but not a significant emphasis on it. After not finding as many as I had hoped I thought that perhaps when people write about SOA for the consumption of the layman they use other terms to make the document more accessible. So I looked for mention of 'agility' or 'reuse' in IT Strategic Plans. This I thought is a way for IT people to speak in code about SOA without mentioning those three letters. I found four of those too, although the connection with SOA was not all that obvious in these documents. I have listed nine IT Strategic plans with no mention or allusion to SOA.

This is not a thorough or unbiased study. The people publishing the plans on the web seem to be government and tertiary education groups. This happens to be my interest. The list is not a random sample and I could have missed some important references within the documents, however I have to conclude that SOA is not getting into as many IT Strategic plans as it should, given its strategic importance.

I am left to hypothesize on the reasons for this:

• SOA is not strategically that important
• SOA is only considered a software development issue and is not considered to relate to other IT issues infrastructure, support services or IT procurement
• SOA is too hard to explain to the non-IT person
• SOA is not as business-centric as some SOA authors would have us believe
• SOA is too new to warrant mention in an IT strategic plan
• SOA is too old and like RDBMS everyone accepts it and will move on without having to bring it up in a strategic plan
• The SOA message has not reached those who write IT strategic plans
• There is little marketing value in SOA

This is a reality check for those espousing the strategic importance of SOA but I would not go as far as those who suggest SOA is only for tactical purposes. Whatever the reasons, I will still push for it to be accepted at the strategic level.

In my contribution to my organization's IT Strategic Plan I started with a business definition such as one of the earlier definitions discuss in my earlier postings on the definition of SOA.

I then went through some for the motivations behind SOA and expanded on some of these in reference to the business of my organization. These motivations were Agility; Reuse; Assist Transition from Legacy; Federation; Extensibility; Manage incremental Change; Better alignment between IT and Business Processes; Intrinsic Interoperability; and Vendor Diversity.

This ended up a fairly long description, so I expect most of what I wrote will end up being cut by the editor of the strategic plan. Whether SOA gets edited out of the IT strategic plan is not important as long as the internal focus is on getting the pieces in place to make SOA happen and we can get some support from the business for this. Perhaps in this process I will encounter some of the reasons that SOA is not so prevelant in strategic plans.


IT Strategic Plans with SOA as central or a core goal

Oxford University, UK:
http://www.ict.ox.ac.uk/strategy/plan/plan.xml.ID=S4

The Global Justice Information Sharing Initiative (Global) Advisory Committee
(GAC) (an advisory body to the Assistant Attorney General, Office of
Justice Programs (OJP), and the U.S. Attorney General)
http://www.it.ojp.gov/documents/200409_GAC_Strategic_Plan.pdf

The Commonwealth of Massachusetts
http://www.mass.gov/Aitd/docs/itdstrategicplan.doc

SURF (A Nederlands higher education foundation) http://www.surffoundation.nl/eng/download/SURF-Strategic-Plan_2007-2010.pdf

IT Strategic Plans where SOA rates a minor mention

Carnegie-Mellon University
http://www.cmu.edu/computing/about/2006StrategicPlan.pdf

State of Texas Department of Information Resources
http://www.dir.state.tx.us/pubs/asp2006/asp2006_abbreviations.htm

The University of Western Australia, Library
http://www.library.uwa.edu.au/__data/assets/pdf_file/11068/Scanning_the_environment_2007_NE.pdf

National Labor Relations Board, USA
http://www.nlrb.gov/nlrb/shared_files/reports/FY2007_StrategicPlan_FINAL.pdf

IT Strategic Plans with Some mention of Agility and Reuse but not SOA

State of Wisconsin:
http://enterprise.state.wi.us/home/StratPlan/enterpriseITplan.pdf

Commonwealth of Virginia
http://www.vita.virginia.gov/uploadedFiles/Library/StakeholderWorkshopResults.pdf

University of Iowa:
http://cio.uiowa.edu/strategicplan/

Department of Family and Community Services, Australia
http://www.facsia.gov.au/internet/facsinternet.nsf/via/ict/$file/FaCS_ICT_Strategic_Plan_2004-07.pdf

IT Strategic Plans where SOA rates no mention
(There actually were many but these are examples)

City of Winston-Salem:
http://www.cityofws.org/Assets/CityOfWS/Documents/forms%20and%20reports/Information%20Systems/IT%20Strategic%20Plan.pdf

State of Maine:
http://www.maine.gov/oit/strategic/SITP.doc

US State Department:
http://www.state.gov/m/irm/rls/c13461.htm

Louisiana State University:
http://www.lsu.edu/departments/its/ocio/FITS/fits.html

State of South Carolina:
http://www.cio.sc.gov/ITPlanning/StateStrategicPlan.pdf

New York State:
http://www.cio.state.ny.us/NYS%20IT%20Strategic%20Plan/NYSITStrategicPlan_20Jun2006.pdf

OHIO State University
http://cio.osu.edu/planit/refresh07.html

Dept of Justice, State of Victoria, Australia
http://www.justice.vic.gov.au/wps/wcm/connect/DOJ+Internet/Home/About+Us/Our+Goals/JUSTICE+-+Department+of+Justice+-+Information+Technology+Strategic+Plan+2002-05

Electoral Commission of Qld, Australia
http://www.ecq.qld.gov.au/data/portal/00000005/content/60401001035252002415.pdf