Coupling Calculus

I have had thread going through this blog on Coupling. Coupling is important because good definitions of SOA are dependent on the concept of 'loose coupling' or more precisely, 'message coupling'. The thread so far has included the following postings:

Cohesion, Coupling and SOA
Bragging Rights on Coupling
AOP: The Worst Form of Coupling
Binding and Coupling
Subroutine Call Coupling
Seven Levels of Coupling or Seven Deadly Sins?
Coupling with Contracts

In the first posting of this series I went back to the origins of the coupling term that was coined by Edward Yourdan to explain good programming practice in terms of subroutines. I then tried to come up with a list of coupling types which would help explain what were good and bad form of coupling.

This served to clarify the concept in my mind but the list of coupling types is not exhaustive and term 'coupling' is still not clearly defined. Coupling is a measure, usually described as 'loose' and 'tight' which correspond to the value judgement of 'bad' and 'good' respectively.

A good measure has numbers that describe it, but this is not yet the case with coupling. I and other authors have attempted to rank coupling types in order of merit with the ranking an implicit measure of coupling.

There have been attempts to measure coupling by looking at program artefacts like variable names and measuring how many times they are referenced. These attempts seem to miss the point by measuring what can be measured rather measuring coupling properly. Some of these approaches are described in footnote 1.

In this posting I try to suggest some definitions relating to coupling. This will not come up with useful numbers but it will try to explore the concept of coupling and may be a basis on which more theory can be developed.

Coupling is about how easy it is to change applications. If one part of an application (module) has to be changed because another part of a program changes then these program parts a tightly coupled. If one module does not have to be changed for changes made to another module then these application modules are loosely coupled. There are a number of programming practices that can be put in place to loosen the coupling between modules. This is good because it reduces the effort required to maintain your code.

The above would seem to be a fairly elementary description. Definitions however can still be made more precise, so that the concept can be analysed in more detail.

Lets say we have two modules A and B in a system S. The number of changes that can be made to A can be defined as C(A).

We can make changes to A because we want system S to do something different, or we can make changes to A for other reasons. We might want S to be produce results more quickly or we just might want A or S to be reorganised to be more easily comprehended. A change to A might not mean any change the output of S.

In order for A to be changed and the result of S to be changed to a desirable result (which may be an unchanged outcome for S) then in some cases some module B would have to change as well.

The coupling between A and B is defined as the effort required to make changes to B as a result of these changes A. For change j then there is work (and cost) associated with this change for systems S say Wj(S). The work in module Wj(A) and module be Wj(B).

Coupling then can be defined in terms of an individual change
Coupling(j) =the work required in B to make change j as a result of the change in Aj.
Coupling(A,B) is the sum of all work require in B for such changes j.

This leaves the concept of 'Coupling Type' to be explained. A system can be organised to reduce coupling between A and B. A coupling type is a way of organising the system. Some ways of organising will mean that a change to A require changes to B. Some ways of organising will mean that B can remain independent of changes in A. Some ways may increase or decrease the work involved in making changes to B when A is changed.

In general a way of organising a system S is O(S) which is another system which produces the same results. This is a change to the structure of the system rather than a change to the outcomes of the system. If we wanted to measure the coupling of a way of organising O then 'all' we have to do is sum all the Coupling(A,B) over all such S, A and B. A coupling type describes a number of ways of organising a system. The coupling associated with a coupling type is therefore the sum of all the Coupling (A,B) over all these ways of organising S.

To express this succinctly:

Coupling between two modules (A and B) of a system (S) is the work required to make changes to a module (B) as result of changes to another module (A) and still produce the desired outcome of a system (S).

A coupling type is scheme for organising the system (S). A measure of the coupling of a coupling type is the sum over all A and B for systems S which conform to this coupling type.

These technical definitions of coupling describe big, practically incomputable numbers. The conclusions we can draw though are

• There is nothing really special about a coupling type. There could be a very large number of these and they are not confined to simple statements about how to pass parameters between modules. My earlier posting attempting to count these is inherently futile expect to provide common examples.
• Coupling is about work relating to a change. I have not defined what I mean by 'work' intentionally. We could count lines of code changed or look at whether logic or data is changing. We count the costs of making these changes (eg. Hours of labour). It is meant to be used in its common sense. If it is necessary to trawl through technical documentation to make a change to one line of code this is still consider then the trawling effort should be consider as part of the work expended for this change.
• Coupling is about systems in general. It is not just code instructions that are relevant. You might have to change constants, configurations, contracts or XML. These need to be considered in the concept of coupling.
• The concept of coupling is relevant to systems theory examples outside of the field of computer science. Systems can also refer to biological or sociological systems (See footnote 2).
• A change may not actually change the behaviour of the system at all but it is still relevant for coupling.
• Coupling at its most elementary level looks at two particular modules, a particular system and a particular change. Thus it sensible to say A is tightly coupled to B in a system S in relation to change j.
• Coupling can aggregated up to the module or level or to the Coupling Type level. Thus we can have A loosely coupled to B. We can talk about a Way of Organisation or a Coupling Type as facilitating loose coupling.
• Restriction have not been placed on A and B. Therefore if B=S we can have A tightly coupled with the rest of the system S. A and B could intersect or A could be a sub-module of B.

It is still important to describe different coupling types and to determine what types promote loose coupling and what types do not. Good practice will reduce coupling for common programming changes.

To determine whether one coupling type is better than another the above suggests we need to enumerate all possible changes and work out which type leaves you with less work involved in one module as result of changes in another. This is obviously impractical but the analyst can do a couple of things to determine a good coupling type. A range of typical changes can be investigation or the coupling type can be looked at in relation to a specific type of change. For instance the benefits of message queuing can be looked at in relation to changing the location or speed of hardware.

This posting has added some precision to the concept of coupling. Counting all the possible ways to change a system is futile, however good and bad coupling will be become evident by looking at coding examples and foreseeing the changes that will be required to any system.


Footnote 1:

The following web pages describe metrics for coupling. These can be based on a static analysis of code or a dynamic tracing of the execution paths. In both cases the emphasis is counting the reference to a variable or type. This has some value in the task of maintaining code but is not a measure of the work required to change one module based on the need to change another. In some cases there may be some correspondence between the amount of work in changing the use of the variable with the number of times it is reference but not generally.

Lethbridge, Timothy C. and Anquetil, Nicolas, Experiments with Coupling and Cohesion Metrics in a Large System, April 2000, https://www.site.uottawa.ca/~tcl/papers/metrics/ExpWithCouplingCohesion.html

Poshyvanyk, Denys and Marcus, Andrian, The Conceptual Coupling Metrics for Object-Oriented Systems, http://www.cs.wayne.edu/~vip/publications/poshyvanyk-ConceptualCoupling.pdf

Glover, Andrew, In pursuit of code quality: Code quality for software architects: Use coupling metrics to support your system architecture, April 2006, http://www.ibm.com/developerworks/java/library/j-cq04256/
FrontEndArt describes one of its products for montoring source code quality include various metrics for coupling, http://www.frontendart.com/monitor/help/node22.html
Diederik Krols provides a blog-site archive which includes the postion "OO-Metrics for Coupling". This provides a coupling metric based on types shared across a module boundary. http://dotbay.blogspot.com/2006_01_01_archive.html

Footnote 2:

The concept of "structural coupling" in systems theory is used to describe co-evolution in biological and social systems. This is where, if entity A evolves then entity B might need to evolve to compensate. There would seem to be more complicated processes at play here than in a computer application.

Kay, Robert and Goldspink, Chris, Towards a Complex Non-linear Systems Theory of Organisation, http://ftp.informatik.rwth-aachen.de/Publications/CEUR-WS/Vol-72/044%20Kay%20Complex.pdf

Conference Summary Served with Spaghetti

I presented at a conference “Achieving Interoperability in Systems Architecture” on Wednesday. The occasional get together of architects, managers and developers is valuable. You can grab the odd new insight and find new approaches but it is also good to see others attacking problems in the same way and validating your own approaches.

After the conference I thought I would finish this summary on the aeroplane home. I have always wondered how to spend the time on long flights profitably and I admired a couple of passengers using laptops on my previous flight. Unfortunately as I try to use the laptop in flight I am in a middle seat with no elbow room on either side, and now I am served dinner in middle of the flight. I am juggling my laptop and the worst Spaghetti Bolognaise I have ever tasted. Still, here is the summary.

Josh Graham of Thoughtworks kept time well and came up with the comment “Why should I expect straight lines on my Business Process Model when business is inherently messy.” He also introduced one of our New Zealand guests as coming from the land of the “Wrong White Crowd” which sounds like a political comment and spoonerism wrapped up into one.

Mark Murphy of Tower Aust Ltd advocated we start small with SOA. He quoted IDC showing growth in SOA from $3.5bn in 2005 to $33bn estimated in 2010. Mark looked at some SOA definitions and aptly described the current situation where people build silo applications and then expect IT to join them together. He showed a couple of models from Sun and Microsoft showing SOA models that illustrated logical SOA.

Stephen Smith of Arcbok presented a simple framework for assessing the readiness of an organization for SOA. The framework consisted of assessing an organization on the dimensions of Leadership & Culture, Business, Technology, Knowledge, Funding and Sustainability. Each dimension has a number of questions to ask about an organization to asses its readiness. The sustainability dimension described whether the organization had the appropriate support, operations and governance in place for sustain the SOA. This also could include appropriate operational monitoring, measuring and standards. A readiness assessment tool is about risk management. Beware vendor maturity assessments that are used as tools to sell more services.

Krist Davood of AGL and Sensis has achieved something that everyone in the room envied. He had success justified SOA as an initiative using fairly hard-nosed accounting principles. He had justified SOA because of its ability to avoid periodic failures in the business because of the current way of doing things. He had been able cost these failures and justify the project to introduce SOA into the organization. SOA as a strategy to eliminate risk had been demonstrated to be effective.

Centrelink is a huge SOA operation and it was good to hear from Rob Doughty who is their Applications Architect. I liked his characterization of users as “Human Task Delegates” in relation to workflow applications. It was interesting for me to hear about the legacy system challenges that Centrelink have, but also how their portal work in benefiting other government departments. The focus of the talk was the deployment of a portal product which they will be basing all their systems on. I liked the comment that “Portals bring people and process together”. Rob talked about the challenge of working with various IT groups to deliver on a particular pattern of use in relation to the portal. He advocated the use of an “Integration Competency Centre”.

I then presented on Enterprise Architecture and its links to metadata and SOA. Some of this material is already on my posting on SOA Definitions, SOA Metadata and the Zachman Framework. I will elaborate in a future posting.

John Fisher from the NZ Ministry of Education showed us some great business modelling techniques that had found favour with his users. The technique involved Business Element Models, Business Component Models, Business Role Responsibilities and Business Service Models. The models were focussed on producing requirements for widely federated (eg. For 2,700 schools) SOA systems. He introduced the analogy of using power adapter so could use a drill brought in the UK to explain SOA to users.

Jan McConchie of SA Government ICT Services provided an overview of the challenges that she has to bring together the South Australian Government Services to present a common face to the South Australian citizen. This was interesting because the SA Government has established and Office of the CIO and this is showing the desire to coordinate and rationalise government IT activities. As one of the agencies involved, my organisation will be affected by standardisation and IT governance changes. Jan drew on research by Gartner and Saugutuck Technology to explain what SOA meant to her and discussed some initiatives in place for the SA Government.

Chris Howard was from the State Revenue Office of Victoria. Chris focused on risk and SOA and quoted Gartner, advising that it would take 3-4 years before investments in SOA payed off. He discussed a long term plan for legacy transition which was underpinned by full and frank statements about costs, benefits and risks with his business. Chris endorsed ITIL as an enabler of SOA.

Josh ran an interactive session where we discussed the issues of SOA and getting the business on board. It was never clear what you will find when you "turn over rocks" in the analysis process. We discussed what services to build first. It was not always best to do the low risk, high value project first. In the end it might just be business or legislative imperatives that help you make the decision. It was difficult to decide when to do those cross-cutting "hygiene" services.

George Cascales from Integral Energy discussed how he was implementing SOA in an out-sourced environment. He had a small group of architects and had succeeded in being able to develop new applications quickly using reused services. His group focused on standards and governance rather than the actual development. A set of tools and standards would be given to contract developers to develop the services and applications.

Rajat Chopra, the international speaker from Bell Canada provided a good description of how Bell embraced SOA. It was important for Bell to provide common services to serve its various customer channels for selling telecommunication facilities. Rajat provided us with a number of analogies for explaining SOA to management including Lego blocks, a car factory and diet. With diets, you can latch onto gimmicks but in the end it is lifestyle change that makes the difference. This is the same with SOA.

Alex Jouravlev representing the ACS presented twice. Alex’s first presentation was titled Prepare IT and Business for SOA. He traces the origins of SOA back to 1988. He made the point that our business cases do not really engage the business and we need to strive for simple messages using language the business understands to achieve a ‘mental ownership’. Alex supported the model driven architecture approaches but not for the reasons that they were developed. Business requirements should be developed with referencing any ‘system’ and should be modelled with Visual UML Models. Alex noted that several modelling practices that were usually acknowledged as important were often missed. These include Conceptual Business Modelling, Conceptual Data Modelling, Business Object Modelling and producing an Enterprise Ontology.

Later in the day Alex was able to present a model based on CMMI maturity model in an easy-to-understand fashion with the purpose of moving the SOA objective from IT to the business as an organisation obtains more SOA maturity. Alex was able to give us a realistic view of the limitations that would be expected at each level of maturity and provide an insight into how the role of key players like the Enterprise Architect and Business Analyst would change at the various levels.

Ram Kumar was from OASIS and provided an insight into the work of this organisation that is emersed in creating SOA standards. He discussed his role in developing the Justice Sector Information Interchange standard. This was of special interest to me because of my involvement in this sector. Ram emphasised the importance of XML governance and advocated this for successful SOA.

Josh summarised the conference and he may discuss this on his own blog site. Many themes resonated with me but one stands out as I wrestle with my in-flight dinner. We had slides from at least three of the speakers that showed insanely complicated maps of their existing systems. It seems we are all lumbered with integration spaghetti and our hope is that SOA will resolve this.

ESB or not ESB?

I work for an organisation that is pretty well fixed on achieving a goal of an SOA but we are still wondering how to get there. The worst people to ask are software vendors, because the answer will always be to buy more software. This stuff costs big time though.

For the cost it took to set up our application servers I could have bought two comfortable suburban homes and then we pay the software maintenance which I have to invest in at the cost of having a couple of staff. It is not as if I can do without the staff though. I now have to recruit someone to administer the application server and find developers with the skills in this particular applications server. So when the software vendor tells me that all I need to is the pay the cost of another suburban house and sacrifice funds for another staff member to buy an ESB I need to be pretty clear that this will provide some benefit.

There is some disagreement about what an ESB is or what benefit it brings. Mark Richards looks at some of the definitions of ESB in his presentation for the NFJS conference (video clip at http://www.infoq.com/presentations/Enterprise-Service-Bus). He then defines the ESB in terms of the capabilities it might bring these are listed as:

1. Routing
2. Message Transformation
3. Message Enhancement
4. Protocol Transformation
5. Service Mapping
6. Message Processing
7. Process Choreography
8. Services Orchestration
9. Transaction Management
10. Security

If software contains about five or more of these capabilities then it is an ESB. This would normally include first three capabilities. Together Mark Richards refers to these as "mediation".

This list is handy because it allows you to look at your needs and determine whether you should purchase such a product with these capabilities. You can determine whether these capabilities are available in your application services already, whether you need to acquire a niche product to address a capability or whether you can handle some of this capability in your own code.

This list is also useful to determine what you are not getting in a particular product offering. One consultancy firm demonstrated to me a great way of addressing the routing problem in application code and claimed I did not need an ESB because of this. It was certainly useful but it will not help if I need the other nine capabilities.

There is certainly some difference of opinion on whether you need and ESB. Vendors will say you need to have an ESB to do SOA. Sandy Carter of IBM in her web presentation (http://www.infoq.com/interviews/sandy-carter-soa) says that an ESB is a a "crucial part of an SOA" and a "spinal chord of your business".

On the other hand a presentation by Jim Webber of Thoughtworks is critical of the ESB approach and believes it works against a good SOA (http://jim.webber.name/downloads/presentations/2006-10-EJA-Guerrilla-SOA.ppt). Jim claims that ESB adds a proprietary middleware to you SOA which will lock you an your software in to a particular vendor. He goes onto to say that "spaghetti" in SOA is inevitable and one of the strengths of SOA is that it copes with this. ESB on the other hand can make spaghetti worse.

One of the best quotes I found was in article by Colleen Frye discussing ESBs with Jason Bloomberg and others:

This bandwagon's a bus and ... vendors positioning in the SOA market have jumped on it over the last few years.

I don't believe my organisation is mature enough in its SOA progress to consider and ESB. We have to develop a critical mass of services before we should look at investing in more glue (as Jim Webber describes it) to organise and bring these services together.

References:

Mark Richards http://www.infoq.com/presentations/Enterprise-Service-Bus
I liked marks insight in that SOA "Presents IT Services as Business Services".
Also he mentioned a couple of open source ESBs
Mule: http://mule.codehause.org/
ServiceMix: http://servicemix.org/ (JBI = JSR-208 compatible).

Sandy Carter: http://www.infoq.com/interviews/sandy-carter-soa
Some good parts of this talk excuse her from being a vendor and almost excuse her from using the word "leverage" too much.
"ESB as a pattern of SOA."
A quote from Steve Mills "SOA will be with us for decades"
"I think that BPM and SOA are two sides of the same coin. I don't think you can do BPM without leveraging SOA."

Jim Webber: http://jim.webber.name/downloads/presentations/2006-10-EJA-Guerrilla-SOA.ppt
This is a powerpoint slide presentation. He is also promoting his book that looks interesting.

Colleen Frye: ESB Market Report, http://searchsoa.techtarget.com/originalContent/0,289142,sid26_gci1226497,00.html