Coupling with Contracts

I listened to a podcast by Thomas Erl yesterday which gave yet another slant to the subject of coupling in SOA. My previous posting about coupling was inspired by a Zapthink arcticle on "The Seven Levels of Loose Coupling" seven levels of coupling. The Thomas Erl podcast provides more information on at least three of the seven levels described by Ronald Schmelzer in the Zapthink article. These are

• the coupling between the service consumer and the services implementation,
• the coupling between the service consumer and the service contract, and
• the coupling between the service consumer and the business process

In his podcast Erl asks the listener to refer to a diagram on his "SOA Principles" web-site. The types of coupling he focuses on in the podcast and the diagram are:

Service consumers are coupled to the service contract

• The service contract can be coupled to a parent business process
• The service contract can be coupled to the service logic
• The service logic can be coupled to the service contract
• The service logic may be coupled to proprietary vendor technology
• The service logic may be coupled to multiple services it may need to compose
• The service logic may be coupled to various resources that are part of the overall implementation environment

These coupling relationships broaden the possibilities explained in the Zapthink article. Firstly all the coupling relations are not based around the service consumer and secondly the coupling is not a symmetric relationship. Erl explains that it is good to couple the logic to the contract as this is contract-driven development but poor practice to have the contract tied to the logic of the service.

The multitude of coupling relationships was referred to in my earlier posting. When discussing coupling, it is important to be clear on what is being coupled when explaining service coupling. Zapthink discusses implementation, contract, policy, business process, data schema, infrastructure and semantic layer. What Erl describes is the service logic is part of what Zapthink describes as the implementation. Erl does not describe policy coupling. The data schema is part of the implementation environment and the infrastructure could be vendor technology.

I would prefer to keep coupling as a symmetric relationship. In other words if logic is coupled to contract then contract is coupled to logic. Erl is trying to cover methodological issues when he talks about contract-driven development. This is an important 'pattern' but it is not about coupling. It is about what a developer does first. The fact is that the contract and the logic are coupled and if this coupling can be made loose then that will provided more flexibility.

The focus of the Zapthink article is on the coupling with the service consumer. Erl has a twin focus on the contract and the service logic. My previous discussions on coupling tended to focus on coupling between services (or program modules). I tend to think that most important coupling concern in SOA is between the consumer and the service implementation in SOA. The consumer may be a user interface or another service. Contract coupling is important but it is more of a vehicle for loosening the coupling between consumer and provider.

One strength of the Zapthink article is that it has enabled me to analyse the Erl model. As is usual for Thomas Erl, his illustrations provide some clarity to a complex interaction of concepts. In this case his diagram should be taken as a sample of the coupling issues rather than a representation of all of the coupling issues relevant to SOA. Also as discussed above I suggest we stick with a symmetric notion of coupling to keep the world of SOA a simpler place.

Does normalization apply to Services?

I was listening to a podcast by Thomas Erl this week where he was talking about a pattern for normalization of services. This was interesting because there is not much in SOA with the same theoretical rigour as database normalization inspired by the work of Edgar Codd in the 1970s and the relational algebra and the Structured Query Language (SQL) that followed it.

The normalization that Erl talks about is fairly limited. On his website he summarizes normalization as

Problem
When delivering services as part of a service inventory, there is a constant risk that services will be created with overlapping functional boundaries, making it difficult to enable wide-spread reuse.
Solution
The service inventory needs to be designed with an emphasis on service boundary alignment.

There does not seem to be a lot of science involved in decomposing services to their basic elements so they can have maximum reuse and be easily composed into higher level services. Perhaps more precisely it is a 'soft' science that is all about governance, process and aligning with user requirements.

Normalization of services is not something that has been picked up strongly by other authors however David Chappell has his description of the normalization project which is consistent with Thomas Erl:

An idealized SOA environment provides a cleanly delineated set of services,
with one way to accomplish each function. This kind of normalization makes reuse
self-enforcing, since each service provides access to a specific group of
information in a specific way.

In his podcast Erl gives an example. If we have a service to get an invoice, then it is redundant to have a service or a method to get an invoice header. To have both is not normalized however there are reasons you may require both and therefore this would be 'denormalized'.

I am a little critical of the use of the term 'normalisation' to describe removing service redundancies. The processs of normalization in data modeling (and mathematics such as linear algebra) is a very precise and well developed technique where the concept used by Erl and Chappell while useful, is fairly loose.

If we wanted to do the equivalent of first normal form in service interfaces we might consider getting rid of the service equivalent of the "repeating groups" that destroys the relational model. If there are many 'invoice items' for an invoice, this means you could not accommodate 'invoice item' in your 'invoice' message. The strength of SOA relies on being able to exchange documents that mean something to the user. This is where attempts to get scientific about normalization fail the cause of SOA.

I will take this normalization discussion to its absurd ultimate. When I did my final year of my honours degree I simulated hardware that consisted entire of only two elements. They were the 'Nand gate' (logical not(and (a,b)) ) and a single binary digit of memory (Flip flop). The computer program that ran on this simulated hardware was simply a wiring diagram. Any computer program that can be represented in normal programming languages can be represented as a combination of these two elements. The sort of programs I ran on this simulation were the full adder, 4 bit timer and 3 bit shift register. Programming at this level is more a level in elementary digital electronics than it is in software development.

This was a distributed network of "services" at the very elementary level, but it is not very useful as a software development approach. This is a completely elementary normalized distributed application that can not be reduced further. This could be described as the ultimate implementation of reuse. You only have two 'services' (Nand and memory) both of which are reused constantly. The challenge is to link them together in meaningful ways. These services however are tightly coupled because of their synchronous nature and therefore this is not an SOA.

As a programming language this is worse than assembler language. Trying to wire gates together is even more detailed than assembler coding. Even if we could get SOA performing brilliantly at lower levels of granularity we do not want to go to such a low level of granularity that business entities are unrecognizable.

Beware dogmatic approaches to normalizing or simplifying a service inventory. I suspect the MEST architectural style with its focus on removing verbs or methods is heading down this type of reductionist thinking.

An overriding concern should be to meet business requirements and this might mean implementing redundant services and poorly structured business documents in services. Designing with a focus to eliminate redundancy is a good idea. It is nevertheless useful to seek out redundancies and investigate the option of reducing these if it does not compromise the ability of the services to represent business processes. Too much science in this process can reduce your services to absurd solutions.

Reference:
Kimber, Antony, The General Logic Array: An Abstract Architecture, Honours Report, The University of Adelaide, Department of Computer Science, November 1985.

This posting may have seemed a flimsy excuse to expose this work to the public domain but I hope it has described a useful anti-pattern for SOA.

The above project was done around the time Sun were having success introducing their Reduced Instruction Set Computer (RISC) chips which included simpler instruction sets that performed more quickly. Reducing down to a couple of elementary components is the extreme end of this strategy. There was also early interest in molecular electronics with the hope that if we could get molecules to behave as electronic components we could produce very small electrical circuits. One of the challenges of the project was working out lattice arrangements that would allow the appropriate connections to the two or more types of elementary components to be made.

Book Review: Loosely Coupled - The Missing Pieces…

This is a review of the book "Loosely Coupled: The Missing Piece of Web Services", by Doug Kaye. In a previous posting I expressed delight that an author had dedicated a book to this important topic. This perhaps is not the definitive book on loose coupling but there is plenty of value for the reader.

Doug has an easily readable style and sticks to a high level but there is enough technical substance to keep and IT architect or IT manager interested. You will find no code snippets and no programming standard specifications described in this book but there are some good diagrams and deep insights into SOA and how an organization should deliver SOA.

This book was published in 2003 and I kept expecting the information to be dated. There was no mention of an Enterprise Service Bus (ESB) but Doug Kaye describes a number network models, mediation tools and deployment options which have been incorporated into what vendors now label as an ESB. More importantly some of the missing pieces he describes in 2003 are still missing at the end of 2007. Security, transaction integrity and business semantics still do not have accepted solutions within SOA and these missing pieces are described in this book. The WS-* standards have progressed somewhat since the writing of "Loosely Coupled" but basics of SOAP, WSDL and UDDI were in place and many for the WS standards are foretold.

Doug has good perspective on what is required in the corporate world. He advises everyone to have your "elevator pitch" ready for that occasion when the CEO gets into the elevator with you and asks something like "So what's this I keep hearing about web services? What are they and why are they so important" (p27). Doug offers a response that suggests he is either a fast talker or works in a building that is much taller than mine, but the concept of having something ready like this is important if you are going evangelize about web services.

There is only one chapter dedicated to Loose Coupling and Doug has not tried to say everything there is to say on the subject. This justified in his statement "…loose coupling is a methodology or style, rather than a set of established rules and specifications" (p. 131). Figure 10-1 provides a table describing 10 types of coupling and descriptions of tightly and loosely coupled approaches to each. This is a good approach as it is not easy to describe coupling succinctly. I found this approach in another posting although interestingly the contents are different. I have found that in my previous posting that it is difficult to label types and coupling and the three column approach will enable me to do this more effectively. The book covers late binding, heterogeneity, routing and transformations well.

It is true to say that this book is more to do about its subtitle "The Missing Pieces of Web Services" than Loose Coupling. Doug makes the bold prediction that when we have solved all the pieces of the puzzle in the future that it will be so commonplace that "Web services won't be mentioned by the IT trade press, and the topic won't be tracked by analysts". The book searches through these missing pieces in most of the chapters. The fact that Web Services is still a strong topic for analysts and vendors is evidence suggesting that we have not yet found all those missing pieces.

Doug Kaye provides a good description of transaction control although I found the book I read earlier by Krafzig et al to have more detail in this area. Doug has two strong chapters on the problems of web services security and covers this topic in a thorough and systematic way. A couple of reviews have suggested that the book by Mark O'Neil also provides good coverage of Web Service security.

The chapter on deployment option talks about commercial opportunities for third parties in the delivery of services and services infrastructure. This discussed Web Service Networks, Distributed Web Service Networks, Web-Service Delivery Networks and Web Service Providers. Doug takes the service vendor view in his final chapter on providing external services. Both these chapters had little relevance to me. Four years after this book was written I have not been flooded with vendors wanting to sell me services or wanting to host infrastructure for me to operate my web service applications. Perhaps the money is not there, or perhaps it is just easier to use open source products or set up our own infrastructure. I still think Web Services provides some opportunities for commercial intermediaries, but until the vendors are out there and dealing in the government sector it will be difficult for me to get excited about this topic.

Doug writes a couple of good chapters on the management of simple and complex web services projects. In particular he deals with timing the project around web services approaches that have not yet been delivered. The chapter on Service Level Agreements (SLAs) was an important piece of the completion of the web services picture.

The book concludes with a number of checklists that anyone embarking on an SOA project should refer to. This should help the SOA project manager get their ducks in line to navigate the difficult road ahead.

In summary, this was a well written book about the evolving phenomenon of Web Services. There are some valuable insights into Loose Coupling but an exhaustive technical treatment of coupling and the programming patterns that can reduce coupling is out of the scope of this book. There are some important insights to be found, especially on the subjects of security, evangelizing and commercial arrangements. I expect I will refer to this book many times in the future.

Loosely Coupled: The Missing Pieces of Web Services, Doug Kaye ; RDS Press 2003, ISBN 1881378241

Seven Levels of Coupling: Seven Deadly Sins or Seventh Heaven?

In this posting I discuss the Zapthink article by Ronald Schmelzer, "The Seven Levels of Loose Coupling".

This week I have found two commentators redefining loose coupling. The first is described in the discussion about virtual endpoints that was the subject of my previous posting and the second was this Zapthink article. I think it is important we clarify our definitions around SOA to avoid the concept being hijacked by commercial interests. So in this posting and the last I have attempted to sort out whether the discussion is adding to our understanding or muddying the waters.

In a previous posting I have identified 18 different types of coupling and I am still finding more. The first thing to realise about the seven levels of coupling is they are looking at a different attribute of coupling than what I refer to as 'types of coupling'. The levels refer to the entities that are being coupled. My discussion on coupling types looks at the coupling of two modules of program code. This code could be in calling programs, objects, subroutines or services.

The following table lists the seven levels. This table makes the distinction between 'Levels of Coupling' and 'Types of Coupling' clear. The core tenet that Zapthink is discussing is the "loose coupling of Service consumers and providers". This has a different meaning than loose coupling between program code in different services. The main difference is that aspects of the service provider, with the exception of the implementation are not program code at all. Another difference is that a service consumer may not be a service either and it also might not contain program code.

	Aspects of the Service Provider	Possible Types of Coupling
Loose Coupling the Implementation	The service implementation	18 types and still counting
Loose Coupling the Service Contract	Service contract	Interface Coupling, Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Service Policy	Service policy	Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Process	Service-oriented process (Trivial if this is a service)	18 types and still counting
Loose Coupling the Data Schema	Data schema of provider dataset and data schema of interface.	Data Services Layer Coupling, Data Transformation Service Coupling
Loose Coupling the Infrastructure	Infrastructure of provider	Single-vendor SOA platform Coupling
Loose Coupling at the Semantic Layer	Meaning of message attributes	Dynamic service definition Coupling

The seven layers are not as clearly layered as the phrase 'Levels' might suggest. Certainly policies could be considered a higher level than contracts and processes considered at higher level to implementation. The other levels could be considered to sit atop the infrastructure level and under the semantic level but normally data and process would sit beside each other with neither the master. I also see contracts sitting beside data and process as equal partners.

Coupling takes account of the dependency between two entities and the clarity of this dependency. Thus two services exchanging a message are loosely coupled if every dependency is clear from the format of the message. Services have different reasons for being dependent. These reasons are what we mean by 'Types of coupling'.

The types of coupling for the seven levels are described in the third column of table above. The first level describes service to service coupling which I have described earlier. The fourth level is interesting because the solution is almost trivial. The business process should look like a service and therefore all the coupling types for coupling between services also apply for coupling between a consumer service and provider service that is a composition of services used to implement a process. The sort of things that might cause dependencies with contacts, polices, infrastructure, data schemas and semantics are different to the program code dependencies.

Loose coupling of data schema is partly addressed by another Zapthink article which advocates the building of data transformation services rather than incorporating transformation logic inside services used to contain business logic.

Tight coupling between the consumer service and the infrastructure of the provider could be based around a number of proprietary technologies. The ones highlighted by Zapthink are the non-standard ESB and the single vendor SOA platform.

The types listed in the third column of the above table are those implied in the Zapthink article. I expect that a thorough investigation would uncover other types of coupling as well. The point of listing them in the table is to suggest that each coupling level will have its own types and each level (with the possible exception of the process level) needs to be investigated separately.

We can find coupling examples everywhere. For example Carlos Perez and David Walend discuss whether a DBMS and its application are loosely coupled if the application is accessing the DBMS using SQL. This is interesting but not relevant to SOA. For the point of view of defining SOA, I do not believe the seven levels of loose coupling matter. You can have an SOA as long as the services are loosely coupled. However as a model to assist good practice, I think the seven levels will provide value even if they do overload the 'loose coupling' term.

The concepts discussed in relation to levels of coupling are certainly valid. We do not want service consumers dying or behaving incorrectly due to changes with service provider's infrastructure, policies or any of the other aspects described in the seven levels. Zapthink continue to guide us on the path to software heaven, however in the process of introducing 'levels of coupling' Zapthink commit the sin of causing confusion around the important 'loose coupling' term. I recommend the "Seven Levels" article to SOA developers and I suggest developers need to be clear on what is being coupled whenever they use or encounter the term 'loose coupling'.

End Note:
One way to distinguish other levels of coupling from service to service implementation coupling would be to call the different levels something different other than 'loose coupling'. The names that occurred to me were 'loose buckling' or even 'loose zipping' which would be reminiscent of the zapthink integration zipper. Unfortunately both these terms have implications of embarrassing failure if the buckling or zipping is too loose. We do not want our metaphorical trousers around our ankles.