Understanding Enterprise SOA: Book Review

I was handed this book by one of my programmers. He lent it to me because he knew I was doing some reading on the subject and trying to work out how to bring my software developers up to speed with SOA. I didn't get the impression that he had got terribly much from the book himself. After reading it I can see there is benefit for managers but I think there was not enough technical meat in the book for programmers and analysts.

This book is not technical. It can be read by managers, salesmen or students and I think these were the intended audience. The explanations are easy to understand although I found them long-winded and overly simplistic on many occasions.

The authors attempted to explain procedural code as if it was museum artifact. This made me feel very old. It begs the questions whether the developers brought up on object-oriented development and SOA really need a detailed explanation of the concept of procedural code.

On the positive side this books tackles issues that others do not. It has a good non-technical discussion about security issues and there is a lot of discussion about human issues such as training, reorganization and corporate culture.

Oddly for a book written in 2006 I did not find any mention of Enterprise Service Bus (ESB) although it was clear the author was encouraging looking at off-the-shelf SOA management solutions as well as SOA security and routing software.

Eric Pulier in his preface is unbridled in his enthusiasm for SOA standards:

In a spectacular cycle of enlightened self-interest, every major company in the world has build on top of standards which have ushered a potentially exponential release of value from investments in information technology.

Elsewhere in the book there are sufficient cautions to temper this enthusiasm. In particular the authors advise that much of SOA is still hard work and that there are no industry standard solutions to problems such as transaction processing and security.

There is a long and detailed case study running through the book. The authors do not hold back on the multitude of issues that confront an organization adoption SOA. Parts are realistic and even painful for those of us who have been through implementations of a difficult project or technology.

The book has a good history of technologies and standards. The authors make a good point about standards conforming to the Tipping Point model (see references below). This is when a standard reaches a certain level adoption that it suddenly becomes desirable for the majority of developers.

This is not a strong academic text. There are not many references to other information sources. The definitions are not clearly stated and often over-simplify the concepts. Concepts like Enterprise Architecture, Real-Time Computing and loose coupling are all not accurately defined.

"Loose-Coupling" is described but not adequately explained. The authors give the impression that loose-coupling is only about where you can locate services.

Web services, in contrast are loosely coupled. Once a piece of software has been exposed as a web service it is simple to move it to another computer.

This is an important aspect of coupling but is not the whole story as I have identified on previous postings.

The book focuses on synchronous services rather than asynchronous services. This is unfortunate as other texts such as Thomas Erl or Krafzig, Banke & Slama seem to advocate asynchronous services.

The book addresses the Subject of Enterprise Architecture again loosely defined. At one point the book states

There is no institute providing credentials for enterprise architects.

I'll give the authors the benefit of doubt. Perhaps in 2006 there were not the credentials around but these days there are a number of places to get recognized credentials as an IT Architect. The Microsoft Certified Architect programs spring to mind although I will not vouch for the content of these programs.

The authors see SOA as a method for replacing parts of the Data Warehouse. I was not particularly convinced by this. This was an interesting subject which I had not seen before. I think there is benefit in moving the content of SOA logging to the data warehouse but I was not convinced that SOA radically changes the typical data warehouse ETL process.

This is not a book with detail examples of SOAP and WSDL. It will not explain how to code or build services. You will not find much detail on WS-* here either.

Contracts and Service Level Agreements (SLAs) are described in the book. The authors also introduce the Business Level Agreement (BLA). This is agreement to monitor business conditions such as an inventory pile-up causing $100k per minute of cost.

The book is well illustrated with copious diagrams and notes attached to the diagrams. I found myself skipping past a lot of the diagrams because they did not seem to be explaining anything particularly special.

I congratulate the authors for including detail about setting up training programs in the book and explaining how the resources are brought in to deal with the adoption of SOA. This is missing from many other texts and is an important issue today.

The authors emphasize that SOA is an enterprise approach. Under SOA, IT departments put in place infrastructure to serve the enterprise rather than just serving particular applications.

I would recommend other books before this one for most audiences. The simplistic and at times pedantic prose would make it hard to recommend to an IT professional. Nevertheless there are some well-made observations in this book that you are not likely to find elsewhere.

References

Understanding Enterprise SOA, Eric Pulier and Hugh Taylor with forward by Paul Gaffney, Manning, 2006

The Tipping Point: How Little Things Can Make a Big Difference (ISBN 0-316-31696-2) is a book by Malcolm Gladwell, first published by Little Brown in 2000.

Book Review: Loosely Coupled - The Missing Pieces…

This is a review of the book "Loosely Coupled: The Missing Piece of Web Services", by Doug Kaye. In a previous posting I expressed delight that an author had dedicated a book to this important topic. This perhaps is not the definitive book on loose coupling but there is plenty of value for the reader.

Doug has an easily readable style and sticks to a high level but there is enough technical substance to keep and IT architect or IT manager interested. You will find no code snippets and no programming standard specifications described in this book but there are some good diagrams and deep insights into SOA and how an organization should deliver SOA.

This book was published in 2003 and I kept expecting the information to be dated. There was no mention of an Enterprise Service Bus (ESB) but Doug Kaye describes a number network models, mediation tools and deployment options which have been incorporated into what vendors now label as an ESB. More importantly some of the missing pieces he describes in 2003 are still missing at the end of 2007. Security, transaction integrity and business semantics still do not have accepted solutions within SOA and these missing pieces are described in this book. The WS-* standards have progressed somewhat since the writing of "Loosely Coupled" but basics of SOAP, WSDL and UDDI were in place and many for the WS standards are foretold.

Doug has good perspective on what is required in the corporate world. He advises everyone to have your "elevator pitch" ready for that occasion when the CEO gets into the elevator with you and asks something like "So what's this I keep hearing about web services? What are they and why are they so important" (p27). Doug offers a response that suggests he is either a fast talker or works in a building that is much taller than mine, but the concept of having something ready like this is important if you are going evangelize about web services.

There is only one chapter dedicated to Loose Coupling and Doug has not tried to say everything there is to say on the subject. This justified in his statement "…loose coupling is a methodology or style, rather than a set of established rules and specifications" (p. 131). Figure 10-1 provides a table describing 10 types of coupling and descriptions of tightly and loosely coupled approaches to each. This is a good approach as it is not easy to describe coupling succinctly. I found this approach in another posting although interestingly the contents are different. I have found that in my previous posting that it is difficult to label types and coupling and the three column approach will enable me to do this more effectively. The book covers late binding, heterogeneity, routing and transformations well.

It is true to say that this book is more to do about its subtitle "The Missing Pieces of Web Services" than Loose Coupling. Doug makes the bold prediction that when we have solved all the pieces of the puzzle in the future that it will be so commonplace that "Web services won't be mentioned by the IT trade press, and the topic won't be tracked by analysts". The book searches through these missing pieces in most of the chapters. The fact that Web Services is still a strong topic for analysts and vendors is evidence suggesting that we have not yet found all those missing pieces.

Doug Kaye provides a good description of transaction control although I found the book I read earlier by Krafzig et al to have more detail in this area. Doug has two strong chapters on the problems of web services security and covers this topic in a thorough and systematic way. A couple of reviews have suggested that the book by Mark O'Neil also provides good coverage of Web Service security.

The chapter on deployment option talks about commercial opportunities for third parties in the delivery of services and services infrastructure. This discussed Web Service Networks, Distributed Web Service Networks, Web-Service Delivery Networks and Web Service Providers. Doug takes the service vendor view in his final chapter on providing external services. Both these chapters had little relevance to me. Four years after this book was written I have not been flooded with vendors wanting to sell me services or wanting to host infrastructure for me to operate my web service applications. Perhaps the money is not there, or perhaps it is just easier to use open source products or set up our own infrastructure. I still think Web Services provides some opportunities for commercial intermediaries, but until the vendors are out there and dealing in the government sector it will be difficult for me to get excited about this topic.

Doug writes a couple of good chapters on the management of simple and complex web services projects. In particular he deals with timing the project around web services approaches that have not yet been delivered. The chapter on Service Level Agreements (SLAs) was an important piece of the completion of the web services picture.

The book concludes with a number of checklists that anyone embarking on an SOA project should refer to. This should help the SOA project manager get their ducks in line to navigate the difficult road ahead.

In summary, this was a well written book about the evolving phenomenon of Web Services. There are some valuable insights into Loose Coupling but an exhaustive technical treatment of coupling and the programming patterns that can reduce coupling is out of the scope of this book. There are some important insights to be found, especially on the subjects of security, evangelizing and commercial arrangements. I expect I will refer to this book many times in the future.

Loosely Coupled: The Missing Pieces of Web Services, Doug Kaye ; RDS Press 2003, ISBN 1881378241

SOA Best Practices: A Book Review

"Enterprise SOA: Service-Oriented Architecture Best Practices" is by Dirk Krafzig, Dirk, Karl Banke and Dirk Slama. These are three Germans from the banking and insurance sector who present a wealth of practical experience to the reader. They attempt to talk about SOA without favoring any particular technology which is hard to do. I would still say that there experience with EJBs and CORBA provided a certain slant to the discussions however they were also using SOAP, WSDL and XML examples in their discussions.

The book had the feel of describing the real world rather than discussing and ideal world. By contrast the book by Erl was from a Web Services purist's point of view and glossed over some practical issues like what to do about user interfaces, performance problems and legacy transaction integrity. On the other hand Erl had the advantage of not having to describe the concepts in terms of differing technologies.

Legacy integration issues were covered well by Krafzig et al. There was an excellent chapter on process integrity and a good chapter on infrastructure issues like Scalability, Interoperability and Security. They covered the range of process integrity techniques from "Eyes Closed and Praying" to "Two Phase Commit over a heterogeneous environments" and "SAGAs with complex compensation graphs".

Krafzig et al provide more of "Horses for Courses" approach. Their approach does not necessarily advocate the same approach for fine grain and course grain services. Their approach even accepts that synchronous as well as asynchronous service buses might need to work side by side.

One theme was the importance of the registry and repository for SOA. The book gives some practical suggest about what to put in these stores. Another theme is the importance of governance of the development process. This theme is taken up in a later chapter which discusses SOA-driven project management, configuration management and testing. The book also discusses a thin-thread approach to software development. This is accompanied by discussions on how to fund SOA development and how to manage the risk of these initiatives.
The climax and conclusion of the book is four case studies. These are real-life examples with solutions that had to meet compromises and struggled with incorporating legacy environments with their solutions. It would have been nice to have an example that was not a finance company but it seems that is where the money is when it comes to SOA (and any big information systems for that matter).

The book did not provide a motivating definition of SOA although the thoughts on architecture I found enlightening. Such as "architecture … is designed to last longer than one or two of these technology innovation cycles" and that its purpose is to simplify development. How easy it is to forget that our new inventions in IT are just to make the world a simpler place.
I enjoyed this book. It was good to read a European context on SOA. They referenced Niklaus Wirth (a hero to anyone who did computer science when I did it) a couple of times and referred to other European IT Gurus which is refreshing break from the US IT heritage. I commend it to those who want a technology-agnostic look at SOA.

References
Erl, Thomas, Service-Oriented Architecture: Concepts, Technology, and Design, 2005 Prentice Hall

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004

Getting Independent Advice

Last Tuesday we got a couple of consultants from interstate to advise us on how my organization is sailing with its initial steps toward SOA. Their brief was fairly broad. They spent a couple of days meeting with various groups including management, legacy developers, java developers and architects.

The idea was to get some idea of what we are doing wrong, what we doing right and what we should do in the future. This is a good thing to do if you can find respected practitioners who can provide this advice. It is easy to be insular and head off in your own direction without considering other options. This can apply to the organization, to groups of organisations and it can even apply to whole regions. I always object when the Federal Government or the eastern states regard South Australia as a regional outpost, but when it comes to best practice in IT we have to acknowledge that our expertise in major enterprise application development can be thin.

I was involved in a few of the meetings. I expect them to focus on our run-time stack, our development standards and development methodologies. The major focus was on our legacy software. This is a big problem and one which I was happy for them to take head on. We have many large systems written on the mainframe in technologies that are out of date and do not easily lend themselves to being accessed through services. The few tools that are available to assist are prohibitively expensive.

The report from the consultants has not come in yet but I was able to glean areas that the consultants seemed to be targeting. One was how to address issues such as Data Integrity, Process Integrity, processing exceptions and auditing in relation to making providing services using our legacy applications. These are the concepts discussed in chapter 8 of Krafzig et al "Managing Process Integrity". They are common application integration problems which pre-date SOA and they are the sort of problems that do not yield to simply adopting particular middleware or standards.

The other issue that the consultants were targeting was how much we were investing in our SOA. We are trying to fund SOA by paying for infrastructure and the development of skills out of project funding for the development of particular applications and enhancements. This may not be sufficient to get the critical momentum we need to get to a desired maturity with our SOA. It would be a good outcome if our independent advisers can report to our executive that we need more money specifically for SOA. It is a bit hard to do this from the perspective of the IT Department. The executive would suspect we were investing the money for no good business reason, perhaps just for buying new toys for IT. The opinion of the consultants may help to put our case.

Of course the consultants might be hoping we spend more money on them. We have to be on lookout for such bias in consultant findings. In general though it is always valuable to get a second opinion and I eagerly await the report from this exercise.

Reference

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004

Putting a Face to SOA

There seems to be some things missing from much of the discussion on SOA. In a couple of books I've read SOA as touted as an all-encompassing approach to developing systems but issues on data modelling and user interface are almost completely ignored. These are pretty fundamental parts of or software. Early computer systems were nothing but user interface and database. Now there is so much in between such as business logic and middleware that this all needs to be regarded as the most important part of the architecture. So are there recommendations about how to do the data model and user interface in SOA?

Firstly, does SOA affect the logical data model compared with traditional architectures? My answer is that the method of obtaining the data model may be different but I'm not sure whether the resulting data model is any different from traditional methods. The focus has to be on the message format if the development is to be driven WSDL first. I will probably expand on this in later post but for now I will focus on the user interface for the rest of this post.

The book Service-Oriented Architecture: Concepts, Technology, and Design by Thomas Erl provides good description of SOA and web services but the issue of user interface is never broached. The reader is left to wonder whether the user interface is somehow built into the services or sits outside it. Another book Enterprise SOA: Service-Oriented Architecture Best Practices by Dirk Krafzig, Karl Banke and Dirk Slama explicitly states that the user interface sit outside the service. An article by some IBM authors provides both scenarios. They observe that " The Model-View-Controller (MVC) paradigm underlies most modern UI application frameworks. SOA operations provide the model layer, and UIs occupy the view layer". (Hepper et al)

The IBM article discusses Web Services for Remote Portlet (WSRP) to enable a portal to aggregate content from multiple sources. The intention of WSRP is so that "Content and application provider services can be discovered and plugged into standards-compliant applications without any extra programming effort". This described more succinctly in another article "A remote portlet is a portlet that's deployed in a different environment than the portal that actually uses it" (Ort). The approach assumes the software developer has a portal for the portlet deployment.

The IBM article also looks at the human workflow. If all human interactions can be managed by forms within a workflow product then programmatic user interfaces could presumably disappear. The human workflow is included in the BPEL process designed to orchestrate the SOA. This may be good for simple forms but this is not going to suit all purposes.

There seems to be a weight of industry acceptance behind Rich Internet Applications in general and AJAX in particular. This user interface for SOA uses makes good use of the XML used in web services messages uses asynchronous messaging and is sufficient rich to provide users all they need from a simple web browser. This approach seems to be supported by Zapthink, Tibco and Oracle.

A number of products seem to be appearing that target the SOA user interface space. BEA systems, Sun Microsystems, CapeClear Software, Tibco (see footnote) all plan to add Ajax tools to their SOA-centric products (Meehan). Other solutions will also become evident from vendors like Corizon that use approaches to SOA user interfaces that differ to the AJAX approach.

The combination of AJAX and WSRP seems well accepted by the industry. "These two technologies complement each other in terms of ease of sharing of aggregated content and better usability of such content" (Padmanabhuni et al). AJAX support will be included in WSRP version 3 (Dovey). These technologies have been included in both the IBM Websphere and BEA Weblogic portal products. AJAX allows the interfaces with sufficient richness to be developed for Web Service applications and WSRP allows interfaces to be deployed and aggregated into portals easily.

It looks like the choice to use SOA and the choice over interface technology will always remain fairly independent although not unconnected. This could be an advantage as it will mean that that good user interface technology can develop reasonably independently for the evolving web services standards. A portal seems a generally accepted starting point for an SOA user interface. AJAX and WSRP are popular technologies to take advantage of web services. The human workflow approach as way for users to interact with SOA is worth watching. This will be an effective way to build workflow applications that have simple form-based user interfaces.

Footnote
Comments received on this blog suggest that Tibco have already released Ajax tools for their SOA product offering.

References
Erl, Thomas, Service-Oriented Architecture: Concepts, Technology, and Design, 2005 Prentice Hall

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004

Hepper, S, Liesche, S and Stockton, M, http://www.ibm.com/developerworks/library/ws-soa-progmodel5/index.html

Ort, Ed, What's New in SOA and Web Services? http://java.sun.com/developer/technicalArticles/WebServices/soa2/WhatsNewArticle.html

Meehan, Michael, Vendors look to Ajax to make SOA shine
http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1144429,00.html

Padmanabhuni, S, Kunti, K, Sahoo, L and Bharti, S, Coupling RDF/RSS, WSRP and AJAX for Dynamic Reusable Portlets: An Approach and a Use Case, http://doi.ieeecomputersociety.org/10.1109/SERVICES.2007.26

Dovey, Matthew, WSRP2 and JSR286,
http://www.nesc.ac.uk/talks/686/10-00_Dovey-WSRP2-JSR286.ppt

Seeley, Rich, Ajax tools from IBM and real-time Java for SOA from BEA,
http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1213063,00.html

Seeley, Rich, BEA bets its portal on WSRP, Ajax,
http://searchwebservices.techtarget.com/originalContent/0,289142,sid26_gci1198691,00.html

Davies, David, SOA at the user interface, http://www.looselycoupled.com/opinion/2006/davies-ui-dev0424.html

Tibco, Rich Portals: The Ideal User Interface for SOA,
http://www.tibco.com/resources/mk/rich_portals.pdf

Bloomberg, Jason, Corizon: Leveraging User Interface Services for Rapid Application Creation
http://www.zapthink.com/report.html?id=ZTZN-1216

Bragging Rights on Coupling

I am reading "Enterprise SOA: Service-Oriented Architecture Best Practices" by Krafzig, Banke and Slam and I have come across a section on "Tight Versus Loose Coupling". This is a topic of my recent post" Cohesion, Coupling and SOA.

In my earlier post I described the taxonomy from loose coupling to tight coupling provided in a Pressman textbook
· No direct coupling
· Data Coupling
· Stamp Coupling
· Control Coupling
· External Coupling
· Common Coupling
· Content Coupling

I then added to this Message Coupling which was suggested by Thomas Erl. Now Krafzig et al is suggesting there is more to message coupling than just a single category. The book describes seven dimensions of messaging coupling. The 'coupledness' created by messages will depend on
The physical link (eg. Whether messages are put on a queue or the communication is a point to point communication).

• The communication style (eg. Asynchronous or synchronous)
• The type system (eg. Is the type determined by the called service interface or the message payload)
• The interaction pattern (eg. A service is more loosely coupled if it receives a message, then processes it and then can forget about it such as a stateless service. Tight coupling is required if for example the service uses an internal variable to keep track of the state)
• Control of Process Logic (eg. Additional coupling can be added to impose transactional integrity)
• Service discovery and binding (eg. Are services discovered through UDDI or similar means or are they directly referenced? )
• Platform dependencies (eg. Tighter coupling results if the service must always invoked from the same technical platform).

This threatens to make a mess of the neat one-dimensional coupling spectrum but we must remember that the spectrum of coupling types is only a guide and that it was always possible to have combinations of coupling types. I develop a priority listing below.

What was meant by message coupling in my earlier post was focusing on the asynchronous nature of messages. All the Pressman coupling types assume synchronous procedure calling and this was proposed before the time of distributed computing so they all are fundamentally coupled by the fact the modules were running on the same piece of hardware. Distributed computing warrants its own sort of coupling. The new spectrum might look like (from loose to tight coupling):

0. No direct coupling
1. Services Registry Coupling
2. Message Payload Coupling
3. Queue Coupling
4. Asynchronous Coupling
5. Plane Old Message Coupling
6. Bind-Time Coupling (added from later posting)
7. Process Control Coupling
8. Creation Coupling (added from later posting)
9. Process State Coupling
10. Platform Coupling
11. Distributed Coupling
12. Data Coupling
13. Stamp Coupling
14. Control Coupling
15. External Coupling
16. Common Coupling
17. Code Coupling (added from a later posting)

Rather than use the dimensions of Krafzig et al I have used labels of describing and example of a a coupling category within that dimension. In most cases it is the looser coupling category. This is consistent with the earlier coupling types. Therefore I use 'Queue Coupling' for the 'Physical Link' of Krafzig et al and 'Asynchronous Coupling' is used instead of 'Communication Style'.

I have inserted "Plane Old Message Coupling" in at number 5. In fact all coupling types from 1 to 9 are message coupling. Coupling types 7,8 and 9 increase the coupling between services. Coupling type 1 to 4 loosen the coupling compared to a synchronous message, called using its inteface without the assistance of queuing or a registry.

As discussed in the earlier posting the classic coupling types are still relevant to SOA. If you use a tight coupling type then it does not matter what good things you have to decouple your services there will still be tight coupling.

The numbers in the above list can be used as score. A low score is good loose coupling. For example if you are sending messages with Boolean values that significantly effect the processing in the called services then you have control coupling and you score badly with a 14. It does not matter that you are using asynchronous message and type is based on the message payload. The services are still tightly coupled. Similarly if you send a huge amount of XML and the service only uses a part of this then this is 'Stamp Coupling' and can only score an 13.

Loose coupling in our ideal SOA applications requires our services to be recorded in a registry, to be driven by the message payload, to use queues to communicate, to be asynchronous, to not have overriding process control and to be stateless. These services are platform and hardware independent. The payload should not contain extraneous data, or control values. The operation of the services should not communicate using external methods or use common shared data areas.

If I ever write a software development text I think this would be useful content. Perhaps this can be used as bragging rights however I am not the sort of person to say something like "How many types of coupling do you know, I know 18".

References

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004, Section 3.5

Pressman, Roger, Software Engineering: A Practitioners Approach, 1982 McGraw Hill

Erl, Thomas, Service-Oriented Architecture: Concepts, Technology, and Design, 2005 Prentice Hall