Oracle Redefines Middleware

Oracle Fusion Middleware 11g is the first real attempt to integrate the Oracle Fusion suite with the recent BEA products it acquired in 2008. According to Oracle this "Completes the integration of BEA into Oracle".

Oracle seems to be redefining "middleware" to include such things as Business Intelligence, Content Management and Collaboration which has typically been considered user-facing software. At the other end of the software stack Oracle looks at infrastructure services such as virtualization utilities.

"We take a broad view of what is included in Middleware" Rick Schultz of Oracle says on his podcast. According to the materials that Oracle make available, Middleware seems to be defined as a "Set of components that all applications need". This includes

• User Interaction (Web 2.0, RIA, Mobile, Search)
• Enterprise Performance Management (Planning, Budgeting, Scorecards)
• Business Intelligence
• Content Management
• SOA and Process Management
• Application Server
• Grid Infrastructure
• Development Tools
• Enterprise Management
• Identity Management

Depending on the spin, redefining middleware could be a good or a bad thing. I always felt I understood what middleware was. I agree with Wikipedia that "Middleware is computer software that connects software components or applications." To put it more plainly it is the glue that holds together different parts of a computer application and helps integrate computer applications.

The Oracle list is huge, and I suggest Oracle needs to come up with another name for all this software infrastructure (like Software Infrastructure Suite) which does not suggest it is simply the glue that holds the application together. User facing software infrastructure such as Web 2.0 collaboration facilities, Search, budgeting software, score card software and business intelligence is much more user facing than most custom built software. Other products on the list are part of the operating environment. This is infrastructure that the software developer should have little concern with.

This is not to say that the Oracle Suite is a bad idea. We have to accept that with SOA there is far more of an application that can delivered out of the box rather than written anew by developers. True middleware such as the ESB, is still a significant part of the Oracle Suite but the other components also need to be part of the solution architecture. Oracle focuses on some important aspects from the traditional middleware space including:
• Support for SCA (SOA Component Architecture).
• Support for Event Driven Architecture.

Oracle now has a concerted push to explain its offering to the world. This has resulted in seminars and workshops around the world. I attended a couple and they are worth the time to get an idea about what Oracle is proposing. I wish to thank the presenter for the descriptive term "Marketecture" (aka Marchitecture). This term seemed to gain popularity through Dilbert cartoons in 2009, after having some more serious uses back in 2003. Marketecture is the IT architecture as presented by marketing people and therefore has limited detail and may bear little resemblance to the technical architecture.

In conclusion, there seems to be much goodness in the new Oracle Suite. SOA can benefit from a large number of products to support the approach. Oracle Suite is much more than middleware, but Oracle should not try to cover everything under the "middleware" term just because its long list of acquisitions allows it to do so.

Other references:

http://www.dilbert.com/strips/comic/2009-03-02/, Marketect cartoon
http://www.uml.org.cn/success/pdf/marketecture.pdf, Marketecture vs Tarchitecture
http://www.it-director.com/technology/content.php?cid=6173, Architecture Marchitecture (spelt differently but same idea)
http://www.oracle.com/technology/products/middleware/index.html, The list of products in the Oracle Middleware stack
http://www.oracle.com/products/middleware/docs/microsite09-flashmedia-pdfs/fmw11g-overview-solution-brief.pdf, Overview of Oracle Middleware

Understanding Enterprise SOA: Book Review

I was handed this book by one of my programmers. He lent it to me because he knew I was doing some reading on the subject and trying to work out how to bring my software developers up to speed with SOA. I didn't get the impression that he had got terribly much from the book himself. After reading it I can see there is benefit for managers but I think there was not enough technical meat in the book for programmers and analysts.

This book is not technical. It can be read by managers, salesmen or students and I think these were the intended audience. The explanations are easy to understand although I found them long-winded and overly simplistic on many occasions.

The authors attempted to explain procedural code as if it was museum artifact. This made me feel very old. It begs the questions whether the developers brought up on object-oriented development and SOA really need a detailed explanation of the concept of procedural code.

On the positive side this books tackles issues that others do not. It has a good non-technical discussion about security issues and there is a lot of discussion about human issues such as training, reorganization and corporate culture.

Oddly for a book written in 2006 I did not find any mention of Enterprise Service Bus (ESB) although it was clear the author was encouraging looking at off-the-shelf SOA management solutions as well as SOA security and routing software.

Eric Pulier in his preface is unbridled in his enthusiasm for SOA standards:

In a spectacular cycle of enlightened self-interest, every major company in the world has build on top of standards which have ushered a potentially exponential release of value from investments in information technology.

Elsewhere in the book there are sufficient cautions to temper this enthusiasm. In particular the authors advise that much of SOA is still hard work and that there are no industry standard solutions to problems such as transaction processing and security.

There is a long and detailed case study running through the book. The authors do not hold back on the multitude of issues that confront an organization adoption SOA. Parts are realistic and even painful for those of us who have been through implementations of a difficult project or technology.

The book has a good history of technologies and standards. The authors make a good point about standards conforming to the Tipping Point model (see references below). This is when a standard reaches a certain level adoption that it suddenly becomes desirable for the majority of developers.

This is not a strong academic text. There are not many references to other information sources. The definitions are not clearly stated and often over-simplify the concepts. Concepts like Enterprise Architecture, Real-Time Computing and loose coupling are all not accurately defined.

"Loose-Coupling" is described but not adequately explained. The authors give the impression that loose-coupling is only about where you can locate services.

Web services, in contrast are loosely coupled. Once a piece of software has been exposed as a web service it is simple to move it to another computer.

This is an important aspect of coupling but is not the whole story as I have identified on previous postings.

The book focuses on synchronous services rather than asynchronous services. This is unfortunate as other texts such as Thomas Erl or Krafzig, Banke & Slama seem to advocate asynchronous services.

The book addresses the Subject of Enterprise Architecture again loosely defined. At one point the book states

There is no institute providing credentials for enterprise architects.

I'll give the authors the benefit of doubt. Perhaps in 2006 there were not the credentials around but these days there are a number of places to get recognized credentials as an IT Architect. The Microsoft Certified Architect programs spring to mind although I will not vouch for the content of these programs.

The authors see SOA as a method for replacing parts of the Data Warehouse. I was not particularly convinced by this. This was an interesting subject which I had not seen before. I think there is benefit in moving the content of SOA logging to the data warehouse but I was not convinced that SOA radically changes the typical data warehouse ETL process.

This is not a book with detail examples of SOAP and WSDL. It will not explain how to code or build services. You will not find much detail on WS-* here either.

Contracts and Service Level Agreements (SLAs) are described in the book. The authors also introduce the Business Level Agreement (BLA). This is agreement to monitor business conditions such as an inventory pile-up causing $100k per minute of cost.

The book is well illustrated with copious diagrams and notes attached to the diagrams. I found myself skipping past a lot of the diagrams because they did not seem to be explaining anything particularly special.

I congratulate the authors for including detail about setting up training programs in the book and explaining how the resources are brought in to deal with the adoption of SOA. This is missing from many other texts and is an important issue today.

The authors emphasize that SOA is an enterprise approach. Under SOA, IT departments put in place infrastructure to serve the enterprise rather than just serving particular applications.

I would recommend other books before this one for most audiences. The simplistic and at times pedantic prose would make it hard to recommend to an IT professional. Nevertheless there are some well-made observations in this book that you are not likely to find elsewhere.

References

Understanding Enterprise SOA, Eric Pulier and Hugh Taylor with forward by Paul Gaffney, Manning, 2006

The Tipping Point: How Little Things Can Make a Big Difference (ISBN 0-316-31696-2) is a book by Malcolm Gladwell, first published by Little Brown in 2000.

SOA in a Virtual World

Wikipedia defines a virtual world as a computer-based simulated environment intended for its users to inhabit it. Often a computer application is a business process made virtual by the use of computer technology. We therefore have to be careful when discussing the virtualization of computer environments as this may imply a virtual virtual reality.

The only example of virtual virtual reality I have seen is in a Futurama episode. It displayed an arcade booth with the label 'Virtual Reality' complete with a player using stereoscopic goggles and wired gloves. It then displayed another arcade booth with man sitting in a simple chair imaging he was wearing stereoscopic goggles and wired gloves. The caption to this booth was "Virtual Virtual Reality".

If we talk about "Virtual SOA" we must be talking about something that is not an SOA at all. Judith Hurwitz comes close to this when she discusses a "Virtualized SOA" in her article "Is virtualization the foundation of SOA?"

Andrew Doble writes about building a virtual SOA without investing in in-house software like an ESB. I would argue that this is in fact legitimate SOA. The only thing "virtual" about Andrew's suggestion is the ownership of the software tools.

Virtualization and SOA however do go together. Judith Hurwitz defines Virtualization as

"a technique for abstracting the physical characteristics of computing resources
in order to more easily leverage hardware systems, applications, operating
systems, networks, graphics, data or storage so they can be repurposed based on
customer need."

I believe that loose-coupling is the foundation of SOA but virtualization in many cases allows the benefits from this loose-coupling to be realised. The basic examples of virtualization are virtual servers, storage area networks and thin client platforms such as Citrix. These provide infrastructure flexibility for the deployment of SOA software. The design of the software for an application written using SOA is not affected. The SOA discipline should make sure that flexible deployment options can be used.

Manufacturers such as IBM are building business around SOA infrastructure. One thing to remember is that you do not need virtualization to build SOA. An SOA application can be built on a single server if you required, although it should also give you the flexibility to distribute this application and take advantage of virtualization options.

I agree with Lorraine Lawson in her posting "SOA and Virtualization: Complex, Fuzzy Ideas That Go Great Together? ". These two concepts mix well. However SOA is from the software development world and virtualisation is from the infrastructure world. We should beware of talking in terms of virtualizing our SOA. Our CIOs may consider we are sitting in a corner thinking we have an SOA when this is just a virtual reality.

Fry from "Futurama" with a virtual Lucy Liu.

© 2001 Twentieth Century Fox. All Rights Reserved

More on ESBs

One of my for more popular postings has been "ESB or not ESB". I suspect the only reason was the word "ESB" appeared twice in the title and the Google search engine gave it a high priority when people searched on this acronym.

This was cited in an Italian language blog posting by Andrea Gumina which also cited a good posting by Steve Vinoski . Steve's posting is perceptive and has many good comments on it.

Steve is commenting on an earlier posting by Patrick Logan. Patrick describes the concept of an ESB as a "mangy mutt" because it is a " grab-bag of mechanisms that form no coherent whole". I agree with this assessment but feel that the ESB concept will mature. The ESB may be a mechanism to provide some plumbing. Many organisations SOA efforts need that plumbing because the system delivery has a number of leaks.

My organization was galvanized into inaction and has not yet bought an ESB. However we are fairly typical I would guess because we want the vendors products to do 'everything' for us. IT is not our core business and we do not want to worry about location or services, security, integration, messaging etc. We want our developers to focus on business logic and data models specific to the business. If the vendor came up with an ESB that really did remove the need to spend time on software infrastructure (and did not charge a fortune for it), then we would be interested.

I would also like our developers to settle on a "one best way" to develop applications rather than the organization investing in training and support for a multitude of programming languages and technologies. Steve advocates using the right language for the job and not being confined to one language. This is a noble cause but it is hard work learning another language and becoming proficient in it. Most developers I know have the one language that they would prefer to write everything in, just as most people I know don't communicate in anything but English. If I weren't so lazy I'd teach myself Italian so I could find out what that Italian site was saying about my blog posting. Similarly I think many developers can't realistically be expected to be truly multilingual when it comes to programming languages.

The discussion of Steve's posting gets a hijacked by the REST vs Web Services debate. This is unfortunate because the ESB discussion should be a separate issue. Using SOAP and Web Services does not require an ESB. ESBs have been conceived to assist with REST such as open source ESBs WSO2 and Mule. The more popular arrangements seem to be that Web Services use an ESB and REST does not, but this need not be the case. The REST vs Web Services argument is taken up elsewhere and in some people's minds have already been settled. Ronald Schmelzer of Zapthink states:

In many ways, however, the debate about Web services and REST is as pointless as arguing whether a hammer or a screwdriver is a better tool.

In the discussion in Steve Vinoski's posting I found myself agreeing with Jonas Ekstrom and Paul H that the ESB market is not yet mature. Jonas put this eloquently:

Religions give easy answers, but there are no easy answers. Natural selection will strike again and again until we have reached the perfect SOA.

Paul H puts a similar slant on this:

I look forward to the day when we’re in a similar stage of maturity … and there exists generally accepted, standardized, manageable, and self-federating mediation technologies … until then, we’ll all have to suffer through endless arguments along the lines of 'should I use a BEA ESB or a Sonic ESB? is an ESB really necessary, or can I get away with REST + Dynamic Languages?'

I will continue to wait until the need for an ESB (or some part of one) becomes self-evident. While an implementation of SOA can function without and ESB there seems little point in making the investment.

Book Review: Loosely Coupled - The Missing Pieces…

This is a review of the book "Loosely Coupled: The Missing Piece of Web Services", by Doug Kaye. In a previous posting I expressed delight that an author had dedicated a book to this important topic. This perhaps is not the definitive book on loose coupling but there is plenty of value for the reader.

Doug has an easily readable style and sticks to a high level but there is enough technical substance to keep and IT architect or IT manager interested. You will find no code snippets and no programming standard specifications described in this book but there are some good diagrams and deep insights into SOA and how an organization should deliver SOA.

This book was published in 2003 and I kept expecting the information to be dated. There was no mention of an Enterprise Service Bus (ESB) but Doug Kaye describes a number network models, mediation tools and deployment options which have been incorporated into what vendors now label as an ESB. More importantly some of the missing pieces he describes in 2003 are still missing at the end of 2007. Security, transaction integrity and business semantics still do not have accepted solutions within SOA and these missing pieces are described in this book. The WS-* standards have progressed somewhat since the writing of "Loosely Coupled" but basics of SOAP, WSDL and UDDI were in place and many for the WS standards are foretold.

Doug has good perspective on what is required in the corporate world. He advises everyone to have your "elevator pitch" ready for that occasion when the CEO gets into the elevator with you and asks something like "So what's this I keep hearing about web services? What are they and why are they so important" (p27). Doug offers a response that suggests he is either a fast talker or works in a building that is much taller than mine, but the concept of having something ready like this is important if you are going evangelize about web services.

There is only one chapter dedicated to Loose Coupling and Doug has not tried to say everything there is to say on the subject. This justified in his statement "…loose coupling is a methodology or style, rather than a set of established rules and specifications" (p. 131). Figure 10-1 provides a table describing 10 types of coupling and descriptions of tightly and loosely coupled approaches to each. This is a good approach as it is not easy to describe coupling succinctly. I found this approach in another posting although interestingly the contents are different. I have found that in my previous posting that it is difficult to label types and coupling and the three column approach will enable me to do this more effectively. The book covers late binding, heterogeneity, routing and transformations well.

It is true to say that this book is more to do about its subtitle "The Missing Pieces of Web Services" than Loose Coupling. Doug makes the bold prediction that when we have solved all the pieces of the puzzle in the future that it will be so commonplace that "Web services won't be mentioned by the IT trade press, and the topic won't be tracked by analysts". The book searches through these missing pieces in most of the chapters. The fact that Web Services is still a strong topic for analysts and vendors is evidence suggesting that we have not yet found all those missing pieces.

Doug Kaye provides a good description of transaction control although I found the book I read earlier by Krafzig et al to have more detail in this area. Doug has two strong chapters on the problems of web services security and covers this topic in a thorough and systematic way. A couple of reviews have suggested that the book by Mark O'Neil also provides good coverage of Web Service security.

The chapter on deployment option talks about commercial opportunities for third parties in the delivery of services and services infrastructure. This discussed Web Service Networks, Distributed Web Service Networks, Web-Service Delivery Networks and Web Service Providers. Doug takes the service vendor view in his final chapter on providing external services. Both these chapters had little relevance to me. Four years after this book was written I have not been flooded with vendors wanting to sell me services or wanting to host infrastructure for me to operate my web service applications. Perhaps the money is not there, or perhaps it is just easier to use open source products or set up our own infrastructure. I still think Web Services provides some opportunities for commercial intermediaries, but until the vendors are out there and dealing in the government sector it will be difficult for me to get excited about this topic.

Doug writes a couple of good chapters on the management of simple and complex web services projects. In particular he deals with timing the project around web services approaches that have not yet been delivered. The chapter on Service Level Agreements (SLAs) was an important piece of the completion of the web services picture.

The book concludes with a number of checklists that anyone embarking on an SOA project should refer to. This should help the SOA project manager get their ducks in line to navigate the difficult road ahead.

In summary, this was a well written book about the evolving phenomenon of Web Services. There are some valuable insights into Loose Coupling but an exhaustive technical treatment of coupling and the programming patterns that can reduce coupling is out of the scope of this book. There are some important insights to be found, especially on the subjects of security, evangelizing and commercial arrangements. I expect I will refer to this book many times in the future.

Loosely Coupled: The Missing Pieces of Web Services, Doug Kaye ; RDS Press 2003, ISBN 1881378241

Seven Levels of Coupling: Seven Deadly Sins or Seventh Heaven?

In this posting I discuss the Zapthink article by Ronald Schmelzer, "The Seven Levels of Loose Coupling".

This week I have found two commentators redefining loose coupling. The first is described in the discussion about virtual endpoints that was the subject of my previous posting and the second was this Zapthink article. I think it is important we clarify our definitions around SOA to avoid the concept being hijacked by commercial interests. So in this posting and the last I have attempted to sort out whether the discussion is adding to our understanding or muddying the waters.

In a previous posting I have identified 18 different types of coupling and I am still finding more. The first thing to realise about the seven levels of coupling is they are looking at a different attribute of coupling than what I refer to as 'types of coupling'. The levels refer to the entities that are being coupled. My discussion on coupling types looks at the coupling of two modules of program code. This code could be in calling programs, objects, subroutines or services.

The following table lists the seven levels. This table makes the distinction between 'Levels of Coupling' and 'Types of Coupling' clear. The core tenet that Zapthink is discussing is the "loose coupling of Service consumers and providers". This has a different meaning than loose coupling between program code in different services. The main difference is that aspects of the service provider, with the exception of the implementation are not program code at all. Another difference is that a service consumer may not be a service either and it also might not contain program code.

	Aspects of the Service Provider	Possible Types of Coupling
Loose Coupling the Implementation	The service implementation	18 types and still counting
Loose Coupling the Service Contract	Service contract	Interface Coupling, Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Service Policy	Service policy	Bind-Time Coupling, Service Intermediary Coupling, Infrastructure configuration Coupling
Loose Coupling the Process	Service-oriented process (Trivial if this is a service)	18 types and still counting
Loose Coupling the Data Schema	Data schema of provider dataset and data schema of interface.	Data Services Layer Coupling, Data Transformation Service Coupling
Loose Coupling the Infrastructure	Infrastructure of provider	Single-vendor SOA platform Coupling
Loose Coupling at the Semantic Layer	Meaning of message attributes	Dynamic service definition Coupling

The seven layers are not as clearly layered as the phrase 'Levels' might suggest. Certainly policies could be considered a higher level than contracts and processes considered at higher level to implementation. The other levels could be considered to sit atop the infrastructure level and under the semantic level but normally data and process would sit beside each other with neither the master. I also see contracts sitting beside data and process as equal partners.

Coupling takes account of the dependency between two entities and the clarity of this dependency. Thus two services exchanging a message are loosely coupled if every dependency is clear from the format of the message. Services have different reasons for being dependent. These reasons are what we mean by 'Types of coupling'.

The types of coupling for the seven levels are described in the third column of table above. The first level describes service to service coupling which I have described earlier. The fourth level is interesting because the solution is almost trivial. The business process should look like a service and therefore all the coupling types for coupling between services also apply for coupling between a consumer service and provider service that is a composition of services used to implement a process. The sort of things that might cause dependencies with contacts, polices, infrastructure, data schemas and semantics are different to the program code dependencies.

Loose coupling of data schema is partly addressed by another Zapthink article which advocates the building of data transformation services rather than incorporating transformation logic inside services used to contain business logic.

Tight coupling between the consumer service and the infrastructure of the provider could be based around a number of proprietary technologies. The ones highlighted by Zapthink are the non-standard ESB and the single vendor SOA platform.

The types listed in the third column of the above table are those implied in the Zapthink article. I expect that a thorough investigation would uncover other types of coupling as well. The point of listing them in the table is to suggest that each coupling level will have its own types and each level (with the possible exception of the process level) needs to be investigated separately.

We can find coupling examples everywhere. For example Carlos Perez and David Walend discuss whether a DBMS and its application are loosely coupled if the application is accessing the DBMS using SQL. This is interesting but not relevant to SOA. For the point of view of defining SOA, I do not believe the seven levels of loose coupling matter. You can have an SOA as long as the services are loosely coupled. However as a model to assist good practice, I think the seven levels will provide value even if they do overload the 'loose coupling' term.

The concepts discussed in relation to levels of coupling are certainly valid. We do not want service consumers dying or behaving incorrectly due to changes with service provider's infrastructure, policies or any of the other aspects described in the seven levels. Zapthink continue to guide us on the path to software heaven, however in the process of introducing 'levels of coupling' Zapthink commit the sin of causing confusion around the important 'loose coupling' term. I recommend the "Seven Levels" article to SOA developers and I suggest developers need to be clear on what is being coupled whenever they use or encounter the term 'loose coupling'.

End Note:
One way to distinguish other levels of coupling from service to service implementation coupling would be to call the different levels something different other than 'loose coupling'. The names that occurred to me were 'loose buckling' or even 'loose zipping' which would be reminiscent of the zapthink integration zipper. Unfortunately both these terms have implications of embarrassing failure if the buckling or zipping is too loose. We do not want our metaphorical trousers around our ankles.

Is Your Enterprise Architecture SOA Compliant?

In August I wrote about the Zachman framework and SOA. This was the basis for a couple more slides at the Ark Group conference on interoperability in which I was a presenter. I have covered material from the presentation in couple for previous postings, "Live up to the Legacy that was Left" and "Why SOA?". I will focus on the main theme of my presentation in this posting which was "Ensuring your Enterprise Architecture is SOA Compliant"

Firstly I would like to point out some work that is being done by the SOA Consortium along these lines. They have announced the Enterprise Architecture 2010 Working Group which is to define the high-level roles and deliverables of the Enterprise Architecture team. There is not much tangible yet from this group as it has just been set up but these people also brought us the SOA practitioners' guides so this could prove a valuable initiative.

At the conference mention in the first paragraph I went over the Zachman Framework as I have done in my earlier posting. One thing I pointed out is that different software development approaches can be described in terms of the framework. I then went on a described my organisations use of the Zachman framework using he colour codes I added to the Zachman grid.

When we talk about top-down approaches conveniently you work from the top row (Scope Row) down to the bottom row (Detailed Representation). Similarly when you work bottom-up you work from the bottom of the framework representation.

Assuming that most good software development is done top-down (and I am subscriber to this approach) you can choose different methodologies depending on the column you choose.

If you drive down the 'Data' column you have a Data Driven Architecture. The Enterprise Information Model in this approach is all important. The other cells of the framework will either support the data column or be influenced by it.

The SOA approach is usually focused on business process and this means it is being driven by the 'Function' column. Software development can be driven from Business Process Modelling and 'Function' column without being SOA.

If we are two focused driving our software development from the 'Network' column we may end up with a siloed application based on organisational locations. This generally software development anti-pattern as modern enterprise software should be independent of location.

Similarly, too much focus on the 'People' column can lead to application focussing around individual or organisational units. This is a very common siloed approach and anti-pattern. More positively a people driven architecture will look at the user experience. This is a common approach for smaller applications to look at screen designs, report layouts and user interaction scenarios that will meet user requirements. For enterprise application though it is important to look at the bigger picture, to question what the organisation is trying to achieve and to model the process from start to finish otherwise the people siloed anti-pattern may result.

If we drive down the 'Time' column we will end up with an Event Driven Architecture (EDA). This is an alternate approach to SOA. We look at events and how they initiate and affect business processes. The 'Function' columns and 'Time' columns can be worked with in tandem to produce and event driven SOA application.

If we work down the 'Motivation' column of the Zachman frame work then we are using a Business Rules Approach. My organisation has not made great use of this framework column. For more information about the Business Rules Approach see http://www.businessrulesgroup.org/ . Also this column is valuable for expert systems and decision management.

With the Zachman framework you do not have to fill every cell. You certainly do not only have one enterprise model in each cell. How you uses the Zachman framework is important. So I will describe how our organisation uses it in terms of the diagram above.

The yellow cells in the above diagram are areas where we have not got any significant documentation.

The green cells contain documentation the business has prepared with out prodding from the IT department. The brown cells are documented by IT but with the interaction of the business.

The top two layers do not change much. These are business level documents. Producing good business process models is more important with SOA. The business process is the key in the SOA process not the application.

The blue is driven by SOA. It starts with your Application Architecture. Distributed Architecture works differently with SOA. You might consider and ESB for instance which will give a different architecture than if you use point-to-point communication. These upper two blue cells drive changes at the more detail level below them.

The pink areas are affected by SOA. Data modelling and Human interface architecture are not radically different but they have to take into account the new architecture.

I will now look at the link between EA and SOA in a different way.

Zapthink has a good article on Zachman and SOA where they use arrows emanating from the Application Architecture cell of the framework. I prefer to represent the influence direction as predominately down representing a top-down approach.

We will start in the 'Data' column again. I do not think the Information Model should be different as a result of SOA. However the Logical Data Model probably should. It should document the interfaces and be able to transform the representation in XML. It should also document the persistent data stores. You now have at least two levels of logical data modelling.

The physical data model and data definition can then be XML, XSD and WSDL. At the same time you have your modelling of persistent data which would be your normal Class Model or ER diagram. This data model would eventually be represented in data dictionaries and database schemas.

We will move across to the 'Function' column now. The Business Process Model should influence your application architecture. Application Architecture should advise you of what services you need and what services to reuse. A service becomes part of the enterprise architecture because it is built for reuse. A new service is also part of a solution architecture. The service has to meet both solution and enterprise needs. Also in this cell is your application stack which supports the service model. The service model then drives the more detailed System Design and the programming.

SOA is a distributed architecture so the lower rows of the 'Network' column takes on increased significance compared with traditional software development approaches. There is more flexibility about where you do your processing. This needs to be built into your Distributed System Architecture.

Business logistics (and particularly whether you are dealing with agencies external to the organisation) will impact your applications architecture. There is a stronger linkage from Application Architecture to Distributed System Architecture because SOA is an enabling technology. It enables you to distribute processing, to look at load balancing and to look at interfaces with other agencies. There is therefore a strong influence on distributed architecture increasing the importance of good architecture in this space.

As with the other columns the influence downward is strong. Your Technology Architecture and Network Architecture needs to be built to cope with the new Distributed Systems Architecture.

Human interface architecture in the 'People' column may change to suit SOA but it does not have to. Some interface technologies work better with SOA. Some ways of interacting with an application work better with SOA but if you want to drive it the other way by specifying your Human Interface Architecture first then this can also work.
Security Architecture gets its own arrow. It is not a logical progression from Presentation Architecture. You may have to do security differently under SOA. You may have to make better use of a standard infrastructure. You may have to use SOA security standards.

SOA for my organisation is business process driven although there is still some bottom up development resulting from services based on mainframe data entities. The focus on reuse and the question for what we already have available to reuse demands some bottom-up consideration.

To be SOA compliant your architecture needs to include:

• Documented business processes,
• Enough bottom-up analysis to reuse of existing services, and
• Data modeling of web service interfaces.

This is in addition to SOA infrastructure and distributed models required the lower levels of your architecture. SOA is an Enterprise Architecture approach. You cannot have an effective SOA unless you have some level of Enterprise Architecture. If you focus on a single solution or application and do not look at the wider needs of the enterprise then you are going to miss the major benefits of SOA.

The business process focus and implications around reuse mean that the business needs to be aware of the SOA approach. It is a change to business practice that the SOA consortium is looking at in its Enterprise Architecture 2010 Working Group.

ESB or not ESB?

I work for an organisation that is pretty well fixed on achieving a goal of an SOA but we are still wondering how to get there. The worst people to ask are software vendors, because the answer will always be to buy more software. This stuff costs big time though.

For the cost it took to set up our application servers I could have bought two comfortable suburban homes and then we pay the software maintenance which I have to invest in at the cost of having a couple of staff. It is not as if I can do without the staff though. I now have to recruit someone to administer the application server and find developers with the skills in this particular applications server. So when the software vendor tells me that all I need to is the pay the cost of another suburban house and sacrifice funds for another staff member to buy an ESB I need to be pretty clear that this will provide some benefit.

There is some disagreement about what an ESB is or what benefit it brings. Mark Richards looks at some of the definitions of ESB in his presentation for the NFJS conference (video clip at http://www.infoq.com/presentations/Enterprise-Service-Bus). He then defines the ESB in terms of the capabilities it might bring these are listed as:

1. Routing
2. Message Transformation
3. Message Enhancement
4. Protocol Transformation
5. Service Mapping
6. Message Processing
7. Process Choreography
8. Services Orchestration
9. Transaction Management
10. Security

If software contains about five or more of these capabilities then it is an ESB. This would normally include first three capabilities. Together Mark Richards refers to these as "mediation".

This list is handy because it allows you to look at your needs and determine whether you should purchase such a product with these capabilities. You can determine whether these capabilities are available in your application services already, whether you need to acquire a niche product to address a capability or whether you can handle some of this capability in your own code.

This list is also useful to determine what you are not getting in a particular product offering. One consultancy firm demonstrated to me a great way of addressing the routing problem in application code and claimed I did not need an ESB because of this. It was certainly useful but it will not help if I need the other nine capabilities.

There is certainly some difference of opinion on whether you need and ESB. Vendors will say you need to have an ESB to do SOA. Sandy Carter of IBM in her web presentation (http://www.infoq.com/interviews/sandy-carter-soa) says that an ESB is a a "crucial part of an SOA" and a "spinal chord of your business".

On the other hand a presentation by Jim Webber of Thoughtworks is critical of the ESB approach and believes it works against a good SOA (http://jim.webber.name/downloads/presentations/2006-10-EJA-Guerrilla-SOA.ppt). Jim claims that ESB adds a proprietary middleware to you SOA which will lock you an your software in to a particular vendor. He goes onto to say that "spaghetti" in SOA is inevitable and one of the strengths of SOA is that it copes with this. ESB on the other hand can make spaghetti worse.

One of the best quotes I found was in article by Colleen Frye discussing ESBs with Jason Bloomberg and others:

This bandwagon's a bus and ... vendors positioning in the SOA market have jumped on it over the last few years.

I don't believe my organisation is mature enough in its SOA progress to consider and ESB. We have to develop a critical mass of services before we should look at investing in more glue (as Jim Webber describes it) to organise and bring these services together.

References:

Mark Richards http://www.infoq.com/presentations/Enterprise-Service-Bus
I liked marks insight in that SOA "Presents IT Services as Business Services".
Also he mentioned a couple of open source ESBs
Mule: http://mule.codehause.org/
ServiceMix: http://servicemix.org/ (JBI = JSR-208 compatible).

Sandy Carter: http://www.infoq.com/interviews/sandy-carter-soa
Some good parts of this talk excuse her from being a vendor and almost excuse her from using the word "leverage" too much.
"ESB as a pattern of SOA."
A quote from Steve Mills "SOA will be with us for decades"
"I think that BPM and SOA are two sides of the same coin. I don't think you can do BPM without leveraging SOA."

Jim Webber: http://jim.webber.name/downloads/presentations/2006-10-EJA-Guerrilla-SOA.ppt
This is a powerpoint slide presentation. He is also promoting his book that looks interesting.

Colleen Frye: ESB Market Report, http://searchsoa.techtarget.com/originalContent/0,289142,sid26_gci1226497,00.html