Live up to the Legacy that was Left …

I presented at a conference last week. After discussing my organisation I provided my definition for SOA and a bit about why my organisation was interested in SOA. I then got onto the topic of this posting which is our legacy transition and in particular how we were going to use SOA to move away from mainframe technologies.

First I presented an age profile of our applications which showed many of our more strategic systems were over 20 years old. Our more integrated systems are written using the IDMS database and ADSO/COBOL. Our more siloed applications are written using the ADABAS database and the NATURAL language.

The mainframe is costly and inflexible environment. It is hard to find developers who know IDMS and the software vendors don’t provide much support for this database technology any more. It is not as agile as a relational database management system. The same is true for our ADABAS applications. It is difficult to find skills. There is at least more of an upgrade path provided by Software AG to move onto newer technologies.

I presented a slide of a plan that was developed more than 4 year ago. It is illustrated here.

The above slide depicts moving from a mainframe environment to a transition stage consisting of an environment where we have some mainframe applications and some J2EE applications accessing mainframe data. The business logic layer is going to be moved off the mainframe leaving applications accessing mainframe data using SQL adapters. The next step is that we move completely off our legacy databases onto relational databases.

This is still the plan. We have made some progress, but progress is slow.

On the right hand side we had n-tier J2EE applications. Now we hope to target a Services Oriented Architecture. This will give us good integration to other agencies and good integration to national bodies. It will assist us to break up the legacy applications and migrate bits of them to newer technologies. It will assist SAPOL to be more agile in its development of software changes.

There is a flaw in this diagram though. This is meant to be a gradual process. We are not going to rewrite a whole suite of applications quickly. We do not have a big budget to break the back of this work. We do not want to risk a huge project that takes years to implement.

Where is the business logic in the old mainframe environment? It is on the mainframe.

Where is the business logic in the co-existence? It is all in the newer technology (blue) areas. What we are finding is that there is so much we have to move to the newer area before we have our first significant application. The step to the transition stage is too big and too risky.

Version 2 of our transition to legacy is illustrated here. Here we have repeated the transition stage and the new environment stage. There is not much difference in the New Environment stage. Except we have encapsulated our business logic and data access in services.

The main difference in the transition stage s that Business Services that are depicted as sitting on the mainframe.

The hope is that we can feasibly wrap up all the mainframe logic in services and reorganise it in custom written business services and combine it with services already written in the new environment.

We would have to make some compromises about what these services might look like, and this would drive a bottom up process in the lower levels of the architecture. But this means that we gradually move our business services over from mainframe to the new environment without having to write huge amounts of business logic at once.

This is done at present using some custom built techniques that duplicates some of the business logic in the mainframe. This is probably not sustainable for rewriting all of the mainframe business logic. Another solution is to encapsulate with more automation. This can be done according to vendors (Neon and iWay). We can encapsulate our existing green screen forms into services without much fuss according the vendors. In actual fact these service adapters cost an arm and a leg and installation and configuration is agony, but that is a familiar story.

The full quote in the title of this posting is

It is up to us to live up to the legacy that was left for us, and to leave a legacy that is worthy of our children and of future generations.

We look forward to making the most of a legacy environment and providing something better for those that follow us.

Reference:
The above quote is from Christine Gregoire, Governor of Washington State, USA in her Inaugural Address, January 12, 2005

SOA Best Practices: A Book Review

"Enterprise SOA: Service-Oriented Architecture Best Practices" is by Dirk Krafzig, Dirk, Karl Banke and Dirk Slama. These are three Germans from the banking and insurance sector who present a wealth of practical experience to the reader. They attempt to talk about SOA without favoring any particular technology which is hard to do. I would still say that there experience with EJBs and CORBA provided a certain slant to the discussions however they were also using SOAP, WSDL and XML examples in their discussions.

The book had the feel of describing the real world rather than discussing and ideal world. By contrast the book by Erl was from a Web Services purist's point of view and glossed over some practical issues like what to do about user interfaces, performance problems and legacy transaction integrity. On the other hand Erl had the advantage of not having to describe the concepts in terms of differing technologies.

Legacy integration issues were covered well by Krafzig et al. There was an excellent chapter on process integrity and a good chapter on infrastructure issues like Scalability, Interoperability and Security. They covered the range of process integrity techniques from "Eyes Closed and Praying" to "Two Phase Commit over a heterogeneous environments" and "SAGAs with complex compensation graphs".

Krafzig et al provide more of "Horses for Courses" approach. Their approach does not necessarily advocate the same approach for fine grain and course grain services. Their approach even accepts that synchronous as well as asynchronous service buses might need to work side by side.

One theme was the importance of the registry and repository for SOA. The book gives some practical suggest about what to put in these stores. Another theme is the importance of governance of the development process. This theme is taken up in a later chapter which discusses SOA-driven project management, configuration management and testing. The book also discusses a thin-thread approach to software development. This is accompanied by discussions on how to fund SOA development and how to manage the risk of these initiatives.
The climax and conclusion of the book is four case studies. These are real-life examples with solutions that had to meet compromises and struggled with incorporating legacy environments with their solutions. It would have been nice to have an example that was not a finance company but it seems that is where the money is when it comes to SOA (and any big information systems for that matter).

The book did not provide a motivating definition of SOA although the thoughts on architecture I found enlightening. Such as "architecture … is designed to last longer than one or two of these technology innovation cycles" and that its purpose is to simplify development. How easy it is to forget that our new inventions in IT are just to make the world a simpler place.
I enjoyed this book. It was good to read a European context on SOA. They referenced Niklaus Wirth (a hero to anyone who did computer science when I did it) a couple of times and referred to other European IT Gurus which is refreshing break from the US IT heritage. I commend it to those who want a technology-agnostic look at SOA.

References
Erl, Thomas, Service-Oriented Architecture: Concepts, Technology, and Design, 2005 Prentice Hall

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004

Start Somewhere

This is my first Blog entry. I'm hoping after putting this entry in I can go back later and edit or delete it. I have no idea what this going to look like but I have to start somewhere.

Blogging is one of those things that as an IT person everyone thinks I know about. But how can you keep up with all the crazy things the IT industry is throwing up? If it is not Blogs, mashing and Web 2.0 then its a host of new open standards and no-one can tell which standards are actually mature or useful.

I'm not too proud to say I started as a Fortran and Cobol programmer. There are still people in gainful employment who write Cobol. Unfortunately some are now my staff. Yes I look after a group of programmers. Most of which work on a mainframe. We are trying to update. I've now got some Java staff and a number of Sun boxes. We have got a weblogic application platform and we are trying to move forward to adopt SOA. What a struggle that is. It's got to better than programming for green screens but why isn't the path better trod? Why isn't the road to the widely accepted new paradigm as obvious as the main highway?

This blog is expected to be a fairly long lasting chronicle of a hapless and anonymous government department in its quest of the holy grail of SOA. This is going to take a while and my observation is that the goalposts change on the journey and that everyone has to find their own path - a personal journey to geek enlightenment.