ITIL Services

ITIL Services are different from Services Oriented Architecture (SOA) Services. The Information Technology Infrastructure Library (ITIL) is an accepted standard for operational IT issues such as incident management, problem management and change management. ITIL builds up a large set of terms and tries to use these terms consistently throughout its discussion of Service Management.

The conflict over the term "service" can lead to some confusing conversations between developers and operations staff. I have posted earlier on the definition of services in SOA. The ITIL definition is

"A service is a means of delivering value to customers by facilitating outcomes
customers want to achieve without the ownership of specific cost and risks. "
ITSMWatch

Two very different examples of a service in ITIL could be a payroll system and the resetting of a LAN password. The services of an IT organization should be available on a "service catalogue" and the clients of the IT organization can use this catalogue to choose and understand the services they receive.

This is different from an SOA service. An SOA service is a software component which:
• Hides its implementation,
• Is based on standards,
• Is location transparent,
• Is message coupled, and
• Is accessible through defined platform independent interfaces.

The main difference is that an SOA service is a software component in SOA. It is not the service that a help desk provides when they reset your password and it is generally smaller than a whole payroll system. An SOA service can be hidden away from the users in the cloud of application components. Ideally it is granular enough to have meaning to users but a user does not need to be able to interface with a service directly.

There can be coincidental overlap between an SOA service and an ITIL service. A service for paying a bill may also be presented directly to the users of an IT Department or organization and therefore be a service on their catalogue. This only seems to confound the issue. It is safer to assume that ITIL stalwarts and SOA engineers are talking about different things when they refer to a service.

IT Service Management (ITSM) under ITIL however can provide an appropriate means for providing governance for your SOA and for providing the operational platform for your SOA. SOA may contain more component parts than traditional systems and a disciplined approach to configuring, operating and changing these parts is required which is exactly what ITIL offers.

Interestingly ITIL also seems to redefine SOA as "Service Offerings and Agreements" and "Service Outage Analysis" which also makes life complicated.

Notes:
Some good articles on ITIL and SOA follow:
http://www.ca.com/Files/WhitePapers/soa_wp_jlong_33146.pdf
http://www.ibm.com/developerworks/ibm/library/ar-soaitil/
http://www.ebizq.net/blogs/bethgb/2009/03/can_itil_help_fix_soa.php
http://www.biske.com/blog/?p=467

The book I am reading which proffered the payroll as a service example is:

Klosterboer, Larry, 'Implementing ITIL Change and Release Management', IBM Press, 01 Dec 2008.

SOA Adoption for Dummies Such as Managers

Software AG has commissioned another SOA publication to "Dummies" series and released it to their customers. This follows on from their "BPM Basics for Dummies" book which was a referred to in an earlier post. I have to confess to listening to the podcast version of this book while watching the cricket rather than reading the book in detail the conventional way.

This book is focused on SOA Adoption rather than SOA. It is not a technical book in SOA. References to XML, SOAP and WSDL are consigned to a footnote. This book is evangelical and pitched at management. I question whether it is right to consider that dummies would be involved in SOA adoption. Your average person involved in an SOA Adoption decision is a highly credentialed senior IT manager on one hand but on the other hand the time taken over each decision of a manager can be minimal and display all the appearances of being made by a dummy.

The authors should be congratulated for taking SOA Adoption as subject in itself as it is a difficult for enterprises to adopt SOA and it is much more than just the technical challenge that needs to be addressed. The book and podcast series are well produced and easy to absorb.

Do not expect any great depth from this book however. The authors call their SOA adoption approach "Rocket Science" but this approach simply consists of:

1. Keep the pointy end of the rocket up.

2. Keep moving up.

3. Don’t stop till you are weightless.

This is good motherhood advice for any significant undertaking but I do not see it as being particularly insightful for SOA adoption.

The authors do a good job of SOA evangelism and have lived up to the creation of simple book for "Dummies" by missing a lot of the technical detail. There is nothing particularly radical in this book but it does give you some important things to watch such as SOA Governance, Organization structure, SOA infrastructure, the SOA competency centre and organization structural issues that might arise from SOA.

In summary, this is a good book to give a dummy or manager who needs to be brought on board to get with your SOA program. It will not get you to SOA weightlessness without a lot of other expertise, skills and resources. It is not the bible for someone wishing to be the main driver behind SOA adoption.

Reference

Miko Matsumura, Bjoern Brauel and Jignesh Shah, SOA Adoption for Dummies , Wiley Publishing Inc., 2009
http://www.softwareag.com/Corporate/res/books/soa_adoption_for_dummies/default.asp

Does normalization apply to Services?

I was listening to a podcast by Thomas Erl this week where he was talking about a pattern for normalization of services. This was interesting because there is not much in SOA with the same theoretical rigour as database normalization inspired by the work of Edgar Codd in the 1970s and the relational algebra and the Structured Query Language (SQL) that followed it.

The normalization that Erl talks about is fairly limited. On his website he summarizes normalization as

Problem
When delivering services as part of a service inventory, there is a constant risk that services will be created with overlapping functional boundaries, making it difficult to enable wide-spread reuse.
Solution
The service inventory needs to be designed with an emphasis on service boundary alignment.

There does not seem to be a lot of science involved in decomposing services to their basic elements so they can have maximum reuse and be easily composed into higher level services. Perhaps more precisely it is a 'soft' science that is all about governance, process and aligning with user requirements.

Normalization of services is not something that has been picked up strongly by other authors however David Chappell has his description of the normalization project which is consistent with Thomas Erl:

An idealized SOA environment provides a cleanly delineated set of services,
with one way to accomplish each function. This kind of normalization makes reuse
self-enforcing, since each service provides access to a specific group of
information in a specific way.

In his podcast Erl gives an example. If we have a service to get an invoice, then it is redundant to have a service or a method to get an invoice header. To have both is not normalized however there are reasons you may require both and therefore this would be 'denormalized'.

I am a little critical of the use of the term 'normalisation' to describe removing service redundancies. The processs of normalization in data modeling (and mathematics such as linear algebra) is a very precise and well developed technique where the concept used by Erl and Chappell while useful, is fairly loose.

If we wanted to do the equivalent of first normal form in service interfaces we might consider getting rid of the service equivalent of the "repeating groups" that destroys the relational model. If there are many 'invoice items' for an invoice, this means you could not accommodate 'invoice item' in your 'invoice' message. The strength of SOA relies on being able to exchange documents that mean something to the user. This is where attempts to get scientific about normalization fail the cause of SOA.

I will take this normalization discussion to its absurd ultimate. When I did my final year of my honours degree I simulated hardware that consisted entire of only two elements. They were the 'Nand gate' (logical not(and (a,b)) ) and a single binary digit of memory (Flip flop). The computer program that ran on this simulated hardware was simply a wiring diagram. Any computer program that can be represented in normal programming languages can be represented as a combination of these two elements. The sort of programs I ran on this simulation were the full adder, 4 bit timer and 3 bit shift register. Programming at this level is more a level in elementary digital electronics than it is in software development.

This was a distributed network of "services" at the very elementary level, but it is not very useful as a software development approach. This is a completely elementary normalized distributed application that can not be reduced further. This could be described as the ultimate implementation of reuse. You only have two 'services' (Nand and memory) both of which are reused constantly. The challenge is to link them together in meaningful ways. These services however are tightly coupled because of their synchronous nature and therefore this is not an SOA.

As a programming language this is worse than assembler language. Trying to wire gates together is even more detailed than assembler coding. Even if we could get SOA performing brilliantly at lower levels of granularity we do not want to go to such a low level of granularity that business entities are unrecognizable.

Beware dogmatic approaches to normalizing or simplifying a service inventory. I suspect the MEST architectural style with its focus on removing verbs or methods is heading down this type of reductionist thinking.

An overriding concern should be to meet business requirements and this might mean implementing redundant services and poorly structured business documents in services. Designing with a focus to eliminate redundancy is a good idea. It is nevertheless useful to seek out redundancies and investigate the option of reducing these if it does not compromise the ability of the services to represent business processes. Too much science in this process can reduce your services to absurd solutions.

Reference:
Kimber, Antony, The General Logic Array: An Abstract Architecture, Honours Report, The University of Adelaide, Department of Computer Science, November 1985.

This posting may have seemed a flimsy excuse to expose this work to the public domain but I hope it has described a useful anti-pattern for SOA.

The above project was done around the time Sun were having success introducing their Reduced Instruction Set Computer (RISC) chips which included simpler instruction sets that performed more quickly. Reducing down to a couple of elementary components is the extreme end of this strategy. There was also early interest in molecular electronics with the hope that if we could get molecules to behave as electronic components we could produce very small electrical circuits. One of the challenges of the project was working out lattice arrangements that would allow the appropriate connections to the two or more types of elementary components to be made.

SOA Governance

I listened to a good Podcast on SOA governance today. It was Phil Windley and Scott Lemon interviewing Tod Biske and Ed Vasquez of Momentum SI. It is clear to me that SOA governance is important in managing SOA and SOA metadata. This podcast provides some ideas on what shape this governance can take and how products may or may not assist with this.

SOA has often been compared with Object Oriented development or even structured programming. The question was asked in the Podcast about why Governance was such an issue for SOA in comparison to these earlier approches. The answer is that the implication of an SOA decision goes beyond the IT group and single applications. The users of the service are significant stakeholders in an SOA and are directly affected by the policies. Organisation partners or organisation divisions sharing services need to agree on a service approach and comply with service interfaces in order to provide effective reuse.

Todd Biske emphasised three key aspects of governance in relation to policies that need to in place. These three aspects can be assisted by automation. They were

• Enforcement of policies
• Creation of policies
• Storage of policies

Later in the discussion two more aspects of governance were provided:

• Communication of polices
• Feedback on policies (policies evolve)

A strong theme of the podcast was that you cannot buy governance. It has to be appropriate for the organisation and a product is not yet out there that provides integrated assistance over all the aspects of governance. The process of building governance occurs by assessing the current situation, working out what processes are involved and then providing the appropriate level of manual operations and automation to support these processes.

Storage of policies is provided by registry and repository products. Registry products are moving toward providing more repository functions. The term "Regository" was coined in this podcast. A parallel can be drawn with LDAP. The registry is becoming much more of a managed data set which is configurable and provides information to humans not just automated processes.

Feedback on policies and services is very important. Without proper feedback the organisation can become a lame duck. Portfolio management is important for services as well as for applications and projects. The discussion reminded me of getting feedback on this statistics for my blog site (using http://www.mybloglog.com/). This tells me where my links are coming from and also gives me a warm feeling that people actually see this stuff. The same thing is true of getting feedback on a website with a product like Webtrends. Services are no different. Metrics about who is using a service, when they are using it and how often, provide valuable feedback.

This feedback and the process of understanding the customers of your services gets the IT person closer to the business. It was suggested in the podcast that in general SOA involves IT doing more tasks previously understood to be business tasks. If you are not experiencing the pressure to do more work related to the business then you are not really doing SOA.

There can be hundreds of policies associated with SOA. Design time policies will include technology choices, product choices and methodologies. Run-time policies include your service contracts and service level agreements (SLAs). These SLAs may apply to each producer-consumer pair and include security policies, operational policies, compliance policies and integration policies.

SOA policy should not be a police state. The approach to implementing policy should be that "The path of following policy is the path of least resistance". An organisation should not reach a gridlock because there are so many policies that the developer cannot move. Policies are something that involves discussion with service users and should not be confined to the IT group.

Effective policy agreements will save time by avoiding the need to build using more than one technology (eg. SOAP, REST and JMS). The value of an agreement is that it saves the cost of building with multiple standards.

Greater governance enables the services in SOA to be more loosely coupled. Loose coupling is about reducing dependencies between services and making those dependencies explicit. The fact that the service contracts are written outside service code and are clearly defined is a key determinate of loose coupling.

Todd's two most important points were

1. Use a service or application portfolio technique to identify what services are good, bad, expensive or good value.
2. To get started on a project ask
   - What services dose this solution expose or use?
   - What business events does this solution publish or subscribe to?
   - What business processes do this solution support?

There were some good insights in this podcast. Governance in SOA is important. It may stop your SOA becoming JBOWS (Just a Bunch of Old Web Services). It makes sure that your organisation gets the best of reuse and does not get lost in mire of standards, products and methodologies.

Footnote:
JBOWS (or JaBOWS) was a term I heard this week in a talk from Glenn Smyth. There is a good post from Joe McKendrick on this acronym. Another reference can by found in an article by Rich Seeley which in turn refers to another artlicle by Brent Carlson which uses the acronym ABOS (A Bunch of Services). It would have been more accurate to write about ABOS in my posting but JBOWS has a nicer ring to it and seems more likely to catch on.

Concluding Conference Comments

This is the final of a series of postings on an SOA conference presentation I gave for the Ark Group in Sydney. I seemed to have a great deal of material which probably explains why I ran short of time. The topics I covered with the relevant postings linked were:

• The definition of SOA
• The motivation behind SOA
• Legacy Transition
• My organisation's software stack
• The link between SOA and Enterprise Architecture
• The metadata required for SOA
• SOA and Metadata: Where you can Put It

Of course, there were other speakers as well, and they were all highly competent and informative. I summarized these in an earlier posting.

To wrap this up series up I will go through my conclusions. Firstly, the changes we need to make to support SOA in an organization involve:
• New Components, New Tools, New Skills
• New Application/Service Architecture
• Governance around reuse and business process
• Less emphasis on applications, more governance at the service level
• Focus on interfaces and contracts particularly in the information model
• More Metadata
• Changes to methodology

New Components, New Tools, New Skills
When we talk about changes it is important to understand the baseline from which you are changing. Some organisations will have well developed skills in distributed portal-based applications. My organisation is yet to deploy its portal product and is still developing skills outside of the mainframe environment. We are still building our skills in Unix and Java. We are still putting together our development environment. None of these components, tools or skills are peculiar to SOA. What we have done specifically to implement SOA is to purchase the BEA Weblogic platform and we are looking at ways of changing our development approach. We find we need to quickly ramp up skills in our new environment as well as SOA Development, SOA Testing and management of metadata.

New Application/Service Architecture
There was a critical cell in the Zachman Framework called “Application Architecture”. This could be relabelled “Service Architecture”. It is services that now should be the focus of our attention. This is working at a different granularity.

Governance around reuse and business process
Governance is required around reuse of services and reuse of artefacts. There is more work in the Enterprise Architecture (EA) space than there is for traditional application-centric development. This should be balanced by relatively less work in the solution architecture space. Modelling Business Processes is more important with SOA.

Less emphasis on applications, more governance at the service level
Application Development is more focused on assembly of services and the development of a presentation layer so there is less emphasis on applications themselves. There will need to be governance changes to introduce SOA. There needs to be stronger EA governance. Now you not only have to find an owner and sponsor of an application. You may have to find and owner or sponsor of a service. This may be easier than finding owner for monolithic application in some cases (and in other cases it may not).

Focus on interfaces and contracts particularly in the information model
The modelling of persistent data store won’t change but in addition we need to model the interfaces and contracts between services. This will be required to drive service development.

More Metadata
All this means we have more enterprise metadata. This presents challenges for managing this metadata. This was the subject of an earlier posting.

Changes to methodology
Software development methodology includes more work on interface, contracts and SLAs. The development of testing services presents a challenge. This is because you may have to test without having and application to test it with. You have to test on behalf of applications that may use it but do not exist yet.


I make the following architecture recommendations to facilitate SOA:

• Have an Enterprise Architecture
• Manage the application portfolio
• Drive the enterprise from business processes or from events
• Apply good governance to services and metadata
• Data modeling should include interface/contract design

SOA must be driven from and EA. It is not something that can be put in place for a single application. The benefits of SOA will not accumulate until you have a number of applications completed.

When you build a service you build it for all the applications that potentially use it. Therefore you have to know about these applications. My organisation has a centralised IT department. This helps enable both EA and SOA. A federated model can work but someone still needs a clear picture of the application portfolio and needs to define interfaces and contracts of your services.

Business Processes will drive SOA. This is to ensure the services have meaning to the business and that they can be combined into processes in the same way that business combines its tasks into processes.

The Governance changes in SOA. I’ve talked about the governance of services and the governance of metadata. Both are important for successful SOA.

Once the business process is sorted out the interface and contract between the services must be modelled. This will then drive the development of the services. This information needs to be kept up-to-date and is the key metadata required for reusing the service.


Some of the points brought up at the conference interactive session were true of most software development:

• You need good business sponsorship
• Choose your first SOA projects carefully
• If the business does not see the need then it's not going to work
• Talk to the business in language it understands
• Get executive support of your SOA program

So my concluding recommendation is not to lose sight of what has worked in the past. A lot of what makes conventional development work, will also help make SOA work.

Is Your Enterprise Architecture SOA Compliant?

In August I wrote about the Zachman framework and SOA. This was the basis for a couple more slides at the Ark Group conference on interoperability in which I was a presenter. I have covered material from the presentation in couple for previous postings, "Live up to the Legacy that was Left" and "Why SOA?". I will focus on the main theme of my presentation in this posting which was "Ensuring your Enterprise Architecture is SOA Compliant"

Firstly I would like to point out some work that is being done by the SOA Consortium along these lines. They have announced the Enterprise Architecture 2010 Working Group which is to define the high-level roles and deliverables of the Enterprise Architecture team. There is not much tangible yet from this group as it has just been set up but these people also brought us the SOA practitioners' guides so this could prove a valuable initiative.

At the conference mention in the first paragraph I went over the Zachman Framework as I have done in my earlier posting. One thing I pointed out is that different software development approaches can be described in terms of the framework. I then went on a described my organisations use of the Zachman framework using he colour codes I added to the Zachman grid.

When we talk about top-down approaches conveniently you work from the top row (Scope Row) down to the bottom row (Detailed Representation). Similarly when you work bottom-up you work from the bottom of the framework representation.

Assuming that most good software development is done top-down (and I am subscriber to this approach) you can choose different methodologies depending on the column you choose.

If you drive down the 'Data' column you have a Data Driven Architecture. The Enterprise Information Model in this approach is all important. The other cells of the framework will either support the data column or be influenced by it.

The SOA approach is usually focused on business process and this means it is being driven by the 'Function' column. Software development can be driven from Business Process Modelling and 'Function' column without being SOA.

If we are two focused driving our software development from the 'Network' column we may end up with a siloed application based on organisational locations. This generally software development anti-pattern as modern enterprise software should be independent of location.

Similarly, too much focus on the 'People' column can lead to application focussing around individual or organisational units. This is a very common siloed approach and anti-pattern. More positively a people driven architecture will look at the user experience. This is a common approach for smaller applications to look at screen designs, report layouts and user interaction scenarios that will meet user requirements. For enterprise application though it is important to look at the bigger picture, to question what the organisation is trying to achieve and to model the process from start to finish otherwise the people siloed anti-pattern may result.

If we drive down the 'Time' column we will end up with an Event Driven Architecture (EDA). This is an alternate approach to SOA. We look at events and how they initiate and affect business processes. The 'Function' columns and 'Time' columns can be worked with in tandem to produce and event driven SOA application.

If we work down the 'Motivation' column of the Zachman frame work then we are using a Business Rules Approach. My organisation has not made great use of this framework column. For more information about the Business Rules Approach see http://www.businessrulesgroup.org/ . Also this column is valuable for expert systems and decision management.

With the Zachman framework you do not have to fill every cell. You certainly do not only have one enterprise model in each cell. How you uses the Zachman framework is important. So I will describe how our organisation uses it in terms of the diagram above.

The yellow cells in the above diagram are areas where we have not got any significant documentation.

The green cells contain documentation the business has prepared with out prodding from the IT department. The brown cells are documented by IT but with the interaction of the business.

The top two layers do not change much. These are business level documents. Producing good business process models is more important with SOA. The business process is the key in the SOA process not the application.

The blue is driven by SOA. It starts with your Application Architecture. Distributed Architecture works differently with SOA. You might consider and ESB for instance which will give a different architecture than if you use point-to-point communication. These upper two blue cells drive changes at the more detail level below them.

The pink areas are affected by SOA. Data modelling and Human interface architecture are not radically different but they have to take into account the new architecture.

I will now look at the link between EA and SOA in a different way.

Zapthink has a good article on Zachman and SOA where they use arrows emanating from the Application Architecture cell of the framework. I prefer to represent the influence direction as predominately down representing a top-down approach.

We will start in the 'Data' column again. I do not think the Information Model should be different as a result of SOA. However the Logical Data Model probably should. It should document the interfaces and be able to transform the representation in XML. It should also document the persistent data stores. You now have at least two levels of logical data modelling.

The physical data model and data definition can then be XML, XSD and WSDL. At the same time you have your modelling of persistent data which would be your normal Class Model or ER diagram. This data model would eventually be represented in data dictionaries and database schemas.

We will move across to the 'Function' column now. The Business Process Model should influence your application architecture. Application Architecture should advise you of what services you need and what services to reuse. A service becomes part of the enterprise architecture because it is built for reuse. A new service is also part of a solution architecture. The service has to meet both solution and enterprise needs. Also in this cell is your application stack which supports the service model. The service model then drives the more detailed System Design and the programming.

SOA is a distributed architecture so the lower rows of the 'Network' column takes on increased significance compared with traditional software development approaches. There is more flexibility about where you do your processing. This needs to be built into your Distributed System Architecture.

Business logistics (and particularly whether you are dealing with agencies external to the organisation) will impact your applications architecture. There is a stronger linkage from Application Architecture to Distributed System Architecture because SOA is an enabling technology. It enables you to distribute processing, to look at load balancing and to look at interfaces with other agencies. There is therefore a strong influence on distributed architecture increasing the importance of good architecture in this space.

As with the other columns the influence downward is strong. Your Technology Architecture and Network Architecture needs to be built to cope with the new Distributed Systems Architecture.

Human interface architecture in the 'People' column may change to suit SOA but it does not have to. Some interface technologies work better with SOA. Some ways of interacting with an application work better with SOA but if you want to drive it the other way by specifying your Human Interface Architecture first then this can also work.
Security Architecture gets its own arrow. It is not a logical progression from Presentation Architecture. You may have to do security differently under SOA. You may have to make better use of a standard infrastructure. You may have to use SOA security standards.

SOA for my organisation is business process driven although there is still some bottom up development resulting from services based on mainframe data entities. The focus on reuse and the question for what we already have available to reuse demands some bottom-up consideration.

To be SOA compliant your architecture needs to include:

• Documented business processes,
• Enough bottom-up analysis to reuse of existing services, and
• Data modeling of web service interfaces.

This is in addition to SOA infrastructure and distributed models required the lower levels of your architecture. SOA is an Enterprise Architecture approach. You cannot have an effective SOA unless you have some level of Enterprise Architecture. If you focus on a single solution or application and do not look at the wider needs of the enterprise then you are going to miss the major benefits of SOA.

The business process focus and implications around reuse mean that the business needs to be aware of the SOA approach. It is a change to business practice that the SOA consortium is looking at in its Enterprise Architecture 2010 Working Group.

SOA and So Much Metadata

Everyone probably knows that metadata is "data about data". This concept has been around throughout the short history of information technology and before. The library catalogue is the archetypal embodiment of metadata. Data structures and then databases need description and these were more technical type declarations, schemas and Entity-Relationship diagrams. XML deserves special mention in this one paragraph of metadata history as it has revolutionised the way metadata is done. Now those library catalogues, those data structures alike can be described in XML.

Now metadata and XML has a particularly important role in SOA. Metadata is the glue that holds services together in SOA. WSDL and SOAP which are both based on XML are used at runtime to communicate with services. This is not the only importance of metadata in SOA though. Metadata has an increasing role during design time if you are going to get the most reusability from your SOA.

The amount of metadata require at design time is huge. To appreciate this it is instructive to have a look at the Zachman architecture framework grid. This 30 cell grid is full of architectures that use metadata models. Some might argue that some of the material from the top rows of the Zachman grid (scope and business model) is pretty far removed from working systems and should not be bundled in with metadata more directly relevant to implemented systems. Program code might not be considered by some as metadata but nevertheless it is an abstraction of data submitted to a CPU for execution. The point is that there is an awful lot of design time metadata in enterprise architecture. It might be XML, or UML, or other diagrams or lists but it should all be recorded and it all is interrelated.

The challenge has always been to store this metadata in a way that allows it to be easily retrieved, in a way that makes the connections to other metadata explicit and a way that can be discovered for new projects. Reusability has not been something that was discovered with SOA. Business process and data models have always been relevant over multiple projects and it has always been important to be able to find metadata when the need arises or when maintenance work needs to be done on old applications.

Finding a repository for all this metadata is not easy. Where these artefacts are stored with the tools that create them it makes it difficult to combine into cohesive store of information. Taking my organisation as an example, we use Telelogic System Architect for the higher level enterprise architecture document and Rational Software Architect for some of models used in our applications. We also have and issue register, a call logging system and diagrams done in Microsoft Visio. For data warehouse we use Oracle Designer and Oracle Data Warehouse builder. We store system information in CVS (software and document versioning), our file server, our intranet and our Wiki. No wonder we sometimes have trouble finding something.

How can I be sure that a developer has looked up all the previous work that has been done previously before the developer proceeds to develop a new service or interface? We could buy a metadata repository to hold all this information, but without good governance of the process of creating metadata this would be just one more data store in which to hide our artefacts. If it is clear what goes where then multiple repositories is not such a problem although it is obviously easier, the fewer different repositories you have.

In a sequel to this posting I will write more about registries and repositories, which are technical aids to solving the metadata flood. In this posting I hope I have established a need for dealing with metadata effectively.

Getting Independent Advice

Last Tuesday we got a couple of consultants from interstate to advise us on how my organization is sailing with its initial steps toward SOA. Their brief was fairly broad. They spent a couple of days meeting with various groups including management, legacy developers, java developers and architects.

The idea was to get some idea of what we are doing wrong, what we doing right and what we should do in the future. This is a good thing to do if you can find respected practitioners who can provide this advice. It is easy to be insular and head off in your own direction without considering other options. This can apply to the organization, to groups of organisations and it can even apply to whole regions. I always object when the Federal Government or the eastern states regard South Australia as a regional outpost, but when it comes to best practice in IT we have to acknowledge that our expertise in major enterprise application development can be thin.

I was involved in a few of the meetings. I expect them to focus on our run-time stack, our development standards and development methodologies. The major focus was on our legacy software. This is a big problem and one which I was happy for them to take head on. We have many large systems written on the mainframe in technologies that are out of date and do not easily lend themselves to being accessed through services. The few tools that are available to assist are prohibitively expensive.

The report from the consultants has not come in yet but I was able to glean areas that the consultants seemed to be targeting. One was how to address issues such as Data Integrity, Process Integrity, processing exceptions and auditing in relation to making providing services using our legacy applications. These are the concepts discussed in chapter 8 of Krafzig et al "Managing Process Integrity". They are common application integration problems which pre-date SOA and they are the sort of problems that do not yield to simply adopting particular middleware or standards.

The other issue that the consultants were targeting was how much we were investing in our SOA. We are trying to fund SOA by paying for infrastructure and the development of skills out of project funding for the development of particular applications and enhancements. This may not be sufficient to get the critical momentum we need to get to a desired maturity with our SOA. It would be a good outcome if our independent advisers can report to our executive that we need more money specifically for SOA. It is a bit hard to do this from the perspective of the IT Department. The executive would suspect we were investing the money for no good business reason, perhaps just for buying new toys for IT. The opinion of the consultants may help to put our case.

Of course the consultants might be hoping we spend more money on them. We have to be on lookout for such bias in consultant findings. In general though it is always valuable to get a second opinion and I eagerly await the report from this exercise.

Reference

Krafzig, Dirk, Banke, Karl and Slama, Dirk, Enterprise SOA: Service-Oriented Architecture Best Practices, Prentice Hall, 2004