The Importance of Meta-Projects

The feature video on International Association of Software Architects (IASA) site last week provided. I wish I could reference the speaker and the name of the video but alas last weeks feature video has been replaced by this weeks feature video.

The talk was reasonably pragmatic discussion of Architecture by a staff member of delta airlines. What inspired me to write this posting was that a slide that depicted "Business Enablements" things, like agility and collaboration. It also depicted lists for Business strategy and IT strategy. This is nothing new but it reminded me I need to lift my eyes from the day to day IT projects and see what my organisation is doing for software development as a whole.

This is timely because its time for my branch to put together a plan for next year. Typically this has been very project-focussed. Of course I also hope to put together a good graphic although I doubt whether I can do as well as the delta-airlines chart.

Each business and each IT department has its projects to achieve some outcome for the business. My focus is on application development projects. Often these projects are part of a bigger business project and commonly involve business process changes and infrastructure changes. Another category of projects do not achieve direct outcomes for the business but allow the business or the IT department to undertake these projects in a more effective manner. For business projects these 'meta-projects' is what was referred to as "business enablements" by the IASA speaker. I will not be using the term further as my spell checker says "enablements" is not a real word, and I tend to agree.

To put these in context see the following table

	Business	IT
Goals/Objectives	Business Goals. Mission, Vision and Strategy	IT Strategy
Meta-Projects	Business meta-projects	IT meta-projects
Projects	Business projects	IT projects

There are strong relationships between each cell in the table and its neighbouring cells. The business goals drive the IT strategy and also the business meta-projects. Business meta-projects may drive business projects but also meta-projects may be driven by projects. Neither the meta-project or project is necessarily the driver. This is the same with IT meta-projects and IT projects. IT strategy should drive the IT meta-projects and IT projects.

The IT Meta-Projects that my IT department is engaged in are:

• SOA
• Enterprise Architecture
• Mobility (Supporting mobile computing devices)
• Business Intelligence (Data Warehouse and Reporting)
• Legacy Transition
• Spatial Systems (Geographic Information Systems)
• Online Project (Customer-facing IT systems)
• Identity Management (Authentication/Privileges/Access Control)
• Service Management (ITIL, Request Management, Change Management)
• Performance Management (Individual Performance)
• Cost Efficiency/Consolidation (ongoing)
• Business Alignment (ongoing)

Some of these meta-projects have explicit planning documents explaining the plan. Others are more just acknowledged as the agreed direction of the IT department. Cost efficiency and business alignment are not specific projects but an ongoing focus of attention as they would be in most IT departments.

Some of these IT meta-projects such as Online Project, Service Management, Performance Management and Costs Efficiency have analogous business meta-projects. Business Alignment parallels the business meta-project of customer alignment.

This list of IT meta-projects has not previously been written out. It is surprisingly long, especially given the much longer list of specific software development projects that are on our books. Ideally each of these meta-projects should be explicitly planned, prioritised and monitored.

Meta-projects provide a foundation to the work an IT department does. They could also be referred to as 'foundation projects' but they are not necessarily just internal projects as they may be programs of projects that directly affect the business user. The meta-projects help build the look and the feel of the IT department. They establish its competitive advantage and its identity. Accordingly, it is important to get these meta-projects right and to provide them with the attention they deserve over the more day to day concerns of getting software out the door.

Does SOA Rate a Mention in the Strategic Plan?

I was asked to draft a contribution to the IT Strategic Plan last week. My organization is a big public organization and the IT Strategic Plan is a glossy public document that has to be more of a marketing document than a technical plan. I wanted to put something in about SOA but I could not rely on reaching a technical readership or a readership that new much about the organization's IT at all.

This raised my curiosity. How many organizations actually put something about SOA in their IT strategic plan, let alone a corporate strategic plan?

Many commentators suggest SOA is a strategic imperative. For instance David Linthicum suggests "when thinking about SOA a long term strategic plan is best". Christopher Koch goes further and suggests that IT will assume responsibility for business innovation.

The ongoing shift to service-oriented architecture (SOA) will improve IT’s
ability to innovate because it requires that IT understand how the business does
its work. SOA represents the highest order of business process innovation

Surely SOA is of such central importance that it is going to be central to most IT Strategic Plans.

If I look to Forrester Research for some help with the IT Strategic plan I am advised that

IT's strategic plan is an essential tool to run IT like a business. The strategic plan for today's IT is different from the strategic plans that IT may have developed five years ago. It is purpose-driven and a complement to IT governance structures and processes.

Unfortunately SOA does not rate a menton in this article. Enterprise Architecture and Project Portfolio Management however bother get a Guernsey.

I spent some time with Google and found no strategic plans containing SOA and so I decided to look at some IT Strategic plans. Some of the findings are detailed at the end of this posting.

I found four IT strategic plans that put SOA as fairly a central goal or support for the plan. I found four where there was a mention of SOA but not a significant emphasis on it. After not finding as many as I had hoped I thought that perhaps when people write about SOA for the consumption of the layman they use other terms to make the document more accessible. So I looked for mention of 'agility' or 'reuse' in IT Strategic Plans. This I thought is a way for IT people to speak in code about SOA without mentioning those three letters. I found four of those too, although the connection with SOA was not all that obvious in these documents. I have listed nine IT Strategic plans with no mention or allusion to SOA.

This is not a thorough or unbiased study. The people publishing the plans on the web seem to be government and tertiary education groups. This happens to be my interest. The list is not a random sample and I could have missed some important references within the documents, however I have to conclude that SOA is not getting into as many IT Strategic plans as it should, given its strategic importance.

I am left to hypothesize on the reasons for this:

• SOA is not strategically that important
• SOA is only considered a software development issue and is not considered to relate to other IT issues infrastructure, support services or IT procurement
• SOA is too hard to explain to the non-IT person
• SOA is not as business-centric as some SOA authors would have us believe
• SOA is too new to warrant mention in an IT strategic plan
• SOA is too old and like RDBMS everyone accepts it and will move on without having to bring it up in a strategic plan
• The SOA message has not reached those who write IT strategic plans
• There is little marketing value in SOA

This is a reality check for those espousing the strategic importance of SOA but I would not go as far as those who suggest SOA is only for tactical purposes. Whatever the reasons, I will still push for it to be accepted at the strategic level.

In my contribution to my organization's IT Strategic Plan I started with a business definition such as one of the earlier definitions discuss in my earlier postings on the definition of SOA.

I then went through some for the motivations behind SOA and expanded on some of these in reference to the business of my organization. These motivations were Agility; Reuse; Assist Transition from Legacy; Federation; Extensibility; Manage incremental Change; Better alignment between IT and Business Processes; Intrinsic Interoperability; and Vendor Diversity.

This ended up a fairly long description, so I expect most of what I wrote will end up being cut by the editor of the strategic plan. Whether SOA gets edited out of the IT strategic plan is not important as long as the internal focus is on getting the pieces in place to make SOA happen and we can get some support from the business for this. Perhaps in this process I will encounter some of the reasons that SOA is not so prevelant in strategic plans.


IT Strategic Plans with SOA as central or a core goal

Oxford University, UK:
http://www.ict.ox.ac.uk/strategy/plan/plan.xml.ID=S4

The Global Justice Information Sharing Initiative (Global) Advisory Committee
(GAC) (an advisory body to the Assistant Attorney General, Office of
Justice Programs (OJP), and the U.S. Attorney General)
http://www.it.ojp.gov/documents/200409_GAC_Strategic_Plan.pdf

The Commonwealth of Massachusetts
http://www.mass.gov/Aitd/docs/itdstrategicplan.doc

SURF (A Nederlands higher education foundation) http://www.surffoundation.nl/eng/download/SURF-Strategic-Plan_2007-2010.pdf

IT Strategic Plans where SOA rates a minor mention

Carnegie-Mellon University
http://www.cmu.edu/computing/about/2006StrategicPlan.pdf

State of Texas Department of Information Resources
http://www.dir.state.tx.us/pubs/asp2006/asp2006_abbreviations.htm

The University of Western Australia, Library
http://www.library.uwa.edu.au/__data/assets/pdf_file/11068/Scanning_the_environment_2007_NE.pdf

National Labor Relations Board, USA
http://www.nlrb.gov/nlrb/shared_files/reports/FY2007_StrategicPlan_FINAL.pdf

IT Strategic Plans with Some mention of Agility and Reuse but not SOA

State of Wisconsin:
http://enterprise.state.wi.us/home/StratPlan/enterpriseITplan.pdf

Commonwealth of Virginia
http://www.vita.virginia.gov/uploadedFiles/Library/StakeholderWorkshopResults.pdf

University of Iowa:
http://cio.uiowa.edu/strategicplan/

Department of Family and Community Services, Australia
http://www.facsia.gov.au/internet/facsinternet.nsf/via/ict/$file/FaCS_ICT_Strategic_Plan_2004-07.pdf

IT Strategic Plans where SOA rates no mention
(There actually were many but these are examples)

City of Winston-Salem:
http://www.cityofws.org/Assets/CityOfWS/Documents/forms%20and%20reports/Information%20Systems/IT%20Strategic%20Plan.pdf

State of Maine:
http://www.maine.gov/oit/strategic/SITP.doc

US State Department:
http://www.state.gov/m/irm/rls/c13461.htm

Louisiana State University:
http://www.lsu.edu/departments/its/ocio/FITS/fits.html

State of South Carolina:
http://www.cio.sc.gov/ITPlanning/StateStrategicPlan.pdf

New York State:
http://www.cio.state.ny.us/NYS%20IT%20Strategic%20Plan/NYSITStrategicPlan_20Jun2006.pdf

OHIO State University
http://cio.osu.edu/planit/refresh07.html

Dept of Justice, State of Victoria, Australia
http://www.justice.vic.gov.au/wps/wcm/connect/DOJ+Internet/Home/About+Us/Our+Goals/JUSTICE+-+Department+of+Justice+-+Information+Technology+Strategic+Plan+2002-05

Electoral Commission of Qld, Australia
http://www.ecq.qld.gov.au/data/portal/00000005/content/60401001035252002415.pdf

Concluding Conference Comments

This is the final of a series of postings on an SOA conference presentation I gave for the Ark Group in Sydney. I seemed to have a great deal of material which probably explains why I ran short of time. The topics I covered with the relevant postings linked were:

• The definition of SOA
• The motivation behind SOA
• Legacy Transition
• My organisation's software stack
• The link between SOA and Enterprise Architecture
• The metadata required for SOA
• SOA and Metadata: Where you can Put It

Of course, there were other speakers as well, and they were all highly competent and informative. I summarized these in an earlier posting.

To wrap this up series up I will go through my conclusions. Firstly, the changes we need to make to support SOA in an organization involve:
• New Components, New Tools, New Skills
• New Application/Service Architecture
• Governance around reuse and business process
• Less emphasis on applications, more governance at the service level
• Focus on interfaces and contracts particularly in the information model
• More Metadata
• Changes to methodology

New Components, New Tools, New Skills
When we talk about changes it is important to understand the baseline from which you are changing. Some organisations will have well developed skills in distributed portal-based applications. My organisation is yet to deploy its portal product and is still developing skills outside of the mainframe environment. We are still building our skills in Unix and Java. We are still putting together our development environment. None of these components, tools or skills are peculiar to SOA. What we have done specifically to implement SOA is to purchase the BEA Weblogic platform and we are looking at ways of changing our development approach. We find we need to quickly ramp up skills in our new environment as well as SOA Development, SOA Testing and management of metadata.

New Application/Service Architecture
There was a critical cell in the Zachman Framework called “Application Architecture”. This could be relabelled “Service Architecture”. It is services that now should be the focus of our attention. This is working at a different granularity.

Governance around reuse and business process
Governance is required around reuse of services and reuse of artefacts. There is more work in the Enterprise Architecture (EA) space than there is for traditional application-centric development. This should be balanced by relatively less work in the solution architecture space. Modelling Business Processes is more important with SOA.

Less emphasis on applications, more governance at the service level
Application Development is more focused on assembly of services and the development of a presentation layer so there is less emphasis on applications themselves. There will need to be governance changes to introduce SOA. There needs to be stronger EA governance. Now you not only have to find an owner and sponsor of an application. You may have to find and owner or sponsor of a service. This may be easier than finding owner for monolithic application in some cases (and in other cases it may not).

Focus on interfaces and contracts particularly in the information model
The modelling of persistent data store won’t change but in addition we need to model the interfaces and contracts between services. This will be required to drive service development.

More Metadata
All this means we have more enterprise metadata. This presents challenges for managing this metadata. This was the subject of an earlier posting.

Changes to methodology
Software development methodology includes more work on interface, contracts and SLAs. The development of testing services presents a challenge. This is because you may have to test without having and application to test it with. You have to test on behalf of applications that may use it but do not exist yet.


I make the following architecture recommendations to facilitate SOA:

• Have an Enterprise Architecture
• Manage the application portfolio
• Drive the enterprise from business processes or from events
• Apply good governance to services and metadata
• Data modeling should include interface/contract design

SOA must be driven from and EA. It is not something that can be put in place for a single application. The benefits of SOA will not accumulate until you have a number of applications completed.

When you build a service you build it for all the applications that potentially use it. Therefore you have to know about these applications. My organisation has a centralised IT department. This helps enable both EA and SOA. A federated model can work but someone still needs a clear picture of the application portfolio and needs to define interfaces and contracts of your services.

Business Processes will drive SOA. This is to ensure the services have meaning to the business and that they can be combined into processes in the same way that business combines its tasks into processes.

The Governance changes in SOA. I’ve talked about the governance of services and the governance of metadata. Both are important for successful SOA.

Once the business process is sorted out the interface and contract between the services must be modelled. This will then drive the development of the services. This information needs to be kept up-to-date and is the key metadata required for reusing the service.


Some of the points brought up at the conference interactive session were true of most software development:

• You need good business sponsorship
• Choose your first SOA projects carefully
• If the business does not see the need then it's not going to work
• Talk to the business in language it understands
• Get executive support of your SOA program

So my concluding recommendation is not to lose sight of what has worked in the past. A lot of what makes conventional development work, will also help make SOA work.