Evolution: The Fittest and the Sexiest

The topic of this blog is "SOA Evolution" so it seems overdue for me to look at the Evolution process. I coined this title to chronicle my organisation's slow but deliberate adoption of SOA. 'Evolution' seemed to be an appropriate term because it was both a slow process but one that would improve our applications. In this posting I will look at evolution more literally and find more appropriateness in the term than I first imagined.

I am reading a book by Richard Dawkins (see references below) on a range of topics but primary about evolution in biology. There are two principal evolutionary processes. One is the survival of the fittest and the other is sexual selection which is the process of influencing the evolution process by the choice of a mate. The success of evolution is its ability to adapt to its environment and to succeed against other species in that environment.

Natural adaptations are gradual and based on the parent life form. The history of the life forms can be discerned in the DNA of the life form. We can determine that humans evolved from an ancestor we share with Chimpanzees from looking at this DNA.

The process of survival of the fittest can be applied to software development technologies. We see various versions of programming languages replaced by later more effective versions. We see whole languages disappear into obscurity (eg. Algol, APL, SmallTalk, ADA) or replaced by new languages. Relational databases took over from set-based or network databases. Software development approaches have evolved from structured programming, to object-oriented programming to SOA. It has been mentioned that SOA was forged from marriage of distributed computing and document-based integration.

Extinction of old approaches is often not as complete as we would like. The Cobol programs and the old database technologies are still around. The old dinosaurs are hard to kill off completely.

As with any evolutionary process the origins of software development are evident when we dig deep. Visual Basic .NET is still saddled with some of the terminology and constructs that were around in DOS Basic. The kernel of some operating systems still contains some anachronisms which would not be used if it were not for historical reasons. Many software systems have parts that were written long ago or copied from other systems that no-one can maintain or just do not make sense in the context of software written today. Some software maintenance tasks are like archaeological expeditions. Unfortunately it is not passive fossils that the software archaeologist finds amongst the code. Some of these historical curiosities actually support critical business functions.

There are some principles from long ago that we do not want to discard though. Just like some of the primitive instincts and emotions buried in the core of our brain there are principles from long ago that should remain in our software. Procedures and functions still have a role in our software as do objects. The development of services does not replace the need for these other means of encapsulation. The principles for strong typing and structured programming still have as much value today as they did thirty years ago.

The mechanism for natural selection process in software technologies is not that different from the animal kingdom. Programmers will rewrite software using new techniques if requirements change. New tools outsell older tools that do not provide the same benefits. New techniques that reduce development time or improve quality replace old techniques.

Just as species become extinct old software technique die from disuse or replacement. Old code does not die off as quickly as some of us would like. It can last forever in our systems if the requirements it fulfils and infrastructure it uses do not shift too markedly.

Charles Darwin had two theories that he espoused in his books "The Origin of the Species" and "The Descent of Man". One could be paraphrased as 'survival of the fittest' and the other once was 'sexual selection'. Sexual selection is where a mate is selected because he or she is bigger, is stronger, is darker, has more colourful tail feathers or has less hair and therefore individuals with these characteristics tend to propagate.

I have briefly explained how the fittest software and development techniques succeed at the cost of inferior versions but it is also true that the sexiest software can be propagated despite any objective superiority. This can happen through good marketing or the support of a major company. Software standards are often agreed upon, even though they may not be the best based on an objective measure of quality. Often the benefit to adopt a standard exceeds the cost of the standard being suboptimal.

'Survival of the fittest' and 'sexual selection' are alive and well in the software industry. I expect SOA to remain an important part for the genetic makeup of future software development even after the term has lost its currency.

References

Dawkins, Richard, A Devil's Chaplain: Selected Essays, Phoenix 2003

Darwin, Charles, The Descent of Man, 1871, http://www.infidels.org/library/historical/charles_darwin/descent_of_man/

Darwin, Charles, The Origin of the Species, 1859, http://www.infidels.org/library/historical/charles_darwin/origin_of_species/

The Passion of Programming

Sometimes I think back to high school and University where programming was a passion. In my kit full of mediocre skills programming stood out as something I was good at and something I had a passion for. It's interesting to reflect on this today when I am asking programmers to be professional, to document their work properly and to follow the standards. There is passion amongst some of them. Sometimes this is detrimental and results in religious wars on technical ideologies but by and large there is not enough passion about the work. It's not like any other job. As a programmer you are building something unique every day.

I used to liken programming to building castles out of air. You create something useful from nothing except logic. It is not quite like that any more. Perhaps it never was. After all you usually needs a big computer, a powerful compiler, some noisy printers and dare I recall, stacks of punched cards and a card reader. This things faded into the environment and what really matter was logic. You seem to need much more to get started these days. Application servers, database servers, browsers, virtual machines and libraries of standard routines, classes and services. A good SOA approach is one that assembles and application from a catalogue of pre-fabricated parts. We do not building castle from air any more we are putting together with prefabricated walls, floors and ceilings, and then furnishing it using IKEA.

This is of course a good thing. We want to be able reuse what we develop and do not want to reinvent the wheel every time we build something new. The art has changed but the skill is still required, and so I hope the passion remains for many.

Like the components the human resource have also changed. There seems to be more specialties than before. I used to work for small organisations that did not really distinguish between programmers, designer and analysts. Now we can't get by without a DBA, a configuration manager, web designers, an application server specialist and a host of architects, test professionals and user support personnel.

Footnote:

Alas! "Building Castles out of Air" was not my original work. It has been bouncing around in my brain for many years first seeded there by a classic in the IT literature. After writing this blog I found the following quote from Fred Brooks' "The Mythical Man-Month". He writes it better than I.

"The programmer, like the poet, works only slightly removed from pure thought-stuff. He builds his castles in the air, from air, creating by exertion of the imagination."

http://www.wordyard.com/2006/10/02/mythical-man-month/