Why Building Software Is So Hard These Days

Consider what you used to use to build a business application — say, the system that allows a bank customer bank to withdraw cash from an ATM — not that many years ago:

  • A proprietary mainframe, designed and built from the ground up for just this sort of work (high-speed, high-volume transaction processing), which the ATM talks to via …
  • … proprietary communications protocols focused on enabling maximum throughput, that run on …
  • … proprietary communications hardware, designed and built for high-speed data transfer, and used to create …
  • … a private network infrastructure

Software-wise, the mainframe would have been running:

  • A proprietary OS
  • A proprietary transaction monitor to host the application software
  • A proprietary database

All of which were also designed and built for high-speed, high-volume transaction processing.  They all probably came from the same vendor, too, and were documented to a high degree of:

  • Comprehensiveness
  • Depth
  • Succinctness
  • Consistency

Your programming of the above stack — using IBM as an example — was accomplished via:

  • The most appropriate programming language (COBOL)
  • The vendor’s provided development environment (ISPF)
  • The vendor’s operating system and transaction monitor (CICS)
  • The vendor’s database (DB2)
  • The vendor’s control language of choice (JCL)

There’s a little more to it than that —single-step debuggers, for instance, or third-party alternatives to any of the above — but that’s the gist of it from a coal-face point of view.  What I’m trying to get across here is that the technology stack was generally highly coherent and consistent, being mostly supplied by a single, vertically integrated vendor.

Finally, let’s remember that testing was carried out on a deployment stack that was installed and maintained by the same people who maintained the production environment, obviating any need for the development team to become experts in wrangling it.

Now, here — as best I remember it — is what I was using a couple of years ago to build what was actually a fairly simple application on a Unix server stack:

  • Oracle database for the application data
  • MySQL database for the control data
  • Java for business code
  • PHP for control code
  • Hibernate for data access management
  • The MyEclipse development environment
  • SiteMesh for providing consistent look-and-feel
  • Spring framework for transaction management and MVC management
  • Any number of other pieces of open-source utilities, both for development and runtime support (there were a lot that I no longer recall)
  • ASP.Net for building web pages
  • CSS for controlling user access to features of the application
  • XML for data encapsulation
  • WebSphere Application Server for running the application
  • IBM HTTP Server for web-facing functions
  • Java Message queues for internal data transfer
  • WebSphere Message Queues for external data transfer
  • WebSphere Message Broker for systems integration
  • Microsoft Active Directory for user authentication
  • Stuff to enable load-balancing, distributed processing and improving scalability
Bear in mind that unit testing tends to be carried out using an environment running on the individual developers’ workstations that is installed and maintained by the developer — which means all of the above becomes the developer’s responsibility.

The points I’m trying to make are that:

  • The amount of technology that an individual working on the modern IT project has to be familiar with is far, far greater than it used to be
  • The technology mix comes from a wide variety of vendors, not all of whom are vendors in the traditional sense but who may be considered more as technology enthusiasts, with the natural impact on the quality of product documentation and support that this might imply
  • The complexion of that technology mix changes pretty quickly, from project to project and over time (how many of the open-source components in my stack ought to be replaced now to reduce the company’s exposure to unsupported, undocumented technology?)

In short:  certain tasks are best done by specialists, and software development is one of them.  By demanding that developers carry a wide and ever-changing set of supporting skills, we are necessarily diluting their core skill set to the detriment of the software development process.

(Visited 37 times, 1 visits today)

Leave a Reply