Introduction

Since we’ve had computers, we’ve had programs running on them. But what these programs look like, as well as how they are written, has changed drastically over the short period we’ve had electronic digital computers. It can be helpful to review this evolution before we delve into software engineering as a discipline, because this storied history has an immense impact on how software is created.

Key Terms

Some key terms to learn in this chapter are:

• Software Engineering
• Programming Paradigms
  • Structured
  • Object-Oriented
  • Functional
  • Logic
• The Software Crisis

Early Programming

You probably remember seeing pictures of the room-sized ENIAC, the first digial computer constructed in the United States, in CIS 115. But do you remember how it was programmed?

That’s actually what these two women - Gloria Ruth Gordon Bolotsky and Esther Gertson - are doing in the picture. They are programming the ENIAC by plugging jumper cables into a switchboard. Effectively, programming the ENIAC consisted of physically rewiring the computer to support the new program. To make matters even more challenging, the women who developed the ENIAC’s programs were not initially allowed to see the machine due to wartime secrecy concerns. Instead, they had to develop their programs from studying schematics of the computer, and technicians would attempt to recreate the programs they wrote.¹

This laborious rewiring process was eventually replaced with stored programs, programs stored in the computer’s memory. The concept was first introduced by Alan Turing in his Universal Turing Machine. In his original conception of Turing Machine, input to the computer was stored on an infinitely long paper tape, and output was written to the same tape - both in the forms of ones and zeros. The program was hard-wired into the head of the machine. Turing had an epiphany when he realized that he could represent instructions to the machine with binary numbers, and then a program using those instructions could also be stored on the paper tape.

These ideas were incorporated into electronic computers by several computer scientists, including J. Presper Eckert and John Mauchly, inventors of the ENIAC, as well as John Von Neumann, who was first to publish such an architecture in his paper “First Draft of a Report on the EDVAC”. For this reason, a computer architecture allowing for stored programs is commonly referred to as a von Neumann architecture, and this is the basis of modern digital computers.

Stored programs also allowed us to develop bootstrap code - libraries of common procedures that could be re-used for future programs, and were loaded into the computer as it was warmed up. This was an important predecessor to modern programming libraries and operating systems.

The next major innovation in software design was the development of programming languages and the associated technologies of compilers and interpreters that allowed programmers to write programs in a higher-level programming language that would then be translated into a machine language for a stored-program computer. Pictured is Grace Hopper, the creator of the first programming language, FLOW-MATIC and influential co-creator of COBOL.

The development of programming languages is especially important in that it allowed us to develop abstractions simplifying development of software and allowing us to express significantly more complex ideas in computer code.

Software Engineering

The Apollo missions were marked for their heavy use of computers. For example, this next picture is the memory core of an Apollo Guidance Computer, developed for the Apollo missions. The two hands you see in the picture belong to professional weavers who were hired to implement the program for the mission. The program was encoded using core rope memory, a kind of read-only memory that stored 0 and 1 values by the inclusion or omission of a wire through a series of tiny magnetic rings. Each computer program was truly hard-coded - any mistake would entail a costly rebuilding of the core. ¹

Thus, there was significant pressure on NASA to make sure programs were correct. This next picture is Margaret Hamilton, one of the software developers who led the effort to create those guidance programs. She insisted a rigorous approach and comprehensive testing was crucial to the success of Apollo. And her approach paid off - No software bugs were ever found in the Apollo guidance software, and it was later adopted for use with the space station Skylab. The term “Software Engineering” was coined by her to describe her work. And just to emphasize the size of that task - that stack of documents next to her is the source code for the Apollo guidance software! ².

Unfortunately, not all software was being built to NASA’s exacting standards. At the 1968 NATO Software Engineering Conference, the same year of the first manned Apollo mission, the term “Software Crisis” was coined to describe a growing problem in the software industry.

Growth of Computing

The unfolding software crisis was coupled to the growth of computing. As Edsger Dijkstra put it in his 1972 ACM Turing Award acceptance speech: ¹

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.

This increase in computing power was matched by increases in affordability, as the sampling of computers in the following table demonstrate.

Not surprisingly, governments, corporations, schools, and even individuals purchased computers in larger and larger quantities, and the demand for software to run on these platforms and meet these customers’ needs likewise grew. Coupled with this rising demand for programs was a demand for skilled software developers, as reflected in the following table of graduation rates in programming-centric degrees:

Unfortunately, this graduation rate often lagged far behind the demand for skilled graduates, and was marked by several periods of intense growth (the period from 1965 to 1985, 1995-2003, and the current surge beginning around 2010). During these surges, it was not uncommon to see students hired directly into the industry after only a course or two of learning programming (coding boot camps are a modern equivalent of this trend).

Try to imagine yourself taking a professional programming job after your first programming course - what quality of software do you think you would be writing?

For that matter, how do you write programs now? Do you read the entire assignment before you start writing your program (if you’re like most of my students, I know you don’t). Do you draw out diagrams of your software’s architecture before you start coding? Do you write down exactly what you want the program to do before you start coding? Or do you just dive in and start writing code?

This latter approach is often called cowboy coding, after the old spaghetti westerns where the protagonist does not play by the rules. Cowboy coding is marked by a lack of discipline and process. Effectively it it the antithesis of software engineering.

Evolution of Programming

The poor programming practices of cowboy coding and the growing numbers of unskilled programmers did not go unremarked. In fact, many of the programming language features you are familiar with were developed explicitly to counter these trends - i.e. to ‘idiot-proof’ programming.

Consider all the control-flow constructs you know: the for loop, the while loop, the if/else statement, the switch statement, the ternary operator, etc. None of these existed in early programming languages. Instead, the role they each played was provided by a single multipurpose command, GOTO. A GOTO would jump program execution to a labeled point in the program, much like a choose your own adventure book would ask you to turn to a page. And much like an unskilled author could create a nightmare of such a book, an unskilled programmer could make an absolute mess.

Structured programming is the programming paradigm that introduced these control-flow structures, initially in ALGOL in the late 1950’s, and have been a staple of programming languages since. In fact, structured programming has become so ingrained in modern programming languages that I wouldn’t be surprised if you had never heard the term. But structured programming didn’t just introduce control-flow constructs, it also gave us procedures and code blocks, the basis of functions. That’s right, the humble concept of a function did not exist in early programming languages!

Other paradigms - Object-Orientation, Functional Programming, and Logic Programming were also born primarily as a way to make programs easier to understand, write, and verify.

The tools used for programming also evolved side-by-side with the languages. Punched Cards were the primary means of creating programs until the mid-1970’s, and many legacy systems continued to use this technology into the 1980s. Punched cards were punched using a specialized keyboard similar to a typewriter known as a keypunch that, instead of leaving inked characters on paper, punched holes in cards. There was no undo button - mistakes had to be repunched.

With the advent of the computer terminal in the late 1960’s

Similarly, the debugging tools available to programmers have evolved over time. Only after the migration from punch-card systems to CRT monitors and keyboards did command-line debuggers appear - simple programs that could dump the contents of memory to the monitor as the program being debugged was running. Symbolic debuggers (those that hold onto symbol names parsed during the compilation process and attach the corresponding in-memory values) and the ability to insert breakpoints (locations in code where execution would be paused) did not appear until much later, as this 1984 PC Magazine ad attests:

The modern Integrated Development Environment (IDE), programs like Visual Studio, XCode, Eclipse, Atom, VS Code, and the various JetBrains products, which combine program editing and debugging, simply didn’t exist for much of programming history. The first IDE was TurboPascal introduced by Borland Ltd. in 1983. Visual Studio did not appear until 1991.

Software Project Management

Beyond the increasing demand for programs, the large-scale recruitment of unprepared programmers, and the lack of good development tools, another factor marked many failed projects, mismanagement.

Software was a new kind of product, and nobody had a really strong grasp of what would be involved in developing it. The Steve Jobs quote this chapter opened with, “When we started off we didn’t know how to spell software” is telling. Similarly, Dick Battin, one of the engineers behind the Apollo Guidance system admitted “… we had no idea how we were going to do this job, other than to try [to] model it after the Mars probe.” ¹

If the programmers and software engineers who were creating the software didn’t have a clear picture of what would be involved in creating software products, the business professionals who were tasked with managing them had even less. Barry Boehm’s 1981 text Software Engineering Economics sought to provide guidance to these project leaders, applying the principles of scientific management to software development and providing detailed models for mathematically estimating project costs, time, and personnel.² The resulting Constructive Cost Model (COCOMO) was a first of its kind, and was widely adopted by the business community.

An example of one of the equations from this text is:

Where:

• $MM$ is the time spent on the project, measured in man months (152 working hours)
• $C$ is a constant
• $KDSI$ is the number of source code lines in the delivered product
• $k$ is another constant

A common falicy arising from the practice of using man months as a metric is the idea that adding more people to a project will get it done sooner. This is the basis for Brooks Law: adding more people to a late software project will make it later. This should be obvious to you as a programmer; bringing new programmers on board to help with an existing project means they must learn the details of the system being built, and the only people who can teach them those details are those that are currently building it. Increasing team size also increases meeting time and bureaucracy involved.

Brooks Law was formulated by Fredrick Brooks in his book “The Mythical Man-Month”.³ Brooks was one of the managers for IBM’s OS/360 project, and this book encompasses many of the mistakes he made managing the project and the lessons he learned from them. These are mistakes he sees other managers continuing to make. He once quipped:⁴

Some people have called the book the "bible of software engineering." I would agree with that in one respect: that is, everybody quotes from it, some people read it, and few people go by it."

Many of the management techniques applied to software development in the early days are based on clear misconceptions of what the endeavor of software development really is. It is both an exacting engineering challenge and a creative activity. Efforts to ‘improve productivity’ often have the inverse effect - as Brooks would attest, making a team larger increases the amount of communication required. Similarly, studies have shown that ‘crunch time’, the practice of asking programmers to work late hours and weekends, actually leads to negative productivity, as tired programmers make errors that require more time to fix than was saved.

Similarly, the metrics used by software managers can be flawed. Lines of code may make sense as a metric when you’re writing COBOL or FORTRAN code, but the use of modern programming languages and libraries greatly reduces the length of programs. For example, in Node you can write a fully functioning static web server with just one library and four lines of code:

const express = require('express');
const app = new express();
app.use(express.static('public'));
app.listen(80);

And, if you don’t care too much about readability, you could reduce this to one line! Conversely, if you are being evaluated by the number of lines written, you can certainly make such a program far, far longer. This leads to bloat, and also makes your code more difficult to read and maintain. Either way is a losing proposition - it is far better to write programs in a comfortable-to-read manner, as this makes them easier to debug and maintain.

Unfortunately, many of the mistakes Brooks identified continue to crop up in modern software development projects, making poor project management another major contributor to the software crisis.

The Software Crisis

Now that we know some of the causes, what exactly was the software crisis? It is a period of time marked by bungled software projects, whose common problems included:

• Projects that ran over-budget
• Projects that ran over-time
• Software that made inefficient use of calculations and memory
• Software was of low quality
• Software that failed to meet the requirements it was developed to meet
• Projects that became unmanageable and code difficult to maintain
• Software that never finished development

Let’s review some representative examples.

The IBM OS/360 Project (1963-1965)

This ambitious project was undertaken by IBM to create a single operating system that would run on all mainframes manufactured by IBM. While operating systems that can run on different models of computers are commonplace today, at one time this was a very novel idea. IBM was offering half a dozen mainframe products at the time, and each had is own, incompatible, operating systems. On April 7th, 1964 IBM announced a new lineup of products - 6 models of computers, with 44 peripherals, and all were to be compatible with each other! The new OS/360 would make this feat possible.

But the software development staff struggled to accomplish the central goal of OS/360 - the ability to run more than one program at a time, and other issues began to crop up. IBM responded by hiring 1,000 more developers to assist the project, spending more in a single year than had been planned for the entire product development process! The operating system was eventually shipped, but a month late and far over budget.¹

The Therac-25 (1985-1987)

Perhaps the most chilling software failure in the history of Computer Science is the Therac-25, a machine to deliver radiation therapy to cancer patients. The software used was repurposed from the earlier Therac-6 and Therac-20 model, which had hardware interlocks to prevent certain kinds of failure conditions. The Therac-25 removed these in favor of software-based safeties. A race condition in the controller software, along with poor user interface design and assorted other issues could lead to incorrectly delivered doses of radiation - as much as 100 times the intended dose. Between 1985 and 1987 at least six patients were exposed to dangerous doses of radiation and developed symptoms of radiation sickness; three of these patients died ².

The Denver Airport Baggage System (1995)

The Denver International Airport Baggage System was designed to be the most advanced baggage system in the world, automating the handling of baggage throughout the airport. The project was a spectacular failure, leaving the airport inoperable for 16 months after the rest of the airport was ready, running $560 million dollars over budget, and the final system achieved only a fraction of the intended functionality. ³

The Ariane 5 Rocket (1996)

The European Space Agency’s unmanned Ariane 5 Rocket exploded just seconds after launch. This was the sad culmination of a decade-long project costing $7 billion. An investigation into the failure showed that the issue was an integer overrun error in the control software that occurred when a 64-bit floating point was converted into a 16-bit signed integer - the resulting integer value was too large to be represented by a 16-bit integer, leading to a crash of the primary system. The backup system encountered the same problem, leading to a destabilized flight and subsequent explosion. ⁴

The German Toll Collect system (2003)

Germany contracted with the Toll Collect, a syndicate involving large corporations including DaimlerChrysler, Deutsche Telekob, and Cofiroute, to establish an automated toll-collection system that would use GPS receivers mounted in trucks to collect Autobahn usage data and automatically bill truckers’ accounts for the usage. The project was slated to open in August 2003, but was eventually cancelled by the German government when the project failed to be ready by February 2004, after Toll Collect admitted the system would not likely be ready until 2006. Over 156 million euro of revenue was lost due to uncollected tolls before the contract was canceled. ⁵

The Healthcare.gov Launch (2003)

The Healthcare.gov marketplace was a cornerstone of the Affordable Care Act - a website where Americans without access to employer-based health insurance could enroll in a competitively-priced insurance plan. At launch the site was plagued with technical issues, and only an estimated 1% of its users were able to complete an enrollment, and many of these applications were failed for missing information.

FAA NextGen (2003-)

In 2003, Congress authorized the Federal Aviation Administration to replace the now 45 year old Air Traffic Control System used to control all flights over the US, Host. The new system, named, NextGen, is a modernized approach using GPS and able to hand-off control to multiple redundant systems. Its development has been plagued with issues, with many features pushed back five or more years and costs ballooning an estimated $2.6 billion over the initial projected budget. ⁶

US National Grid Gas Company (2012)

The New York based utility, National Grid, sought to replace its old Oracle system with a cloud-based system built using SAP technology, at an initial cost of $383.8 million. The resulting system had serious errors, including making incorrect payments to workers that culminated in lawsuits against the utility. National Grid had to hire and additional 450 contractors to fix the software flaws, and an additional 400 workers to manually perform the work the system was supposed to be doing while it was fixed. The additional costs were estimated to be an additional $561.30 million, a cost passed on to their customers. ⁷

Heartbleed (2014)

In 2014, the bug heartbleed was discovered in the open-source library OpenSSL, used to create secure connections across the internet. The bug allows any internet-connected adversary to read the memory of systems using unpatched OpenSSL-implementing software. OpenSSL is one of the most widely used TLS libraries, and is integrated into both Apache and nginx open-source server software, which together host over 66% of the web sites on the Internet. The bug was the result of a simple programming error in the library. ⁸

Meltdown and Spectre Exploits (2018)

The duo of Meltdown and Spectre are processor-based vulnerabilities found in 2018. The Meltdown attack exploits out-of-order execution, a performance enhancing feature of modern processors, to allow a malicious program to access memory outside of that assigned to it by the operating system - i.e. it can read memory belonging to other applications, and even the OS. Spectre exploits branch prediction and speculative execution, a feature of modern processors that allow them to “guess” what the program might need to do next and execute it ahead of time. The result is that some actions that would not have been carried out by the program are nonetheless executed; a spectre attack takes advantage of this to “trick” the processor into speculatively executing a properly-designed program in a way that leaves some data open for the exploit to read. Both are the results of optimizing features that break the traditional program execution model, and therefore the security expectations programs are designed against. ⁹

Boeing 737 Max MCAS (2018)

For their 737 Max model jetliner, Boeing increased the engine size on a 737 airframe to increase fuel efficiency. To fit the oversize engines on the wings without changing other aspects of the airframe (which would require significant retooling of their factories), they shifted these large engines forward, which changed the aerodynamic properties of the jet. This change could result in engines stalling if the nose of the plane was pitched too far forward. To prevent this, a computer system was added, the MCAS, which would bring the nose back down when the angle of attack was too steep, effectively cutting the pilot out of the control loop. The software on this system focused on the output of a single angle-of-attack sensor, rather than redundant sensors, and this interaction with faulty sensors led to two deadly crashes shortly after it began operations, and the eventual grounding of all Boeing 737 Max planes. ¹⁰

Summary

If you paid attention to the dates in the previous section, then you know the software crisis is still very much with us, despite all the improvements in programming languages, tools, and management practices. The best defense against project failure may well be found in software engineering processes, which is the subject we will be studying for the rest of the book.

But just like any process, if it is misapplied, poorly followed, or simply blindly followed for the sake of procedure it will not succeed. Many of the software projects listed in the previous section did engage in a software engineering process, yet failed nonetheless. Keep this in mind as you learn these processes, and try to understand the underlying purpose of each step.