Welcome

Welcome to CC 410 - Advanced Programming. This course is designed to be a capstone experience at the end of the Computational Core program, building upon our prior knowledge and experience to help us become a truly effective programmer. In this course, we’ll not only learn new skills and techniques, but we’ll try to pull back the curtain and explain the history of programming and why we do some of the things we do.

Big Ideas

In this course, we’re going to cover a lot of content. However, it can be grouped into a few big ideas in programming:

• How can we write professional looking code that is easy for others to understand?
• How can we effectively debug and test our programs to minimize the number of bugs?
• What is object-oriented programming, really, and why is it so popular?
• How can we develop programs that have a graphical user interface (GUI)?
• What is event-driven programming, and how does it relate to the development of GUIs?
• What are some common design patterns that we can use in our code?
• How can we interface with applications on the Internet?
• How do we design and develop our own programs from scratch to solve a particular problem?

We’ll spend some time covering each of these in more detail as we go through the course. In this module, we’ll start working on the first two - writing professional code and minimizing bugs through testing and debugging.

Getting Started

Before we dive too deeply into this topic, let’s take a step back and examine some of the history of programming that lead to our current state of the art that revolves around object-oriented programming. To do that, we’ll need to explore the software crisis and the topic of structured programming.

The Growth of Computing

By this point, you should be familiar enough with the history of computers to be aware of the evolution from the massive room-filling vacuum tube implementations of ENIAC, UNIVAC, and other first-generation computers to transistor-based mainframes like the PDP-1, and the eventual introduction of the microcomputer (desktop computers that are the basis of the modern PC) in the late 1970s. Along with a declining size, each generation of these machines also cost less:

This increase in affordability was also coupled with an increase in computational power. Consider the ENIAC, which computed at 100,000 cycles per second. In contrast, the relatively inexpensive Commodore 64 ran at 1,000,000 cycles per second, while the more pricey IBM PC ran 4,770,000 cycles per second.

Not surprisingly, governments, corporations, schools, and even individuals purchased computers in larger and larger quantities, and the demand for software to run on these platforms and meet these customers’ needs likewise grew. Moreover, the sophistication expected from this software also grew. Edsger Dijkstra described it in these terms:

The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem. – Edsger Dijkstra, The Humble Programmer (EWD340), Communications of the ACM

Coupled with this rising demand for programs was a demand for skilled software developers, as reflected in the following table of graduation rates in programming-centric degrees (the dashed line represents the growth of all bachelor degrees, not just computer-related ones):

Unfortunately, this graduation rate often lagged far behind the demand for skilled graduates, and was marked by several periods of intense growth (the period from 1965 to 1985, 1995-2003, and the current surge beginning around 2010). During these surges, it was not uncommon to see students hired directly into the industry after only a course or two of learning programming (coding boot camps are a modern equivalent of this trend).

All of these trends contributed to what we now call the Software Crisis.

The Software Crisis

Video Materials

At the 1968 NATO Software Engineering Conference held in Garmisch Germany, the term “Software Crisis” was coined to describe the current state of the software development industry, where common problems included:

• Projects that ran over-budget
• Projects that ran over-time
• Software that made inefficient use of calculations and memory
• Software was of low quality
• Software that failed to meet the requirements it was developed to meet
• Projects that became unmanagable and code difficult to maintain
• Software that never finished development

The software development industry sought to counter these problems through a variety of efforts:

• The development of new programming languages with features intended to make it harder for programmers to make errors.
• The development of Integrated Development Environments (IDEs) with developer-centric tools to aid in the software development process, including syntax highlighting, interactive debuggers, and profiling tools
• The development of code repository tools like SVN and GIT
• The development and adoption of code documentation standards
• The development and adoption of program modeling languages like UML
• The use of automated testing frameworks and tools to verify expected functionality
• The adoption of software development practices that adopted ideas from other engineering disciplines

This course will seek to instill many of these ideas and approaches into your programming practice through adopting them in our everyday work. It is important to understand that unless these practices are used, the same problems that defined the software crisis continue to occur!

In fact, some software engineering experts suggest the software crisis isn’t over, pointing to recent failures like the Denver Airport Baggage System in 1995, the Ariane 5 Rocket Explosion in 1996, the German Toll Collect system canceled in 2003, the rocky healthcare.gov launch in 2013, and the massive vulnerabilities known as the Meltdown and Spectre exploits discovered in 2018.

Language Evolution

Video Materials

One of the strategies that computer scientists employed to counter the software crisis was the development of new programing languages. These new languages would often 1) adopt new techniques intended to make errors harder to make while programming, and 2) remove problematic features that had existed in earlier languages.

A Fortran Example

Let’s take a look at a working (and in current use) program built using Fortran, one of the most popular programming languages at the onset of the software crisis. This software is the Environmental Policy Integrated Climate (EPIC) Model, created by researchers at Texas A&M:

Environmental Policy Integrated Climate (EPIC) model is a cropping systems model that was developed to estimate soil productivity as affected by erosion as part of the Soil and Water Resources Conservation Act analysis for 1980, which revealed a significant need for improving technology for evaluating the impacts of soil erosion on soil productivity. EPIC simulates approximately eighty crops with one crop growth model using unique parameter values for each crop. It predicts effects of management decisions on soil, water, nutrient and pesticide movements, and their combined impact on soil loss, water quality, and crop yields for areas with homogeneous soils and management. -- EPIC Homepage

You can download the raw source code and the accompanying documentation. Open and unzip the source code, and open a file at random using your favorite code editor. See if you can determine what it does, and how it fits into the overall application.

Try this with a few other files. What do you think of the organization? Would you be comfortable adding a new feature to this program?

New Language Features

You probably found the Fortran code in the example difficult to wrap your mind around - and that’s not surprising, as more recent languages have moved away from many of the practices employed in Fortran. Additionally, our computing environment has dramatically changed since this time.

Symbol Character Limits

One clear example is symbol names for variables and procedures (functions) - notice that in the Fortran code they are typically short and cryptic: RT, HU, IEVI, HUSE, and NFALL, for example. You’ve been told since your first class that variable and function names should express clearly what the variable represents or a function does. Would rainFall, dailyHeatUnits, cropLeafAreaIndexDevelopment, CalculateWaterAndNutrientUse(), CalculateConversionOfStandingDeadCropResidueToFlatResidue() be easier to decipher? (Hint: the documentation contains some of the variable notations in a list starting on page 70, and some in-code documentation of global variables occurs in MAIN_1102.f90.).

Believe it or not, there was an actual reason for short names in these early programs. A six character name would fit into a 36-bit register, allowing for fast dictionary lookups - accordingly, early version of FORTRAN enforced a limit of six characters for variable names¹. However, it is easy to replace a symbol name with an automatically generated symbol during compilation, allowing for both fast lookup and human readability at a cost of some extra computation during compilation. This step is built into the compilation process of most current programming languages, allowing for arbitrary-length symbol names with no runtime performance penalty.

Structured Programming Paradigm

Another common change to programming languages was the removal of the GOTO statement, which allowed the program execution to jump to an arbitrary point in the code (much like a choose-your-own adventure book will direct you to jump to a page). The GOTO came to be considered too primitive, and too easy for a programmer to misuse ².

However, the actual functionality of a GOTO statement remains in higher-order programming languages, abstracted into control-flow structures like conditionals, loops, and switch statements. This is the basis of structured programming, a paradigm adopted by all modern higher-order programming languages. Each of these control-flow structures can be represented by careful use of GOTO statements (and, in fact the resulting assembly code from compiling these languages does just that). The benefit is using structured programming promotes “reliability, correctness, and organizational clarity” by clearly defining the circumstances and effects fo code jumps ³.

Object-Orientation Paradigm

The object-orientation paradigm was similarly developed to make programming large projects easier and less error-prone. We’ll examine just how it seeks to do so in the next few chapters. But before we do, you might want to see how language popularity has fared since the onset of the software crisis, and how new languages have appeared and grown in popularity in this animated chart from Data is Beautiful:

Interestingly, the four top languages in 2019 (Python, JavaScript, Java, and C#) all adopt the object-oriented paradigm - though the exact details of how they implement it vary dramatically.

The term “Object Orientation” was coined by Alan Kay while he was a graduate student in the late 60s. Alan Kay, Dan Ingalls, Adele Goldberg, and others created the first object-oriented language, Smalltalk, which became a very influential language from which many ideas were borrowed. To Alan, the essential core of object-orientation was three properties a language could possess: ⁴

• Encapsulation
• Message passing
• Dynamic binding

We’ll take a look at each of these in the next few chapters.

Writing Professional Code

Video Materials

As we saw earlier in this module, the software development industry adopted many new processes and ideas to help combat the issues that arose during the software crisis. One of the major things they focused on was how to write code that is easy to understand, easy to maintain, and works as intended with a minimal amount of bugs. Let’s review a few of the concepts that came from those efforts, which we’ll learn more about throughout this semester.

Object-Oriented Programming

The use of object-oriented programming languages was one major outcome of the software crisis. An object-oriented language allows developers to build code that represents real-world concepts and ideas, making it easier to reason about large software programs. In addition, the concept of encapsulation helped ensure data stored and manipulated by one part of the program wasn’t inadvertently changed by a bug in another part. Finally, through message passing and dynamic binding, we could write more advanced functions that allowed our code to be very modularized, flexible, and highly reusable. We’ll spend the next several modules in this course covering object-oriented programming in much greater detail.

Unit Testing

Another major movement in the software industry was toward the use of automated testing frameworks and the use of unit testing. Unit testing involves writing detailed tests for small units of a program’s source code, often individual functions, that exercise the expected functionality of the code as well as checking for any edge cases or expected errors.

In theory, if the unit tests are properly written and perform all possible operations that the code should perform, than any code passing the tests should be considered complete and ready for use. Of course, coming up with a set of unit tests that can account for all possible scenarios is just as impossible as writing software that doesn’t contain any bugs, but it can be a great step toward writing better software.

A common software development methodology today is test-driven development or TDD. In test-driven development, the unit tests are developed first, based on the software specification, before the source code is ever written. In that way, it is easy to know if the software actually does what the requirements says it should, instead of the test simply being written to match the code that exists. (It is shockingly common for unit tests to be written based on the code it should test, which is equivalent of looking at the answers when doing a word scramble - you’ll find what you expect to find, but won’t actually learn anything useful from it.)

Another useful feature of unit tests is the ability to re-run tests on the program after an update has been developed, which is known as regression testing. If the program previously passed all available unit tests, then failed some of those tests after an update, we know that we introduced some unintended bugs in the code that can be repaired before publishing an update. In that way, we can avoid sending out an update that ends up making things even worse.

Code Coverage

Along with unit testing, another useful technique is calculating the code coverage of a set of tests. Ideally, you’d like to make sure that each and every line of code in the program is executed by at least one test - otherwise, how can you really say that that line does what it should? This is especially difficult in programs that contain multiple conditional statements and loops, or any code that checks for and handles exceptions.

There are various ways to measure code coverage, including this list from Wikipedia:

• Function coverage - has every function been called?
• Statement coverage - has every statement been executed?
• Edge coverage - has every edge in the control flow graph been executed?
• Branch coverage - has every branch in each control structure been executed?
• Condition coverage - has every boolean expression been evaluated to both true and false?

There are various different ways to measure code coverage that we’ll discuss later in this course, but for now we’ll just look at statement coverage. Thankfully, there are some great tools for computing the code coverage of a set of unit tests. Our goal is always to get as close to 100% coverage as possible.

Documentation

Another major focus among professional coders is the inclusion of documentation directly in the source code itself. Many languages, such as Java, Python, and C#, include standards for documenting what various pieces of the code are for. This includes each individual source code file, classes, functions, attributes, and more. In many cases, this is done by including specially structured code comments in various places throughout the source code.

To make those comments easier to read and understand, many languages also include tools to automatically create developer documents based on those comments. A prime example of this is the Java API Documentation, which is nearly entirely generated directly from comments in the Java source code. In fact, you can compare the source code for the ArrayList class and the ArrayList Documentation in the Java API to get an idea of how this works.

Static Code Analysis

Finally, there are many tools available today that can perform static code analysis of source code, helping developers find and fix errors without ever even compiling and running the code. Some static code analysis tools are quite powerful, able to find logic errors or completely validate that the software meets a specification. These tools are commonly used in the development of critical software components, such as medical devices and avionics for aircraft, but they are also quite difficult to use.

In this course, we’re going to focus on a simpler form of static code analysis that will help us maintain good coding style. These tools are sometimes commonly referred to as “linters,” named for the old Unix ’lint’ tool that performed this task for code written in the C programming language. Of course, the use of the term “lint” is a reference to the tiny bits of fiber and fuzz that are shed by clothing, with the idea that by removing the “lint” that makes our code messy, we can have code that is cleaner and easier to read and maintain.

In fact, you may have already encountered these tools in your programming experience. Development environments such as the one used by Codio, as well as other integrated development environments (IDEs) such as Visual Studio Code, PyCharm, IntelliJ, and others all include support for static code analysis. Usually it takes the form of helpful error messages that show simple syntax and usage errors.

In this course, we’ll learn how to use some more powerful static code analysis tools to enforce a standard coding style across all of our source code. A [coding style] can be thought of as roughly equivalent to a dialect of a spoken or written language - it deals with common conventions and usage, beyond just the simple definitions and syntax rules of the language itself. By following a standardized style, our code will be easier to read and maintain for any developer who is familiar with that style.

Hello Real World

Example Videos

Based on the previous page, it sounds like writing professional code can be quite difficult. There are so many tools and concepts to keep track of, and, in fact, you may end up spending just as much time working with everything else around your code as you do writing the code itself. The benefit of all of this work comes later, when you have to update or maintain the code. If you’ve done a good job writing unit tests, checking for coverage, documenting and styling your code, you’ll end up with fewer bugs overall, and hopefully it will be easier to patch and update the code over the long term that it is in use.

Thankfully, in this course, we’re going to start small in this module with a new project we’re calling “Hello Real World.”

Hello Real World

Most programmers can recall the simple “Hello World” program they wrote when learning to program. For many of us, it is the first program we learned to write, and usually the first thing we write when learning a new language. It is almost a sacred tradition!

We’re going to build upon that in this module by learning to write a “Hello World” program of our own, but one that meets the following requirements:

1. It must be fully object-oriented, with the code placed within a method that is inside of a class, which is part of a package.
2. The code must include unit tests that fully verify that the code works properly in all cases.
3. The unit tests must achieve 100% code coverage of the source code.
4. The source code must contain full documentation for each file, class, and method, as defined by the language’s standard for in-code documentation.
5. The source code must pass all checks enforced through static code analysis based on a common coding style for the language.
6. The entire process should be easily executable at-will from the terminal, while providing opportunities for future full automation.
7. The resulting code should be stored in a version control software system.

That’s quite a tall order, but this is really how a professional software developer would approach writing good and maintainable code. In some languages, such as Java, a few parts of this process are pretty straightforward - Java is already fully object-oriented by default, and Java uses a common standard for creating in-code documentation. Other languages, such as Python, end up becoming more complex to work with as more requirements are added. For Python developers, a simple “Hello World” program is a single line of code, whereas this set of requirements requires multiple files to properly create a Python package. In addition, the Python language itself does not define a common standard for in-code documentation, so we must rely on external resources to determine what coding style we should follow.

Thankfully, we’ll go through this entire process step by step in the example portion of this module, and you’ll be able to follow along and build your own version of “Hello Real World.”

Summary

In this chapter, we’ve discussed the environment in which object-orientation emerged. Early computers were limited in their computational power, and languages and programming techniques had to work around these limitations. Similarly, these computers were very expensive, so their purchasers were very concerned about getting the largest possible return on their investment. In the words of Niklaus Wirth:

Tricks were necessary at this time, simply because machines were built with limitations imposed by a technology in its early development stage, and because even problems that would be termed "simple" nowadays could not be handled in a straightforward way. It was the programmers' very task to push computers to their limits by whatever means available.

As computers became more powerful and less expensive, the demand for programs (and therefore programmers) grew faster than universities could train new programmers. Unskilled programmers, unwieldy programming languages, and programming approaches developed to address the problems of older technology led to what became known as the “software crisis” where many projects failed or floundered.

This led to the development of new programming techniques, languages, and paradigms to make the process of programming easier and less error-prone. Among the many new programming paradigms was structured programming paradigm, which introduced control-flow structures into programming languages to help programmers reason about the order of program execution in a clear and consistent manner. Also developed during this time was the object-oriented paradigm, which we will be studying in this course.

Programming Today

Today, many software developers have adopted techniques designed to produce high quality code. These include the use of automated unit testing and test-driven development, as well as standardized use of code comments and linters to maintain good coding style and ample documentation for future developers. In the project for this module, we’ll explore what this looks like by building a simple “Hello World” program that uses all of these techniques.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.

Introduction

A signature aspect of object-oriented languages is (as you might expect from the name), the existence of objects within the language. In this chapter, we take a deep look at objects, exploring why they were created, what they are at both a theoretical and practical level, and how they are used.

Key Terms

Some key terms to learn in this chapter are:

• Encapsulation
• State
• Class
• Object
• Field
• Attribute
• Method
• Property
• Public
• Private
• Static

To begin, we’ll examine the term encapsulation.

Encapsulation

Video Materials

The first criteria that Alan Kay set for an object-oriented language was encapsulation. In computer science, the term encapsulation refers to organizing code into units, which provide two primary benefits:

• Providing a mechanism for organizing complex software
• The ability to control access to encapsulated data and functionality

Think back to the FORTRAN EPIC model we introduced in an earlier module. All of the variables in that program were declared globally, and there were thousands of them. If we open the code today, could we even find where a variable was declared? Initialized? Used? Could we be sure that we found all the spots it was used?

Also, how easily could we determine what part of the system a particular block of code belonged to? If we knew the program involved modeling hydrology (how water moves through the soils), weather, erosion, plant growth, plant residue decomposition, soil chemistry, planting, harvesting, and chemical applications, could we find the code for each of those processes?

Recall from our discussion on the growth of computing that, as computers grew more powerful, we looked to use them in more powerful ways. The EPIC project grew from that desire - if we could model all the aspects influencing how well a crop grows, then we could use that to make better decisions in agriculture. Likewise, if we could model the processes involved in weather, we could help save lives by predicting dangerous storms! A century ago, the only way to know a tornado was coming when you heard its roaring winds approaching your home. Now we have warnings that conditions are favorable to produce one hours in advance! This is all thanks to our ability to use computers to model some very complex systems.

How do we go about writing those complex systems? We probably wouldn’t want to follow the model that the EPIC software gives us. And, thankfully, neither did most software developers at the time - so computer scientists set out to define better ways to write programs. David Parnas formalized some of the best ideas emerging from those efforts in his 1972 paper “On the Criteria To Be Used in Decomposing Systems into Modules”. ¹

A data structure, its internal linkings, accessing procedures and modifying procedures are part of a single module.

Here he suggests organizing code into modules that group related variables and the procedures that operate upon them. For the EPIC module, this might mean all the code related to weather modeling would be moved into its own module. That means that if we needed to understand how weather was being modeled, we only had to look at the weather module.

They are not shared by many modules as is conventionally done.

Here he is laying the foundations for the concept we now call scope - the concept that a particular symbol (a variable or function name) is accessible only in certain locations within a program’s code. By limiting access to variables to the scope of a particular module, only code in that module can change the value. That way, we can’t accidentally change a variable declared in the weather module from somewhere else, like the soil chemistry module. This would be a very hard error to find, because if the weather module doesn’t seem to be working, that’s where we would probably spend our time looking for the error.

Programmers of the time referred to this practice as information hiding, as we “hid” parts of the program from other parts of the program. Parnas and his peers pushed for not just hiding the data, but also how the data was manipulated. By hiding these implementation details, they could prevent programmers who were used to the globally accessible variables of early programming languages from looking into our code and using a variable that we might change in the future.

The sequence of instructions necessary to call a given routine and the routine itself are part of the same module.

As the actual implementation of the code is hidden from other parts of the program, a mechanism for sharing controlled access to some part of that module in order to use it needed to be made. An interface, for example, that describes how the other parts of the program might trigger some behavior or access some value.

Packages

Let’s start by focusing on encapsulation’s benefits to organizing our code by exploring some examples of encapsulation you may already be familiar with.

Packages

The Java and Python libraries are organized into discrete units called packages. The primary purpose of this is to separate code units that potentially use the same name, which causes name collisions where the compiler or interpreter isn’t sure which of the possibilities you mean in your program. This means you can use the same name to refer to two different things in your program, provided they are in different packages. Many other languages refer to these as namespaces.

For example, there are two definitions for a Date class in Java: java.util.Date and java.sql.Date. While they are related, they serve different purposes, and we wouldn’t want to get them confused. If we needed to create an instance of both in our program, we would use their fully-quantified name to help the compiler know which we mean:

Java

java.sql.Date sqlDate = new java.sql.Date(System.currentTimeMillis());
java.util.Date utilDate = new java.util.Date(System.currentTimeMillis());
System.out.println(sqlDate.toString());
System.out.println(utilDate.toString());

Running that code gives this output:

2020-12-30
Wed Dec 30 17:23:50 GMT 2020

So, as we can see, these two classes are functionally different in some important ways.

While Java does not support aliases in imports, we can use an alias in Python to import two classes with the same name using different identifiers. For example, if there are two User classes in different packages, we could import them like this:

Python

from package_one import User as PackageOneUser
from package_two import User as PackageTwoUser

user_1 = PackageOneUser.User()
user_2 = PackageTwoUser.User()

Encapsulating code within a package helps ensure that the types defined within are only accessible with a fully qualified name, or when the using directive is employed. In either case, the intended type is clear, and knowing the package can help other programmers find the type’s definition.

Creating Packages

We can also declare our own packages, allowing us to use packages to organize our own code just as Java and Python have done with their standard libraries. Below are quick examples for how to do this in Java and Python.

Java

To create a class ClassName in a package cc410.package_name, we would include a package line at the top of the file:

package cc410.package_name;

public class ClassName{
    //code here
}

The ClassName.java file would be stored in app/src/main/java/cc410/package_name/. Basically, the package name corresponds to the folders where the source code is stored.

Python

To create a class ClassName in a package cc410.package_name, we would simply place ClassName.py in the src/cc410/package_name directory. We’d also need to include an __init__.py file in that directory to make it a package.

Finally, if we want the cc410 package to act as a meta-package and be executable we would also include an __init__.py and a __main__.py file in the src/cc410 directory as well.

Type Systems

Video Materials

Before we go further into some object-oriented concepts, let’s briefly review one important concept in programming - data types and type systems.

Primitive Data Types

Most programming languages include several primitive data types, which are the fundamental units of data that can be stored and represented by that programming language. Here’s a short list of those primitive data types for each language:

Any data that is stored by our program must fit into one of these data types. That is an important fundamental rule to remember - no matter how complex our code gets, everything is stored in primitive data types. That’s simply all there is.

Complex Data

What if we want to store more complex data, such as information about a person? Well, we could easily create an integer that stores the person’s age, and perhaps a string for the person’s name. Those are still just primitive data types, so we’re good there.

However, as you probably already know, we can group those items together into classes. However, before we can really understand classes and how they relate to encapsulation, we must look at a precursor to classes first. We’ll cover that later in this module.

Type Systems

The way that programming languages handle these data types is known as the type system of the language. Let’s look at two different ways to categorize type systems to see how they differ.

Static Typing vs. Dynamic Typing

In programming, there are two common ways that programming languages deal with data types. The first is called static typing, where each variable has a particular data type associated with it as soon as it is declared, and that variable can only store items of that data type. Because of this, we can use tools like the Java compiler to analyze our code before we ever execute it, making sure that we always are storing the correct type of data in each variable.

Java is a statically typed language. When we create variables in Java, we must assign data types to them, as in this example:

Java

int x = 5;
double y = 5.5;
String name = "CC 410";

Similarly, when we create methods in Java, we must declare the types of all parameters, as well as the return type of the method.

Python, on the other hand, is a dynamically typed language. That means that variables in Python do not have a particular data type assigned to them, and they can store multiple different types of data throughout the course of the program. Here’s an example:

Python

x = 5
x = 5.5
x = "CC 410"

This is a perfectly valid program in Python, and will execute just fine. However, as we’ll soon learn, this could lead to some preventable errors, and we’ll see how to resolve them.

Strong Typing vs. Weak Typing

Programming languages can also be classified based on their use of type systems in one other way. A strongly typed language always knows what data type is stored in a variable at any given time during the program’s execution. In statically typed languages such as Java, this is trivial - if the program compiles, then we know that the only possible data type that could be stored in a variable is the type listed in that variable’s declaration. It’s pretty straightforward.

However, what about Python? Python is dynamically typed, which means that each variable could store multiple different data types during a single program’s execution, and each time the program executes it could be different. However, at any given instant during the execution of the program, the Python interpreter knows exactly what type of data is being stored in each of the variables in the program. We can use methods such as isinstance() to confirm this. So, Python is also a strongly typed language.

So, what is a weakly typed language? A great example is code written in an assembly language. The computer will simply execute whatever is written, and has no way of keeping track of the types of data stored in each variable. Instead, it depends on the compiler or developer to make sure there are no type errors in the assembly code.

Making Python Statically Typed

As we learned in the “Hello Real World” project, we can add type annotations to Python code to convert Python into a statically typed language. Then, we can use tools such as Mypy to make sure there are no type errors in our code, much like the Java compiler does for Java code. So, here’s a rewritten example of Python code that is statically typed:

Python

x: int = 5
y: float = 5.5
name: str = "CC 410"

By adding these type annotations, we can tell Mypy what type of data we expect to be stored in each of these variables, and it can perform the same type checking process that the Java compiler uses. In this class, we’re going to focus on using statically typed Python code as much as we can.

Why This Matters

We’re spending a little time reviewing types and type systems now because it will help us understand the new concepts being introduced in the next few pages. Before the introduction of object-oriented programming, programmers had to use other tools to build more complex data types than the primitives we’ve discussed here.

Structs

Many object-oriented languages, such as C++ and C#, include the concept of a struct that form the basis of objects. A struct is an example of a compound data type, a data type composed from other types. This allows us to represent data in more complex ways by combining multiple primitive data types into a new type. This too, is a form of encapsulation, as it allows us to collect several values into a single data structure. Consider the concept of a vector from mathematics - if we wanted to store three-dimensional vectors in a program, we could do so in several ways. Perhaps the easiest would be as an array or list:

double[] vector = {3.0, 4.0, 5.0};

vector: List[float] = [3.0, 4.0, 5.0]

However, other than the variable name, there is no indication to other programmers that this is intended to be a three-element vector. And, if we were to accept it in a function, say a dot product, we’d need to check that the length of both arrays or lists was exactly 3:

public double dotProduct(double[] a, double[] b){
    if(a.length != 3 || b.length != 3){
        throw new IllegalArgumentException();
    }
    return a[0] * b[0] + a[1] * b[1] + a[2] * b[2];
}

def dot_product(a: List[float], b: List[float]) -> float:
    if len(a) != 3 or len(b) != 3:
        raise ValueError()
    return a[0] * b[0] + a[1] * b[1] + a[2] * b[2]

A struct provides a much cleaner option, by allowing us to define a type that is composed of exactly three integers. Java and Python don’t directly support structs, but we can use classes with just variables and a constructor to mimic a struct in those languages:

public class Vector3{
    public double x;
    public double y;
    public double z;
    
    public Vector3(double x, double y, double z){
        this.x = x;
        this.y = y;
        this.z = z;
    }
}

class Vector3:
    
    def __init__(self, x: float, y: float, z: float) -> None:
        self.x = x
        self.y = y
        self.z = z

Then, our dot product method can take two arguments of the Vector3 type:

public double dotProduct(Vector3 a, Vector3 b){
    return a.x * b.x + a.y * b.y + a.z * b.z;
}

def dot_product(a: Vector3, b: Vector3) -> float:
    return a.x * b.x + a.y * b.y + a.z * b.z

There is no longer any concern about having the wrong number of elements in our vectors - it will always be three. We also get the benefit of having unique names for these fields (in this case, x, y, and z).

Thus, a struct allows us to create structure to represent multiple values in one variable, encapsulating the related values into a single data structure. We can then use those data structures as new data types in our program. Variables, and compound data types, together represent the state of a program. We’ll examine this concept in detail next.

Modules

It might seem like the kind of modules that Parnas was describing don’t exist in Java or Python, but they actually do - we just don’t call them “modules”. Consider how you would compute the square root of a number:

Math.sqrt(9.5);

math.sqrt(9.5)

The Math or math class in this example is actually used just like a module! We can’t see the underlying implementation of the sqrt() method, it just provides to us a well-defined interface (i.e. you call it with the symbol sqrt and a value as a parameter). This method and other related math functions are encapsulated within the Math or math class.

We can define our own module-like classes by making them static, i.e. we could group our vector math functions into a static VectorMath class.

import java.lang.Math;

public static class VectorMath(){
    
    public static double dotProduct(Vector3 a, Vector3 b){
        return a.x * b.x + a.y * b.y + a.z * b.z;
    }
    
    public static double magnitude(Vector3 a){
        return Math.sqrt(Math.pow(a.x, 2) + Math.pow(a.y, 2) + Math.pow(a.z, 2));
    }
}

Vector3 vect1 = new Vector3(3.0, 4.0, 5.0);
Vector3 vect2 = new Vector3(6.0, 7.0, 8.0);
System.out.println(VectorMath.dotProduct(vect1, vect2));
System.out.println(VectorMath.magnitude(vect1));

import math

class VectorMath:
    
    @staticmethod
    def dot_product(a: Vector3, b: Vector3) -> float:
        return a.x * b.x + a.y * b.y + a.z * b.z
    
    @staticmethod
    def magnitude(a: Vector3) -> float:
        return math.sqrt(a.x ** 2 + a.y ** 2 + a.z ** 2)

vect1: Vector3 = Vector3(3.0, 4.0, 5.0)
vect2: Vector3 = Vector3(6.0, 7.0, 8.0)
print(VectorMath.dot_product(vect1, vect2))
print(VectorMath.magnitude(vect2))

State and Behavior

Video Materials

The data stored in a program at any given moment (in the form of variables, objects, etc.) is the state of the program. Consider a variable:

int a = 5;

The state of the variable a after this line is 5. If we then run:

a = a * 3;

The state is now 15. Consider the Vector3 struct we defined earlier. If we create an instance of that struct in the variable b:

Vector3 b = new Vector3(1.2, 3.7, 5.6);

The state of our variable b is {$1.2, 3.7, 5.6$}. If we change one of b’s fields:

b.x = 6.0;

The state of our variable b is {$6.0, 3.7, 5.6$}.

We can also think about the state of the program, which would be something like:

{$a: 5, b:${$x: 6.0, y: 3.7, z: 5.6$}}

We can therefore think of a program as a state machine. We can in fact, draw our entire program as a state table listing all possible legal states (combinations of variable values) and the transitions between those states. Techniques like this can be used to reason about our programs and even prove them correct!

This way of reasoning about programs is the heart of Automata Theory, a subject you may choose to learn more about if you pursue graduate studies in computer science.

What causes our program to transition between states? If we look at our earlier examples, it is clear that the assignment statement is a strong culprit. Expressions clearly have a role to play, as do control-flow structures, which decide which transformations take place. In fact, we can say that our program code is what drives state changes - the behavior of the program.

Thus, programs are composed of both state (the values stored in memory at a particular moment in time) and behavior (the instructions to change that state).

Now, can you imagine trying to draw the state table for a large program? Something on the order of EPIC?

On the other hand, with encapsulation we can reason about state and behavior on a much smaller scale. Consider this function working with our Vector3 struct:

public static Vector3 scale(Vector3 vec, double scale){
    double x = vec.x * scale;
    double y = vec.y * scale;
    double z = vec.z * scale;
    return new Vector3(x, y, z);
}

@staticmethod
def scale(vec: Vector3, scale: float) -> Vector3:
    x: float = vec.x * scale
    y: float = vec.y * scale
    z: float = vec.z * scale
    return Vector3(x, y, z)

If this method was invoked with a vector {$4.0, 1.0, 3.4$} and a scale $2.0$, our state table would look something like:

Because the parameters vec and scale, as well as the variables x, y, z, and the unnamed Vector3 we return are all defined only within the scope of the method, we can reason about them and the associated state changes independently of the rest of the program. This greatly simplifies both writing and debugging programs.

Classes and Objects

The module-based encapsulation suggested by Parnas and his contemporaries grouped state and behavior together into smaller, self-contained units. Alan Kay and his co-developers took this concept a step farther. Alan Kay was heavily influenced by ideas from biology, and saw this encapsulation in similar terms to cells.

¹

Biological cells are also encapsulated - the complex structures of the cell and the functions they perform are all within a cell wall. This wall is only bridged in carefully-controlled ways, i.e. cellular pumps that move resources into the cell and waste out. While single-celled organisms do exist, far more complex forms of life are made possible by many similar cells working together.

This idea became embodied in object-orientation in the form of classes and objects. An object is like a specific cell. You can create many, very similar objects that all function identically, but each have their own individual and different state. The class is therefore a definition of that type of object’s structure and behavior. It defines the shape of the object’s state, and how that state can change. But each individual instance of the class (an object) has its own current state.

Let’s re-write our Vector3 struct using this concept.

public class Vector3{
    public double x;
    public double y;
    public double z;
    
    public Vector3(double x, double y, double z){
        this.x = x;
        this.y = y;
        this.z = z;
    }
    
    public double dotProduct(Vector3 other){
        return this.x * other.x + this.y * other.y + this.z * other.z;
    }
    
    public void scale(double scalar){
        this.x *= scalar;
        this.y *= scalar;
        this.z *= scalar;
    }
}

class Vector3:
    
    def __init__(self, x: float, y: float, z: float) -> None:
        self.x = x
        self.y = y
        self.z = z
        
    def dot_product(self, other: Vector3) -> float:
        return self.x * other.x + self.y * other.y + self.z * other.z
    
    def scale(self, scalar: float) -> None:
        self.x *= scalar
        self.y *= scalar
        self.z *= scalar

Here we have defined:

1. The structure of the object state - three floating point values, x, y, and z
2. How the object is constructed - the constructor that takes in parameters to set object’s initial state
3. Instructions for how that object’s state can be changed, i.e. our scale() method

We can create as many objects from this class definition as we might want. Each one will have the same behavior but different state.

Vector3 one = new Vector3(1.0, 1.0, 1.0);
Vector3 up = new Vector3(0.0, 1.0, 0.0);
Vector3 a = new Vector3(5.4, -21.4, 3.11);

one: Vector3 = Vector3(1.0, 1.0, 1.0)
up: Vector3 = Vector3(0.0, 1.0, 0.0)
a: Vector3 = Vector3(5.4, -21.4, 3.11)

Conceptually, what we are doing is not that different from using a compound data type like a struct and a module of functions that work upon that struct. But practically, it means all the code for working with vectors appears in one place. This arguably makes it much easier to find all the pertinent parts of working with vectors, and makes the resulting code better organized and easier to maintain and add features to. This highlights why encapsulation is one of the key concepts in object-oriented programming.

Access Modifiers

Video Materials

Now let’s return to the concept of information hiding, and how it applies in object-oriented languages.

Unanticipated changes in state are a major source of errors in programs. Again, think back to the EPIC source code we looked at earlier. It may have seemed unusual now, but it used a common pattern from the early days of programming, where all the variables the program used were declared in one spot, and were global in scope (i.e. any part of the program could reassign any of those variables).

If we consider the program as a state machine, that means that any part of the program code could change any part of the program’s state. Provided those changes were intended, everything works fine. But if the wrong part of the state was changed, problems would ensue.

For example, if we were to make a typo in the part of the program dealing with water run-off in a field, which ends up assigning a new value to a variable that was supposed to be used for crop growth, we’ve just introduced a very subtle and difficult to find error. When the crop growth modeling functionality fails to work properly, we’ll probably spend serious time and effort looking for a problem in the crop growth portion of the code, but the problem doesn’t lie in that code at all!

Java, along with many other object-oriented languages, use access modifiers to implement data hiding. Consider a class representing a student:

public class Student{
    private String first;
    private String last;
    private int wid;
    
    public Student(String first, String last, int wid){
        this.first = first;
        this.last = last;
        this.wid = wid;
    }
}

class Student:
    
    def __init__(self, first: str, last: str, wid: int) -> None:
        self.__first = first
        self.__last = last
        self.__wid = wid

By using the access modifier private in Java, or prefixing the attributes with two underscores in Python, we have indicated that our fields first, last, and wid cannot be accessed (seen or assigned) outside of this code. For example, if we were to create a specific student:

Student willie = new Student("Willie", "Wildcat", 888888888);

willie: Student = Student("Willie", "Wildcat", 888888888)

We would not be able to change that student’s name. The statement willie.first = "Bob" would fail, because the field first is private. In fact, we cannot even see his name, so trying to print that value would also fail.

If we want to allow a field or method to be accessible outside of the object, we must declare it public in Java, or remove the underscores in Python. While we can declare fields public, this violates the core principles of encapsulation, as any outside code can modify our object’s state in uncontrolled ways. This is definitely not what we want.

Instead, in a true object-oriented approach we would write public accessor methods, a.k.a. getters and setters. These are methods that allow us to see and change field values in a controlled way. Adding accessors to our Student class might look like:

public class Student{
    private String first;
    private String last;
    private int wid;
    
    public Student(String first, String last, int wid){
        this.first = first;
        this.last = last;
        this.wid = wid;
    }
    
    public String getFirst(){
        return this.first;
    }
    
    public void setFirst(String value){
        if(value.length() > 0){
            this.first = value;
        }
    }
    
    public String getLast(){
        return this.last;
    }
    
    public void setLast(String value){
        if(value.length() > 0){
            this.last = value;
        }
    }
    
    public int getWid(){
        return this.wid;
    }
}

class Student:
    
    def __init__(self, first: str, last: str, wid: int) -> None:
        self.__first = first
        self.__last = last
        self.__wid = wid
        
    @property
    def first(self) -> str:
        return self.__first
    @first.setter
    def first(self, value: str) -> None:
        if len(value) > 0:
            self.__first = value
    
    @property
    def last(self) -> str:
        return self.__last
    @last.setter
    def last(self, value: str) -> None:
        if len(value) > 0:
            self.__last = value
            
    @property
    def wid(self) -> int:
        return self.__wid

Notice how the setFirst() and setLast() setters in Java, and the first() and last() setters in Python, check that the provided name has at least one character? We can use setters to make sure that we never allow the object state to be set to something that makes no sense.

Also, notice that the wid field only has a getter. This effectively means once a student’s wid is set by the constructor, it cannot be changed (it’s read only). This allows us to share data without allowing it to be changed outside of the class.

willie.setFirst("William");
System.out.println(willie.getFirst());

willie.first = "William"
print(willie.first)

Objects in Memory

We often talk about the class as a blueprint for an object. This is because classes define what properties and methods an object should have, in the form of a constructor. Consider this class representing a planet:

public class Planet{
    
    private double mass;
    public double getMass(){
        return this.mass;
    }
    
    private double radius;
    public double getRadius(){
        return this.radius;
    }
    
    public Planet(double mass, double radius){
        this.mass = mass;
        this.radius = radius;
    }
}

class Planet

    @property
    def mass(self) -> float:
        return self.__mass
    
    @property
    def radius(self) -> float:
        return self.__radius
    
    def __init__(self, mass: float, radius: float) -> None:
        self.__mass = mass
        self.__radius = radius

It describes a planet as having a mass and a radius, which will be stored as the ratio of this planet’s attribute compared to Earth. We can create a specific planet by invoking its constructor, i.e.:

Planet earth = new Planet(1.0, 1.0);

earth: Planet = Planet(1.0, 1.0)

In this example, earth is an instance of the class Planet. We can create other instances, i.e.

Planet mars = new Planet(0.107, 0.53);

mars: Planet = Planet(0.107, 0.53)

We can even create a Planet instance to represent one of the exoplanets discovered by NASA’s TESS:

Planet hd21749b = new Planet(23.20, 2.836);

hd21749b: Planet = Planet(23.20, 2.836)

Let’s think more deeply about the idea of a class as a blueprint. A blueprint for what, exactly? For one thing, it serves to describe the state of the object, and helps us label that state. If we were to check the radius of our variable mars, we would access the getter for the radius field:

mars.getRadius()

mars.radius

But a class does more than just labeling the properties and fields and providing methods to mutate the state they contain. It also specifies how memory needs to be allocated to hold those values as the program runs.

Looking at our Planet class again, we can see it contains two floating point values. So, when we run the constructor for that class, our computer will know that it needs to allocate enough space in memory for those two values (8 bytes each in Java, and 24 bytes each in Python).

State and memory are clearly related - the current state is what data is stored in memory. It is possible to take that memory’s current state, write it to persistent storage (like the hard drive), and then read it back out at a later point in time and restore the program to exactly the state we left it with. This is actually what your operating system does when you put it into hibernation mode.

The process of writing out the state is known as serialization, and it’s a topic we’ll revisit later.

Static Modifier

You might have wondered how the static modifier plays into objects. Essentially, the static keyword indicates the field or method it modifies exists in only one memory location. I.e. a static field references the same memory location for all objects that possess it.

Consider this simple example class:

public class Simple:
    public static int x;
    public int y;
    
    public Simple(int x, int y){
        this.x = x;
        this.y = y;
    }
}

class Simple:
    
    x: int = 0
        
    def __init__(self, x: int, y: int) -> None:
        Simple.x = x
        self.y = y

Notice that the Java language uses the static keyword for fields, whereas in Python the field is simply defined outside of the constructor, and only attached to the class name and not self.

We can also create a couple of instances:

Simple first = new Simple(10, 12);
Simple second = new Simple(8, 5);

first: Simple = Simple(10, 12)
second: Simple = Simple(8, 5)

Once we’ve created both instances, the value of first.x would be 8 - because first.x and second.x reference the same memory location (a static unchanging location), and second.x was set after first.x. If we changed it again:

first.x = 3

Then both first.x and second.x would have the value 3, as they share the same memory location. first.y would still be 12, and second.y would still be 5.

Another way to think about static is that it means the field or method we are modifying belongs to the class and not the individual object. Hence, each object shares a static variable, because it belongs to their class.

Used on a method, the static keyword in Java or the @staticmethod decorator in Python indicates that the method belongs to the class definition, not the object instance. Hence, we must invoke it from the class, not an object instance: i.e. Math.pow().

Finally, when used with a class in Java, static indicates we can’t create objects from the class - the class definition exists on its own. Hence, the Math m = new Math(); is actually an error, as the Math class is declared static. Python does not directly support the static keyword for classes themselves, but classes which only contain static attributes and methods could be considered static classes.

Message Passing

Video Materials

The second criteria Alan Kay set for object-oriented languages was message passing. Message passing is a way to request a unit of code engage in a behavior, i.e. changing its state, or sharing some aspect of its state.

Consider the real-world analogue of a letter sent via the postal service. Such a message consists of: an address the message needs to be sent to, a return address, the message itself (the letter), and any data that needs to accompany the letter (the enclosures). A specific letter might be a wedding invitation. The message includes the details of the wedding (the host, the location, the time), an enclosure might be a refrigerator magnet with these details duplicated. The recipient should (per custom) send a response to the host addressed to the return address letting them know if they will be attending.

In an object-oriented language, message passing primarily take the form of methods. Let’s revisit our example Vector3 class from earlier:

public class Vector3{
    public double x;
    public double y;
    public double z;
    
    public Vector3(double x, double y, double z){
        this.x = x;
        this.y = y;
        this.z = z;
    }
    
    public double dotProduct(Vector3 other){
        return this.x * other.x + this.y * other.y + this.z * other.z;
    }
    
    public void scale(double scalar){
        this.x *= scalar;
        this.y *= scalar;
        this.z *= scalar;
    }
}

class Vector3:
    
    def __init__(self, x: float, y: float, z: float) -> None:
        self.x = x
        self.y = y
        self.z = z
        
    def dot_product(self, other: Vector3) -> float:
        return self.x * other.x + self.y * other.y + self.z * other.z
    
    def scale(self, scalar: float) -> None:
        self.x *= scalar
        self.y *= scalar
        self.z *= scalar

We can also create a couple of instances of the class, and use its dot product method:

Vector3 a = new Vector3(1.0, 1.0, 2.0);
Vector3 b = new Vector3(4.0, 2.0, 1.0);
double c = a.dotProduct(b);

a: Vector3 = Vector3(1.0, 1.0, 2.0)
b: Vector3 = Vector3(4.0, 2.0, 1.0)
c: float = a.dot_product(b)

Consider the invocation of a.dotProduct(b) (Java) or a.dot_product(b) (Python) above. The method name, dotProduct or dot_product provides the details of what the message is intended to accomplish (the letter). Invoking it on a specific variable, i.e. a, tells us who the message is being sent to (the recipient address). The return type indicates what we need to send back to the recipient (the invoking code), and the parameters provide any data needed by the class to address the task (the enclosures).

Let’s define a new method for our Vector3 class that emphasizes the role message passing plays in mutating object state:

public void normalize(){
    double magnitude = Math.sqrt(Math.pow(this.x, 2) + Math.pow(this.y, 2) + Math.pow(this.z, 2));
    this.x /= magnitude;
    this.y /= magnitude;
    this.z /= magnitude;
}

def normalize(self) -> None:
    magnitude: float = math.sqrt(self.x ** 2 + self.y ** 2 + self.z ** 2)
    self.x /= magnitude
    self.y /= magnitude
    self.z /= magnitude

We can now invoke the normalize() method on a Vector3 object to mutate its state, shortening the magnitude of the vector to length 1.

Vector3 f = new Vector3(9.0, 3.0, 2.0);
f.normalize();

f: Vector3 = Vector3(9.0, 3.0, 2.0)
f.normalize()

Note how here, f is the object receiving the message normalize. There is no additional data needed, so there are no parameters being passed in. Our earlier dot product method took a second vector as its argument, and used that vector’s values to mutate its state.

Message passing therefore acts like those special molecular pumps and other gate mechanisms of a cell that control what crosses the cell wall. The methods defined on a class determine how outside code can interact with the object. An extra benefit of this approach is that a method becomes an abstraction for the behavior of the code, and the associated state changes it embodies. As a programmer using the method, we don’t need to know the exact implementation of that behavior - just what data we need to provide, and what it should return or how it will alter the program state. This makes it far easier to reason about our program, and also means we can change the internal details of a class (perhaps to make it run faster) without impacting the other aspects of the program.

Summary

In this chapter, we looked at how object-orientation adopted the concept of encapsulation to combine related state and behavior within a single unit of code, known as a class. We further explored how objects are instances of a class created through invoking a constructor method.

We also discussed several different ways of looking at and reasoning about objects - as a state machine, and as structured data stored in memory. We discussed how a method is really a form of message passing that provides an interface to interact with objects safely.

Finally, we explored how all of these concepts are implemented in both the Java and Python programming languages.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.

Introduction

One of the strategies for combating the challenges of the software crisis is writing clear documentation to support both the end-users who will use the program, as well as other developers who will update and maintain the code. Today, including high-quality documentation along with your code, both in the form of code comments and other external documentation, is seen as an important practice among software developers, especially those working on large projects with multiple developers.

In this chapter, we’ll learn about these terms:

• User Documentation
• Developer Documentation
• HTML
• Markdown
• XML
• Code Comments
• Javadoc
• Python Docstrings
• Generated Documentation

After this chapter and the associated example project, we should be able to write effective documentation within our code using the correct format for our chosen programming language.

Documentation Types

Documentation refers to the written materials that accompany program code. Documentation plays multiple, and often critical roles. Broadly speaking, we split documentation into two categories based on the intended audience:

• User Documentation is meant for the end-users of the software
• Developer Documentation is meant for the developers of the software

As you might expect, the goals for these two styles of documentation are very different. User documentation instructs the user on how to use the software. Developer documentation helps orient the developer so that they can effectively create, maintain, and expand the software.

Historically, documentation was printed separately from the software. This was largely due to the limited memory available on most systems. For example, the EPIC software we discussed had two publications associated with it: a User Manual, which explains how to use it, and Model Documentation which presents the mathematic models that programmers adapted to create the software. There are a few very obvious downsides to printed manuals: they take substantial resources to produce and update, and they are easily misplaced.

User Documentation

As memory became more accessible, it became commonplace to provide digital documentation to the users. For example, with Unix (and Linux) systems, it became commonplace to distribute digital documentation alongside the software it documented. This documentation came to be known as man pages based on the man command (short for manual) that would open the documentation for reading. For example, to learn more about the Linux search tool grep, you would type the following command into a Linux terminal:

man grep 

That would open the documentation distributed with the grep tool. Man pages are written in a specific format; you can read more about it here.

While man pages are a staple of the Unix/Linux operating system, there was no equivalent in the DOS ecosystem (the foundations of Windows) until PowerShell was released in 2007, including the Get-Help tool. You can read more about it here.

However, once software began to be written with graphical user interfaces (GUIs), it became commonplace to incorporate the user documentation directly into the GUI, usually under a “Help” menu. This served a similar purpose to man pages by ensuring user documentation was always available with the software. Of course, one of the core goals of software design is to make the software so intuitive that users don’t need to reference the documentation. It is equally clear that developers often fall short of that mark, as there is a thriving market for books to teach certain software.

¹

Of course, there are also thousands of YouTube channels devoted to teaching users how to use specific programs!

Developer Documentation

Developer documentation underwent a similar transformation. Early developer documentation was often printed and placed in a three-ring binder, as Neal Stephenson describes in his novel Snow Crash: ²

Fisheye has taken what appears to be an instruction manual from the heavy black suitcase. It is a miniature three-ring binder with pages of laser-printed text. The binder is just a cheap unmarked one bought from a stationery store. In these respects, it is perfectly familiar to Him: it bears the earmarks of a high-tech product that is still under development. All technical devices require documentation of a sort, but this stuff can only be written by the techies who are doing the actual product development, and they absolutely hate it, always put the dox question off to the very last minute. Then they type up some material on a word processor, run it off on the laser printer, send the departmental secretary out for a cheap binder, and that's that.

Shortly after the time this novel was written, the Internet became available to the general public, and the tools it spawned would change how software was documented forever. Increasingly, web-based tools are used to create and distribute developer documentation. Wikis, bug trackers, and autodocumentation tools quickly replaced the use of lengthy, and infrequently updated, word processor files.

Documentation Formats

Video Materials

Developer documentation often faces a challenge not present in other kinds of documents - the need to be able to display snippets of code. Ideally, we want code to be formatted in a way that preserves indentation. We also don’t want code snippets to be subject to spelling and grammar checks, especially auto-correct versions of these algorithms, as they will alter the snippets. Ideally, we might also apply syntax highlighting to these snippets. Accordingly, a number of textual formats have been developed to support writing text with embedded program code, and these are regularly used to present developer documentation. Let’s take a look at several of the most common.

HTML

Since its inception, HTML has been uniquely suited for developer documentation. It requires nothing more than a browser to view - a tool that nearly every computer is equipped with (in fact, most have two or three installed). And the <code> element provides a way of styling code snippets to appear differently from the embedded text, and <pre> can be used to preserve the snippet’s formatting. Thus:

<p>This algorithm reverses the contents of the array, <code>nums</code></p>
<pre><code>for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}
</code></pre>

Will render in a browser as:

for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}

JavaScript and CSS libraries like highlight.js, prism, and others can provide syntax highlighting functionality without much extra work.

Of course, one of the strongest benefits of HTML is the ability to create hyperlinks between pages. This can be invaluable in documenting software, where the documentation about a particular method could include links to documentation about the classes being supplied as parameters, or being returned from the method. This allows developers to quickly navigate and find the information they need as they work with your code.

Markdown

However, there is a significant amount of boilerplate involved in writing a webpage (i.e. each page needs a minimum of elements not specific to the documentation to set up the structure of the page). The extensive use of HTML elements also makes it more time-consuming to write and harder for people to read in its raw form. Markdown is a markup language developed to counter these issues. Markdown is written as plain text, with a few special formatting annotations, which indicate how it should be transformed to HTML. Some of the most common annotations are:

• Starting a line with hash (#) indicates it should be a <h1> element, two hashes (##) indicates a <h2>, and so on…
• Wrapping a statement with underscores (_) or asterisks (*) indicates it should be wrapped in a <i> element
• Wrapping a statement with double underscores (__) or double asterisks (**) indicates it should be wrapped in a <b> element
• Links can be written as [link text](url), which is transformed to <a href="url">link text</a>
• Images can be written as ![alt text](url), which is transformed to <img alt="alt text" src="url"/>

Code snippets are indicated with backtick marks (`). Inline code is written surrounded with single backtick marks, i.e. `int a = 1` and in the generated HTML is wrapped in a <code> element. Code blocks are wrapped in triple backtick marks, and in the generated HTML are enclosed in both <pre> and <code> elements. Thus, to generate the above HTML example, we would use:

This algorithm reverses the contents of the array, `nums`
```
for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}
```

Most markdown compilers also support specifying the language (for language-specific syntax highlighting) by following the first three backticks with the language name, i.e.:

```java
String aString = "abc123";
```

Nearly every programming language features at least one open-source library for converting Markdown to HTML. In addition to being faster to write than HTML, and avoiding the necessity to write boilerplate code, Markdown offers some security benefits. Because it generates only a limited set of HTML elements, which specifically excludes some most commonly employed in web-based exploits (like using <script> elements for script injection attacks), it is often safer to allow users to contribute markdown-based content than HTML-based content. Note: this protection is dependent on the settings provided to your HTML generator - most markdown converters can be configured to allow or escape HTML elements in the markdown text.

In fact, both the Codio guides in this course, as well as the website used to store the project milestones, was written using Markdown. Codio includes its own Markdown converter, whereas the website was converted to HTML using the Hugo framework, a static website generator built using the Go programming language.

Additionally, chat servers like RocketChat and Discord support using markdown in posts! Try it out sometime!

GitHub even incorporates a markdown compiler into its repository displays. If your file ends in a .md extension, GitHub will evaluate it as Markdown and display it as HTML when you navigate your repository. If your repository contains a README.md file at the top level of your project, it will also be displayed as the front page of your repository. GitHub uses an expanded list of annotations known as GitHub-flavored markdown that adds support for tables, task item lists, strikethroughs, and others. You can also use Markdown in GitHub pull requests, comments, and more!

XML

Extensible Markup Language (XML) is a close relative of HTML - they share the same ancestor, Standard Generalized Markup Language (SGML). It allows developers to develop their own custom markup languages based on the XML approach, i.e. the use of elements expressed via tags and attributes. XML-based languages are usually used as a data serialization format. For example, this snippet represents a serialized fictional student:

<student>
    <firstName>Willie</firstName>
    <lastName>Wildcat</lastName>
    <wid>8888888</wid>
    <degreeProgram>BCS</degreeProgram>
</student>

While XML is most known for representing data, it can also be used to create documentation, most notably in the Microsoft .NET ecosystem.

Code Comments

Of course, one of the most important ways that developers can add documentation to their software is through the use of code comments. A code comment is simply extra text added to the source code of a program which is ignored by the compiler or interpreter - it is only visible within the source code itself. Nearly every programming language supports the inclusion of code comments to help describe or explain how the code works, and it is a vital way for developers to make notes, share information, and make sure anyone else reading the code can truly understand what it does.

Writing Useful Comments

Unfortunately, there is not a well established rule for what constitutes a useful code comment, or even how many comments should be included in code. Various developers have proposed ideas such as Literate Programming, which involves writing complete explanations of the program’s logic, all the way down to Self-Documenting Code, which proposes the idea that using properly named variables and well structured code will eliminate the need for any documentation at all, and everything in between. There are numerous articles and books written about how to document code properly that can be found through a simple online search.

For the purposes of this course, we recommend writing useful code comments anytime the code contains something interesting or unique, or something that required a bit of thinking and effort to create or understand. In that way, the next time a developer looks at the code, we can reduce the amount of time that developer spends trying to understand what the code is doing.

In short, we should write comments that help us understand our code better, but we shouldn’t focus on commenting every single line or expression, especially when it is pretty obvious what it does. To help with that, we can use properly named variables that accurately describe the data being manipulated, and use simple expressions that are easy to follow instead of complex ones.

Comment Formats

Each programming language defines its own specification for comments. Here is the basic information for both Java and Python.

// Single line Java comments are prefixed by two slashes.

int x = 5; // Comments can be placed at the end of a line.

/*
 * This is an example of a block comment.
 *
 * It begins with a slash and an asterisk, and ends
 * with an asterisk and a slash.
 *
 * By convention, each line is prefixed with an asterisk
 * that is aligned with the starting asterisk, but this is not
 * strictly required.
 */
 
/**
 * This is an example of a documentation comment.
 *
 * It begins with a slash and a two asterisks, and ends
 * with an asterisk and a slash.
 *
 * By convention, each line is prefixed with an asterisk
 * that is aligned with the starting asterisk, but this is not
 * strictly required.
 *
 * These blocks are processed by Javadoc to create documentation.
 */

# Single line Python comments are prefixed by a hash symbol

x = 5 # comments can be placed at the end of a line

""" Python does not directly support block comments.

However, a bare string literal, surrounded by three double-quotes
can be used to create a longer comment. 

Python refers to these comments as docstrings when used
to document elements such as functions or classes
"""

Formal Code Documentation

In addition to comments within the source code, we can also include formal documentation comments about classes and methods in our code. These comments help describe the functionality of parts of our code, and can be parsed to create generated documentation. On the next two pages, we’ll introduce the documentation standard for both Java and Python. Feel free to only read about the language you are learning, but it might be interesting to see how other languages handle the same idea in different ways.

Javadoc

Video Materials

The Java software development kit (SDK) includes a tool called Javadoc, which can create documentation based on the documentation comments included in the code. Both the Javadoc Documentation and the Google Style Guide include information about how those documentation comments should be structured and the information each should contain. This page will serve as a quick guide for the most common use cases, but you may wish to refer to the documentation linked above for more specific examples and information. The Checkstyle tool is also a great way to check that the documentation comments are properly structured.

General Structure

A properly structured Javadoc comment includes a few parts:

1. A summary fragment. This is the first part of the comment, ending with the first period. It should concisely describe the object being commented, but doesn’t have to be a complete sentence.
2. Additional Paragraphs. Following the summary fragment, additional paragraphs may be included to further describe the object. The paragraphs should start with the <p> tag. However, unlike HTML, notice that there is no matching </p> closing tag required.
3. Tags. Javadoc supports many tags. Here are the most common tags, listed in the order in which they should appear:
   • @author (classes and interfaces only)
   • @version (classes and interfaces only)
   • @param (methods and constructors only)
   • @return (methods only)
   • @throws
   • @see

When including multiple @author, @param or @throws tags, there are some rules governing the ordering of the tags as well. You can find much more information about the tags and how they can be used in the Javadoc Documentation.

Class Comment

Let’s begin by looking at the Javadoc comment for a class. Here’s an example:

/**
 * Represents a chessboard and moves chess pieces.
 *
 * <p>This class stores a chessboard in a 2D array and includes
 * methods to move various chess pieces across the board. Squares
 * are labelled using algebraic chess notation.
 *
 * @author Russell Feldhausen russfeld@ksu.edu
 * @version 0.1
 */
public class Chessboard {

This comment includes a summary fragment, and additional paragraph, and the two required tags for a class comment, @author and @version. At a minimum, each class we develop should include this information directly above the class declaration.

This comment provides enough information for us to understand what the class is used for and a bit about how it works, even without seeing the code.

Method Comment

Here’s another example Javadoc comment, this time for a method:

/**
 * Moves a knight from one square to another
 *
 * <p>If a knight is present on <code>source</code> and 
 * can make a legal move to <code>destination</code>, the method 
 * will perform the move. 
 *
 * @param source       the source square in algebraic chess notation
 * @param destination  the destination square in algebraic chess notation
 * @return             <code>true</code> if a piece was captured; 
 *                     <code>false</code> otherwise
 * @throws IllegalArgumentException     if a knight is not present on 
 *                                      <code>source</code> or if that knight 
 *                                      cannot move to <code>destination</code>
 */
public boolean moveKnight(String source, String destination) {

Similar to the comment above, this comment includes enough information for us to understand exactly what the method does. It tells us about the parameters it accepts and the format it expects, the return value, and any exceptions that could be thrown by this code. With this comment alone, we could probably write the code for the method itself!

Other Comments

The two examples above cover most places where we would use Javadoc comments in our code. The only other example would be for any public attributes of a class, as in this example:

/** The Student's Wildcat ID */
public int wid;

However, as we discussed in a previous module, if we follow the concepts of encapsulation and information hiding we shouldn’t have any publicly-accessible attributes, only public accessor methods such as getters and setters, which can be documented as methods. So, we probably won’t end up using this much in our own code.

Python Docstrings

Video Materials

Many Python developers have standardized on the use of docstrings as documentation comments. Both PEP 257 and the Google Style Guide include information about how those documentation comments should be structured and the information each should contain. This page will serve as a quick guide for the most common use cases, but you may wish to refer to the documentation linked above for more specific examples and information. The flake8 tool along with the flake8-docstrings plugin is also a great way to check that the documentation comments are properly structured.

General Structure

A properly structured docstring comment includes a few parts:

1. A summary line. This is the first part of the comment, ending with the first period. It should concisely describe the object being commented, but doesn’t have to be a complete sentence.
2. Additional Paragraphs. Following the summary fragment, additional paragraphs may be included to further describe the object. The paragraphs should start at the same indentation as the first quotation mark. Paragraphs are separated by blank lines.
3. Optional Sections. While not explicitly required by the standard, there are several optional sections that could be included as part of a docstring. For this course, we’ll use the following sections:
   • Author (files only)
   • Version (files only)
   • Attributes (classes with public attributes only)
   • Args (methods and constructors only)
   • Returns (methods only)
   • Raises

You can find more information about the structure of docstrings in the Google Style Guide.

File Comment

Let’s begin by looking at the docstring comment for a file. Here’s an example:

"""Implements a simple chessboard.

This file contains a class to represent a chessboard.

Author: Russell Feldhausen russfeld@ksu.edu
Version: 0.1
"""

The file docstring gives information about the contents of the file. For object-oriented programs where each file contains a single class, this can be a bit redundant, but it is useful information nonetheless. For other Python files, this may be the only comment included in the file.

While the Python documentation format does not require listing the author or the version, it is a nice convention from the Javadoc format that we can carry over into our Python docstrings as well.

Class Comment

Next, let’s look at the docstring comment for a class. Here’s an example:

class Chessboard:
    """Represents a chessboard and moves chess pieces.
    
    This class stores a chessboard in a 2D array and includes
    methods to move various chess pieces across the board. Squares
    are labelled using algebraic chess notation.
    """

This comment includes a summary fragment, and an additional paragraph. Since the class doesn’t include any public attributes, we omit that section. Instead, we’ll document the accessor methods, or getters and setters, as part of the Python property that is used to access or modify private attributes.

This comment provides enough information for us to understand what the class is used for and a bit about how it works, even without seeing the code.

Method Comment

Here’s another example docstring comment, this time for a method:

def move_knight(self, source: str, destination: str) -> bool:
    """Moves a knight from one square to another
    
    If a knight is present on source and 
    can make a legal move to destination, the method 
    will perform the move. 
    
    Args:
        source: the source square in algebraic chess notation
        destination: the destination square in algebraic chess notation
        
    Returns:
        True if a piece was captured; False otherwise
 
    Raises:
        ValueError: if a knight is not present on source or 
          if that knight cannot move to destination
    """

Similar to the comment above, this comment includes enough information for us to understand exactly what the method does. It tells us about the parameters it accepts and the format it expects, the return value, and any exceptions that could be thrown by this code. With this comment alone, we could probably write the code for the method itself!

Generated Documentation

One of the biggest innovations in documenting software was the development of documentation generation tools. These were programs that would read source code files, and combine information parsed from the code itself and information contained in code comments to generate documentation in an easy-to-distribute form (often HTML).

This approach meant that the language of the documentation was embedded within the source code itself, making it far easier to update the documentation as the source code was refactored. Then, every time a release of the software was built, the documentation could be regenerated from the updated comments and source code. This made it far more likely developer documentation would be kept up-to-date.

So, once we have properly documented our code using documentation comments, we can then use tools such as Javadoc for Java or pdoc3 for Python to automatically generate documentation for developers. That documentation contains all of the contents of our documentation comments, and serves as a handy reference for any developers who wish to use our code.

In the Java ecosystem, this is best represented by the Java API itself, which is generated using Javadoc directly from the source code of the Java SDK itself.

For Python, there are many documentation generators available, but we’ve chosen to use pdoc3. An example of its output is the pdoc3 Documentation.

In either case, the use of these tools, combined with up to date documentation comments in our code, means that we can easily generate documentation quickly and easily.

Summary

In this chapter, we examined the need for software documentation aimed at both end-users and developers (user documentation and developer documentation, respectively). We also examined some formats this documentation can be presented in: HTML, Markdown, and XML. We also discussed documentation generation tools, which generate developer documentation from specially-formatted comments in our code files.

We examined the both the Java and Python approach to documentation comments, helping other developers understand our code. For this reason, as well as the ability to produce HTML-based documentation using a documentation generator tool, it is best practice to use documentation comments in all our programs.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.

Introduction

A critical part of the software development process is ensuring the software works! We mentioned earlier that it is possible to logically prove that software works by constructing a state transition table for the program, but once a program reaches a certain size, this strategy becomes less feasible. Similarly, it is possible to model a program mathematically and construct a theorem that proves it will perform as intended. But in practice, most software is validated through some form of testing. This chapter will discuss the process of testing object-oriented systems.

Key Terms

Some key terms to learn in this chapter are:

• Informal Testing
• Formal Testing
• Test Plan
• Test Framework
• Automated Testing
• Assertions
• Unit Tests
• Testing Code Coverage
• Regression Testing

Key Skills

The key skill to learn in this chapter is how to write unit tests in our chosen language. For Java, we’ll be using JUnit 5 to write our tests, and in Python we’ll use pytest as our test framework. We will also explore using the Hamcrest assertion library for both Java and Python.

Manual Testing

Video Materials

As you’ve developed programs, you’ve probably run them, supplied input, and observed if what happened was what you wanted. This process is known as informal testing. It’s informal, because you don’t have a set procedure you follow, i.e. what specific inputs to use, and what results to expect. Formal testing adds that structure. In a formal test, you would have a written procedure to follow, which specifies exactly what inputs to supply, and what results should be expected. This written procedure is known as a test plan.

Historically, the test plan was often developed at the same time as the design for the software (but before the actual programming). The programmers would then build the software to match the design, and the completed software and the test plan would be passed onto a testing team that would follow the step-by-step testing procedures laid out in the testing plan. When a test failed, they would make a detailed record of the failure, and the software would be sent back to the programmers to fix.

This model of software development has often been referred to as the “waterfall model” as each task depends on the one before it:

¹

Unfortunately, as this model is often implemented, the programmers responsible for writing the software are reassigned to other projects as the software moves into the testing phase. Rather than employ valuable programmers as testers, most companies will hire less expensive workers to carry out the testing. So either a skeleton crew of programmers is left to fix any errors that are found during the tests, or these are passed back to programmers already deeply involved in a new project.

The costs involved in fixing software errors also grow larger the longer the error exists in the software. The table below comes from a NASA report of software error costs throughout the project life cycle:

²

It is clear from the graph and the paper that the cost to fix a software error grows exponentially if the fix is delayed. You probably have instances in your own experience that also speak to this - have you ever had a bug in a program you didn’t realize was there until your project was nearly complete? How hard was it to fix, compared to a error you found and fixed right away?

It was realizations like these, along with growing computing power, that led to the development of automated testing, which we’ll discuss next.

Automated Testing

Automated testing is the practice of using a program to test another program. Much as a compiler is a program that translates a program from a higher-order language into a lower-level form, a test program executes a test plan against the program being tested. And much like you must supply the program to be compiled, for automated testing you must supply the tests that need to be executed. In many ways, the process of writing automated tests is like writing a manual test plan - you are writing instructions of what to try, and what the results should be. The difference is with a manual test plan, you are writing these instructions for a human. With an automated test plan, you are writing them for a program.

Automated tests are typically categorized as unit, integration, and system tests:

• Unit tests focus on a single unit of code, and test it in isolation from other parts of the code. In object-oriented programs where code is grouped into objects, these are the units that are tested. Thus, for each class you would have a corresponding file of unit tests.
• Integration tests focus on the interaction of units working together, and with infrastructure external to the program (i.e. databases, other programs, etc).
• System tests look at the entire program’s behavior.

The complexity of writing tests scales with each of these categories. Emphasis is usually put on writing unit tests, especially as the classes they test are written. By testing these classes early, errors can be located and fixed quickly.

Unit Tests

In this course, we’ll focus on the creation of unit tests to effectively test the software we create. At a minimum, our goal is to write enough tests to achieve a high level of code coverage of our program being tested. Recall that code coverage is a measure of the amount of code in a program that is executed by a set of unit tests.

In theory, a good set of unit tests should, at a minimum, execute every line of code in the program at least once. Of course, that doesn’t nearly guarantee that the unit tests are sufficient to find all bugs, or even a majority of bugs, but it is a great place to start and make sure that the unit tests are properly testing the entirety of the program.

On the next few pages, we’ll discuss how to write unit tests for programs written in both Java and Python. Feel free to only read about the language you are learning, but it might be interesting to see how other languages handle the same idea in different ways.

Writing JUnit Tests

Video Materials

Writing tests is in many ways just as challenging and creative an endeavor as writing programs. Tests usually consist of invoking some portion of program code, and then using assertions to determine that the actual results match the expected results. The result of these assertions are typically reported on a per-test basis, which makes it easy to see where your program is not behaving as expected.

Consider a class that is a software control system for a kitchen stove. We won’t write the code for the class itself, because it is important for us to be able to write tests that effectively test the code without even seeing it. It might have properties for four burners, which correspond to what heat output they are currently set to. Let’s assume this is as an integer between 0 (off) and 5 (high). When we first construct this class, we’d probably expect them all to be off! A test to verify that expectation would be:

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;

public class StoveTest{
    
    @Test
    public void testBurnersShouldBeOffAtInitialization(){
        Stove stove = new Stove();
        assertEquals(0, stove.getBurnerOne(), "Burner is not off after initialization");
        assertEquals(0, stove.getBurnerTwo(), "Burner is not off after initialization");
        assertEquals(0, stove.getBurnerThree(), "Burner is not off after initialization");
        assertEquals(0, stove.getBurnerFour(), "Burner is not off after initialization");
    }
}

Here we’ve written the test using the JUnit 5 test framework, which is one of the most commonly used Java unit testing frameworks today.

Notice that the test is simply a method, defined in a class. This is very common for test frameworks, which tend to be written using the same programming language the programs they test are written in (which makes it easier for one programmer to write both the code unit and the code to test it). Above the test method is a method annotation @Test that tells JUnit to use this method as a unit test. Omitting the @Test annotation allows us to build other helper methods within our test classes as needed. Annotations are a way of supplying metadata within Java code. This metadata can be used by the compiler and other programs to determine how it works with your code. In this case, it indicates to the JUnit test runner that this method is a test.

Inside the method, we create an instance of stove, and then use the assertEquals(actual, expected, message) method to determine that the actual and expected values match. If they do, the assertion is marked as passing, and the test runner will display this pass. If it fails, the test runner will report the failure, along with details to help find and fix the problem (what value was expected, what it actually was, and which test contained the assertion).

dependencies {
    // Use JUnit Jupiter API for testing.
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.2', 'org.hamcrest:hamcrest:2.2', 'org.junit.jupiter:junit-jupiter-params'

    // Use JUnit Jupiter Engine for testing.
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine'
    
    // This dependency is used by the application.
    implementation 'com.google.guava:guava:29.0-jre'
}

The JUnit framework provides for two kinds of tests, Test, which are written as functions that have no parameters, and ParameterizedTest, which do have parameters. The values for these parameters are supplied with another annotation, typically @ValueSource. For example, we might test that when we set a burner to a setting within the valid 0-5 range, it is set to that value:

import static org.junit.jupiter.api.Assertions.assertEquals;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

public class StoveTest{
    
    @ParameterizedTest
    @ValueSource(ints = {0, 1, 2, 3, 4, 5})
    public void ShouldBeAbleToSetBurnerOneToValidRange(int setting){
        Stove stove = new Stove();
        stove.setBurnerOne(setting);
        assertEquals(setting, stove.getBurnerOne(), "Burner does not have expected value");
    }
}

The values in the parentheses of the @ValueSource annotation are the values supplied to the parameter list of the parameterized test method. Thus, this test is actually six tests; each test makes sure that one of the settings is working. We could have done all six as separate assignments and assertions within a single test method, but using a parameterized test means that if only one of these settings doesn’t work, we will see that one test fail while the others pass. This level of specificity can be very helpful in finding errors.

So far our tests cover the expected behavior of our stove. But where tests really prove their worth is with the edge cases - those things we as programmers don’t anticipate. For example, what happens if we try setting our range to a setting above 5? Should it simply clamp at 5? Should it not change from its current setting? Or should it shut itself off entirely because its user is clearly a pyromaniac bent on burning down their house? If the specification for our program doesn’t say, it is up to us to decide. Let’s say we expect it to be clamped at 5:

@ParameterizedTest
@ValueSource(ints = {6, 18, 1000000})
public void BurnerOneShouldNotExceedFive(int setting){
    Stove stove = new Stove();
    stove.setBurnerOne(setting);
    assertEquals(5, stove.getBurnerOne(), "Burner does not have expected value");
}

Note that we don’t need to exhaustively test all numbers above 5 - it is sufficient to provide a representative sample, ideally the first value past 5 (6), and a few others. Also, now that we have defined our expected behavior, we should make sure the documentation of our BurnerOne property matches it:

/**
 * Sets the value of Burner One.
 *
 * Should be an integer between 0 (off) and 5 (high)
 * If a value higher than 5 is provided, the burner will be 
 * set to 5 instead.
 *
 * @param value        the value of the burner
 */
public void setBurnerOne(int value){

This way, other programmers (and ourselves, if we visit this code years later) will know what the expected behavior is. We’d also want to test the other edge cases: i.e. when the burner is set to a negative number.

For a complete guide to parameterized tests in JUnit, including how to use enumerations as a value source, refer to the Guide to JUnit 5 Parameterized Tests from Baeldung.

Java Assertions

Like most testing frameworks, the JUnit framework provides a host of specialized assertions. They are all created as static methods within the Assertions class, and many of them are described in the JUnit 5 User Guide.

Boolean Assertions

For example, JUnit provides two boolean assertions:

• assertTrue(condition) - asserts that the value supplied is true
• assertFalse(condition) - asserts that the value supplied is false

As with any assertion statements in JUnit, we can also optionally supply a message string as an additional parameter to these assertion statements. That message will be present in the error message when this assertion fails.

Equality Assertions

The workhorse of the JUnit assertion library are the assertEquals() and assertNotEquals() methods. That method is overloaded, with implementations that accept many different data types. These are all listed in the Assertions documentation, but they all follow the same basic form:

• assertEquals(expected, actual)
• assertNotEquals(expected, actual)

For floating-point values such as the double data type, you can also specify a delta value, such that the values are considered equal as long as their positive difference is less than delta

• assertEquals(expected, actual, delta)
• assertNotEquals(expected, actual, delta)

public static void main(String[] args) 
{ 
    double a = 0.7; 
    double b = 0.9; 
    double x = a + 0.1; 
    double y = b - 0.1; 

    System.out.println("x = " + x); 
    System.out.println("y = " + y ); 
    System.out.println(x == y); 
}

Array Assertions

JUnit also includes assertions for arrays. These methods are also overloaded to handle many different data types:

• assertArrayEquals(expected, actual)

This method is really handy when we need to check that the contents of an entire array match the values we expect it to contain.

For lists of strings (List<String> data type), JUnit also includes a special method to confirm that each line matches what is expected.

• assertLinesMatch(expectedLines, actualLines)

This is very handy for checking that multiple lines of output produced by a program match the expected output.

Reference Assertions

JUnit also includes several helpful assertion methods that allow us to determine if two objects are the same actual object in memory (the same reference), as well as if an object is null:

• assertNull(actual)
• assertNotNull(actual)
• assertSame(expected, actual)
• assertNotSame(expected, actual)

Catching Exceptions

JUnit also includes a special type of assertion that can be used to catch exceptions. This allows us to assert that a particular piece of code being tested should, or should not, throw an exception.

To do this, JUnit uses a lambda expression, which we haven’t covered yet in this course. We’ll discuss lambdas more in a later chapter. Thankfully, the syntax is very simple. Here’s an example, taken from the JUnit 5 User Guide:

@Test
void exceptionTesting() {
    Exception exception = assertThrows(ArithmeticException.class, () ->
        calculator.divide(1, 0));
    assertEquals("/ by zero", exception.getMessage());
}

The assertThrows(expectedType, executable) method is used to assert that the calculator.divide() method will throw an exception, specifically an ArithmeticException. If that method call does not throw an exception, then the assertion will fail.

The second argument to the assertThrows() method is a lambda expression. In Java, a lambda expression can be thought of as an anonymous function - we are defining a block of code that acts like a function, but we’re not giving it a name. That allows us to pass that block of code as a parameter to another method, where it can be executed. See Anonymous Function on Wikipedia for a deeper explanation. As we mentioned before, we’ll learn more about lambda expressions later in this course.

We can also write code to assert that a method does not throw an exception using the assertDoesNotThrow() assertion:

@Test
void noExceptionTesting() {
    assertDoesNotThrow(() ->
        calculator.multiply(1, 0));
}

Fail

JUnit includes one other assertion that is used to simply fail a test:

• fail(message)

By including the fail() method in our unit test, we can cause a test to fail immediately. This allows us to build conditional statements to test complex values that are difficult to express in the provided assertion methods, and then fail a test if the conditional expression reaches the wrong branch. Here’s a quick example:

@Test
void testFail() {
    if(calculator.multiply(1, 0) > calculator.multiply(0, 1)){
        fail("Commutative property violated!");
    }
}

Checking Output

One task we may want to be able to perform in our unit tests is capturing output printed by the program. By default, any output that is printed using System.out is immediately sent to the terminal, but we can actually redirect that output without our tests in order to capture it and examine its contents.

We already saw how to do this in the “Hello Real World” project. Here’s that code once again:

@Test 
public void testHelloWorldMain() {
    HelloWorld hw = new HelloWorld();
    final PrintStream systemOut = System.out;
    ByteArrayOutputStream testOut = new ByteArrayOutputStream();
    System.setOut(new PrintStream(testOut));
    hw.main(new String[]{});
    System.setOut(systemOut);
    assertEquals(testOut.toString(), "Hello World\n", "Unexpected Output");
}

In that code, we start by storing a reference to the existing System.out as a java.io.PrintStream named systemOut. This will allow us to undo our changes at the end of the test.

Then, we create a new java.io.ByteArrayOutputStream called testOut to store the output printed to the terminal, and use the System.setOut method to redirect System.out to a new PrintStream based on our testOut stream. So, anything printed using System.out will be sent to that PrintStream and captured in our testOut variable.

Once we’ve done those changes, we can then execute our code, calling any functions and including any assertions that we’d like to check. When we are finished, we can then reset System.out back to the original reference using the System.setOut(systemOut) line.

Then, to check the output we received, we can use testOut.toString() to get the output it captured as a single string. If multiple lines of output were printed, they would be separated by \n character, so we could use String.split() to split that single string into individual lines if needed.

Java Hamcrest

We can also choose to use the Hamcrest assertion library in our code, either instead of the JUnit assertions or in addition to them. Hamcrest includes some very helpful assertions that are not part of JUnit, and also includes version for many languages, including both Java and Python. Most of the autograders in previous Computational Core courses are written with the Hamcrest assertion library!

Basic Assertions

Hamcrest uses a single basic assertion method called assertThat() to perform all assertions. It comes in two basic forms:

• assertThat(actual, matcher) - asserts that actual passes the matcher.
• assertThat(message, actual, matcher) - asserts that actual passes the matcher. If not, it will print message as part of the failure.

The real power of Hamcrest lies in the use of Matchers, which are used to determine if the actual value passes a test. If not, then the assertThat method will fail, just like a JUnit assertion.

For example, to test if an actual value returned by a fictional calculator object is equal to an expected value, we could use this statement:

assertThat(calculator.add(1, 3), is(4));

As we can see, reading this statement out loud tells us everything we need to know: “Assert that calculator.add(1, 3) is 4!”

Here are a few of the most commonly used Hamcrest matchers, as listed in the Hamcrest Tutorial. The full list of matchers can be found in the Matchers class in the Hamcrest documentation:

• is(expected) - a shortcut for equality - an example of syntactic sugar as discussed below.
• equalTo(expected) - will call the actual.equals(expected) method to test equality
• isCompatibleType(type) - can be used to check if an object is the correct type, helpful for testing inheritance
• nullValue() - check if the value is null
• notNullValue() - check if the value is not null
• sameInstance(expected) - checks if two objects are the same instance
• hasEntry(entry), hasKey(key), hasValue(value) - matchers for working with Maps such as HashMaps
• hasItem(item) - matcher for Collections such as LinkedList
• hasItemInArray(item) - matcher for arrays
• closeTo(expected, delta) - matcher for testing floating-point values within a range
• greaterThan(expected), greaterThanOrEqualTo(expected), lessThan(expected), lessThanOrEqualTo(expected) - numerical matchers
• equalToIgnoringCase(expected), equalToIgnoringWhiteSpace(expected), containsString(string), endsWith(string), startsWith(string) - string matchers
• allOf(matcher1, matcher2, ...), anyOf(matcher1, matcher2, ...), not(matcher) - boolean logic operators used to combine multiple matchers

Syntactic Sugar

Hamcrest includes a helpful matcher called is that makes some assertions more easily readable. For example, each of these assertion statements from the Hamcrest Tutorial all test the same thing:

assertThat(theBiscuit, equalTo(myBiscuit)); 
assertThat(theBiscuit, is(equalTo(myBiscuit))); 
assertThat(theBiscuit, is(myBiscuit));

By including the is matcher, we can make our assertions more readable. We call this syntactic sugar since it doesn’t add anything new to our language structure, but it can help make it more readable.

Examples

There are lots of great examples of how to use Hamcrest available on the web. Here are a couple that are worth checking out:

• Using Hamcrest for Testing - Tutorial from Vogella
• Testing with Hamcrest from Baeldung

Writing pytest Tests

Video Materials

Writing tests is in many ways just as challenging and creative an endeavor as writing programs. Tests usually consist of invoking some portion of program code, and then using assertions to determine that the actual results match the expected results. The result of these assertions are typically reported on a per-test basis, which makes it easy to see where your program is not behaving as expected.

Consider a class that is a software control system for a kitchen stove. We won’t write the code for the class itself, because it is important for us to be able to write tests that effectively test the code without even seeing it. It might have properties for four burners, which correspond to what heat output they are currently set to. Let’s assume this is as an integer between 0 (off) and 5 (high). When we first construct this class, we’d probably expect them all to be off! A test to verify that expectation would be:

from src.hello.Stove import Stove

class TestStove:
    
    def test_burners_should_be_off_at_initialization(self):
        stove = Stove()
        assert stove.burner_one == 0, "Burner is not off after initialization"
        assert stove.burner_two == 0, "Burner is not off after initialization"
        assert stove.burner_three == 0, "Burner is not off after initialization"
        assert stove.burner_four == 0, "Burner is not off after initialization"

Here we’ve written the test using the pytest test framework, which is one of the most commonly used Python unit testing frameworks today.

Notice that the test is simply a method, defined in a class. This is very common for test frameworks, which tend to be written using the same programming language the programs they test are written in (which makes it easier for one programmer to write both the code unit and the code to test it). The test method itself is prefixed with test, as well as the file where the test is stored. In addition, the class name also includes the word Test. These naming conventions help pytest find test methods in the code, as described in the pytest Guide. Omitting the test prefix in the method name allows us to build other helper methods within our test classes as needed.

Inside the method, we create an instance of stove, and then use the assert statement to determine that the actual and expected values match. If they do, the assertion is marked as passing, and the test runner will display this pass. If it fails, the test runner will report the failure, along with details to help find and fix the problem (what value was expected, what it actually was, and which test contained the assertion).

The pytest framework provides for two kinds of tests, standard tests, which are written as functions that have no parameters, and parameterized tests, which do have parameters. The values for these parameters are supplied with a special method annotation, typically @pytest.mark.parametrize. For example, we might test that when we set a burner to a setting within the valid 0-5 range, it is set to that value:

from src.hello.Stove import Stove
import pytest

class TestStove:
        
    @pytest.mark.parametrize("value", [0, 1, 2, 3, 4, 5])
    def test_should_be_able_to_set_burner_one_to_valid_range(self, value):
        stove = Stove()
        stove.burner_one = value
        assert stove.burner_one == value, "Burner does not have expected value"

The values in the parentheses of the @parametrize annotation are the values supplied to the parameter list of the parameterized test method. Thus, this test is actually six tests; each test makes sure that one of the settings is working. We could have done all six as separate assignments and assertions within a single test method, but using a parameterized test means that if only one of these settings doesn’t work, we will see that one test fail while the others pass. This level of specificity can be very helpful in finding errors.

So far our tests cover the expected behavior of our stove. But where tests really prove their worth is with the edge cases - those things we as programmers don’t anticipate. For example, what happens if we try setting our range to a setting above 5? Should it simply clamp at 5? Should it not change from its current setting? Or should it shut itself off entirely because its user is clearly a pyromaniac bent on burning down their house? If the specification for our program doesn’t say, it is up to us to decide. Let’s say we expect it to be clamped at 5:

@pytest.mark.parametrize("value", [6, 18, 1000000])
def test_burner_one_should_not_exceed_five(self, value):
    stove = Stove()
    stove.burner_one = value
    assert stove.burner_one == 5, "Burner does not have expected value"

Note that we don’t need to exhaustively test all numbers above 5 - it is sufficient to provide a representative sample, ideally the first value past 5 (6), and a few others. Also, now that we have defined our expected behavior, we should make sure the documentation of our burner one property matches it:

@property
def burner_one(self) -> int:
   """Sets the value of Burner One.
   
   Should be an integer between 0 (off) and 5 (high)
   If a value higher than 5 is provided, the burner will be 
   set to 5 instead. 
   
   Args:
       value: the value of the burner
   """

This way, other programmers (and ourselves, if we visit this code years later) will know what the expected behavior is. We’d also want to test the other edge cases: i.e. when the burner is set to a negative number.

For a complete guide to parameterized tests in pyunit, refer to the pyunit Guide.

Python Assertions

Unlike many testing frameworks, the pytest framework by default only uses the built-in assert statement in Python. It doesn’t include a large number of specialized assertions, and instead relies on the developer to write Boolean logic statements to perform the desired testing. More information can be found in the pytest documentation

The pytest framework can leverage the assertions already present in other Python unit testing libraries such as the built-in unittest library. So, for developers familiar with that approach, those assertions can be used.

For this course, we’ll discuss how to use the built-in assert statement, as well as the Hamcrest assertion library.

Simple Assertions

In general, an assert statement for pytest includes the following structure:

assert <boolean expression>

For example, to test if the variable actual is equal to the variable expected, we would write the following assertion:

assert actual == expected

We can optionally add an error message describing the assertion, as in this example:

assert actual == expected, "The value returned is incorrect"

This allows us to provide additional information along with the failure. However, by including a message in this way, it may reduce the amount of information that pytest gives us when the test fails. So, we may find it easier to omit these messages, or include them as comments in the code near the assertion, instead of as part of the assertion itself.

Let’s look at some examples to see how we can use the assert statement in various ways.

• Boolean Assertions:
  • assert actual == True
  • assert actual == False
• Equality Assertions
  • assert acutal == expected
  • assert actual != expected
• Approximate Floating-Point Values
  • assert actual == pytest.approx(expected)
• Reference Assertions
  • assert actual is expected - true if both actual and expected are the same object in memory
  • assert actual is None - true if actual is the value None

a = 0.7
b = 0.9
x = a + 0.1
y = b - 0.1
print(x)
print(y)
print(x == y)

Catching Exceptions

The pytest framework also includes a special method that can be used to catch exceptions. This allows us to assert that a particular piece of code being tested should, or should not, throw an exception.

Here’s an example, taken from the pytest documentation:

def test_zero_division():
    with pytest.raises(ZeroDivisionError):
        calculator.divide(1, 0)

The with pytest.raises(ZeroDivisionError) statement is used to assert that the calculator.divide() method will throw an exception, specifically a ZeroDivisionError. If that method call does not throw an exception, then the assertion will fail. We can include multiple lines of code within the with block as well.

Fail

pytest includes one other assertion that is used to simply fail a test:

• fail(message)

By including the fail() method in our unit test, we can cause a test to fail immediately, such as when we reach a state that should be unreachable.

Checking Output

One task we may want to be able to perform in our unit tests is capturing output printed by the program. By default, any output that is printed using print() is immediately sent to the terminal, but we can actually redirect that output without our tests in order to capture it and examine its contents.

We already saw how to do this in the “Hello Real World” project. Here’s that code once again (with full type annotations):

from pytest import CaptureFixture
from _pytest.capture import CaptureResult
from typing import Any
from src.hello.HelloWorld import HelloWorld

def test_hello_world(self, capsys: CaptureFixture[Any]) -> None:
    HelloWorld.main(["HelloWorld"])
    captured: CaptureResult[Any] = capsys.readouterr()
    assert captured.out == "Hello World\n", "Unexpected Output"

In that code, we start by adding a parameter named capsys to the test method declaration. capsys is an example of a fixture in pytest. Fixtures allow us to do build more advanced test functions. The capsys fixture is described in the pytest documentation.

So, by including that parameter in our test function, we’ll gain access to all of the features of the capsys fixture. When we execute our code, we can then use capsys.readouterror() to get a CaptureResult object that contains the text that was output by our program. Then, using captured.out, we can check that text and make sure it matches our expectation in an assertion.

Python Hamcrest

We can also choose to use the Hamcrest assertion library in our code, either instead of the pyunit assertions or in addition to them. Hamcrest includes some very helpful assertions that are not part of pyunit, and also includes version for many languages, including both Python and Java. Most of the autograders in previous Computational Core courses are written with the Hamcrest assertion library!

Basic Assertions

Hamcrest uses a single basic assertion method called assert_that() to perform all assertions. It comes in two basic forms:

• assert_that(actual, matcher) - asserts that actual passes the matcher.
• assert_that(actual, matcher, message) - asserts that actual passes the matcher. If not, it will print message as part of the failure.

The real power of Hamcrest lies in the use of Matchers, which are used to determine if the actual value passes a test. If not, then the assert_that method will fail, just like a pyunit assertion.

For example, to test if an actual value returned by a fictional calculator object is equal to an expected value, we could use this statement:

assert_that(calculator.add(1, 3), is_(4))

As we can see, reading this statement out loud tells us everything we need to know: “Assert that calculator.add(1, 3) is 4!”

Here are a few of the most commonly used Hamcrest matchers, as listed in the Hamcrest Tutorial. The full list of matchers can be found in the Matcher Library in the Hamcrest documentation:

• is_(expected) - a shortcut for equality - an example of syntactic sugar as discussed below. Notice the underscore to differentiate it from the Python keyword is
• equal_to(expected) - will call the actual.equals(expected) method to test equality
• instance_of(type) - can be used to check if an object is the correct type, helpful for testing inheritance
• none() - check if the value is None
• not_none() - check if the value is not None
• same_instance(expected) - checks if two objects are the same instance
• has_entry(key, value), has_key(key), has_value(value) - matchers for working with mapping types like dictionaries
• has_item(item) - matcher for sequence types like lists
• close_to(expected, delta) - matcher for testing floating-point values within a range
• greater_than(expected), greater_than_or_equal_to(expected), less_than(expected), less_than_or_equal_to(expected) - numerical matchers
• equal_to_ignoring_case(expected), equal_to_ignoring_whitespace(expected), cotnains_string(string), ends_with(string), starts_with(string) - string matchers
• all_of(matcher1, matcher2, ...), any_of(matcher1, matcher2, ...), is_not(matcher) - boolean logic operators used to combine multiple matchers

Syntactic Sugar

Hamcrest includes a helpful matcher called is_() that makes some assertions more easily readable. For example, each of these assertion statements from the Hamcrest Tutorial all test the same thing:

assert_that(theBiscuit, equal_to(myBiscuit))
assert_that(theBiscuit, is_(equal_to(myBiscuit)))
assert_that(theBiscuit, is_(myBiscuit))

By including the is_() matcher, we can make our assertions more readable. We call this syntactic sugar since it doesn’t add anything new to our language structure, but it can help make it more readable.

Running Tests

Once we’ve written our unit tests, we can execute them against our code to see how well it works. Tests are usually run with a test runner, a program that will execute the test code against the code to be tested. The exact mechanism involved depends on the testing framework.

As we discovered in the “Hello Real World” project, both JUnit and pytest have a way to automatically discover all of the tests we’ve created, provided we place them in the correct location and possibly give them the correct name.

Outside of Codio, many integrated development environments, or IDEs, support running unit tests directly through their interface. We won’t cover much of that in this class, but it is handy to know that it can be done graphically as well.

Once the test runner is done executing our tests, we’ll be given information about the tests which failed. We’ve also learned how to create an HTML report that gives us helpful information about our tests and why they failed. So, we can look through that information to determine if our code needs to be updated, or if the test is not testing our code correctly.

Occasionally, you may end up with problems executing your tests. So, as with any development process, it is helpful to work incrementally, and run your tests each time you add or change code. This allows you to catch errors as they happen when the code is fresh in your mind, and it will be that much easier to fix the problem.

It’s also a good idea to run all of your previously passed tests anytime you make a change to your code. This practice is known as regression testing, and can help you identify errors your changes introduce that break what had previously been working code. This is also one of the strongest arguments for writing test code rather than performing ad-hoc testing; automated tests are easy to repeat.

Code Coverage

The term test code coverage refers to how much of your program’s code is executed as your tests run. It is a useful metric for evaluating the depth of your test, if not necessarily the quality. Basically, if your code is not executed in the test framework, it is not tested in any way. If it is executed, then at least some tests are looking at it. So aiming for a high code coverage is a good starting point for writing tests.

While test code coverage is a good starting point for evaluating your tests, it is simply a measure of quantity, not quality. It is easily possible for you to have all of your code covered by tests, but still miss errors. You need to carefully consider the edge cases - those unexpected and unanticipated ways your code might end up being used.

Testing Strategies

Unit testing is a small part of a much larger world of software testing strategies that we can employ in our workflow. On this page, we’ll review some of the more common testing strategies that we may come across.

White Box vs. Black Box Testing