Introduction

One of the strategies for combating the challenges of the software crisis is writing clear documentation to support both the end-users who will use the program, as well as other developers who will update and maintain the code. Today, including high-quality documentation along with your code, both in the form of code comments and other external documentation, is seen as an important practice among software developers, especially those working on large projects with multiple developers.

In this chapter, we’ll learn about these terms:

• User Documentation
• Developer Documentation
• HTML
• Markdown
• XML
• Code Comments
• Javadoc
• Python Docstrings
• Generated Documentation

After this chapter and the associated example project, we should be able to write effective documentation within our code using the correct format for our chosen programming language.

Documentation Types

Documentation refers to the written materials that accompany program code. Documentation plays multiple, and often critical roles. Broadly speaking, we split documentation into two categories based on the intended audience:

• User Documentation is meant for the end-users of the software
• Developer Documentation is meant for the developers of the software

As you might expect, the goals for these two styles of documentation are very different. User documentation instructs the user on how to use the software. Developer documentation helps orient the developer so that they can effectively create, maintain, and expand the software.

Historically, documentation was printed separately from the software. This was largely due to the limited memory available on most systems. For example, the EPIC software we discussed had two publications associated with it: a User Manual, which explains how to use it, and Model Documentation which presents the mathematic models that programmers adapted to create the software. There are a few very obvious downsides to printed manuals: they take substantial resources to produce and update, and they are easily misplaced.

User Documentation

As memory became more accessible, it became commonplace to provide digital documentation to the users. For example, with Unix (and Linux) systems, it became commonplace to distribute digital documentation alongside the software it documented. This documentation came to be known as man pages based on the man command (short for manual) that would open the documentation for reading. For example, to learn more about the Linux search tool grep, you would type the following command into a Linux terminal:

man grep 

That would open the documentation distributed with the grep tool. Man pages are written in a specific format; you can read more about it here.

While man pages are a staple of the Unix/Linux operating system, there was no equivalent in the DOS ecosystem (the foundations of Windows) until PowerShell was released in 2007, including the Get-Help tool. You can read more about it here.

However, once software began to be written with graphical user interfaces (GUIs), it became commonplace to incorporate the user documentation directly into the GUI, usually under a “Help” menu. This served a similar purpose to man pages by ensuring user documentation was always available with the software. Of course, one of the core goals of software design is to make the software so intuitive that users don’t need to reference the documentation. It is equally clear that developers often fall short of that mark, as there is a thriving market for books to teach certain software.

¹

Of course, there are also thousands of YouTube channels devoted to teaching users how to use specific programs!

Developer Documentation

Developer documentation underwent a similar transformation. Early developer documentation was often printed and placed in a three-ring binder, as Neal Stephenson describes in his novel Snow Crash: ²

Fisheye has taken what appears to be an instruction manual from the heavy black suitcase. It is a miniature three-ring binder with pages of laser-printed text. The binder is just a cheap unmarked one bought from a stationery store. In these respects, it is perfectly familiar to Him: it bears the earmarks of a high-tech product that is still under development. All technical devices require documentation of a sort, but this stuff can only be written by the techies who are doing the actual product development, and they absolutely hate it, always put the dox question off to the very last minute. Then they type up some material on a word processor, run it off on the laser printer, send the departmental secretary out for a cheap binder, and that's that.

Shortly after the time this novel was written, the Internet became available to the general public, and the tools it spawned would change how software was documented forever. Increasingly, web-based tools are used to create and distribute developer documentation. Wikis, bug trackers, and autodocumentation tools quickly replaced the use of lengthy, and infrequently updated, word processor files.

Documentation Formats

Video Materials

Developer documentation often faces a challenge not present in other kinds of documents - the need to be able to display snippets of code. Ideally, we want code to be formatted in a way that preserves indentation. We also don’t want code snippets to be subject to spelling and grammar checks, especially auto-correct versions of these algorithms, as they will alter the snippets. Ideally, we might also apply syntax highlighting to these snippets. Accordingly, a number of textual formats have been developed to support writing text with embedded program code, and these are regularly used to present developer documentation. Let’s take a look at several of the most common.

HTML

Since its inception, HTML has been uniquely suited for developer documentation. It requires nothing more than a browser to view - a tool that nearly every computer is equipped with (in fact, most have two or three installed). And the <code> element provides a way of styling code snippets to appear differently from the embedded text, and <pre> can be used to preserve the snippet’s formatting. Thus:

<p>This algorithm reverses the contents of the array, <code>nums</code></p>
<pre><code>for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}
</code></pre>

Will render in a browser as:

for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}

JavaScript and CSS libraries like highlight.js, prism, and others can provide syntax highlighting functionality without much extra work.

Of course, one of the strongest benefits of HTML is the ability to create hyperlinks between pages. This can be invaluable in documenting software, where the documentation about a particular method could include links to documentation about the classes being supplied as parameters, or being returned from the method. This allows developers to quickly navigate and find the information they need as they work with your code.

Markdown

However, there is a significant amount of boilerplate involved in writing a webpage (i.e. each page needs a minimum of elements not specific to the documentation to set up the structure of the page). The extensive use of HTML elements also makes it more time-consuming to write and harder for people to read in its raw form. Markdown is a markup language developed to counter these issues. Markdown is written as plain text, with a few special formatting annotations, which indicate how it should be transformed to HTML. Some of the most common annotations are:

• Starting a line with hash (#) indicates it should be a <h1> element, two hashes (##) indicates a <h2>, and so on…
• Wrapping a statement with underscores (_) or asterisks (*) indicates it should be wrapped in a <i> element
• Wrapping a statement with double underscores (__) or double asterisks (**) indicates it should be wrapped in a <b> element
• Links can be written as [link text](url), which is transformed to <a href="url">link text</a>
• Images can be written as ![alt text](url), which is transformed to <img alt="alt text" src="url"/>

Code snippets are indicated with backtick marks (`). Inline code is written surrounded with single backtick marks, i.e. `int a = 1` and in the generated HTML is wrapped in a <code> element. Code blocks are wrapped in triple backtick marks, and in the generated HTML are enclosed in both <pre> and <code> elements. Thus, to generate the above HTML example, we would use:

This algorithm reverses the contents of the array, `nums`
```
for(int i = 0; i < nums.length/2; i++) {
    int tmp = nums[i];
    nums[i] = nums[nums.length - 1 - i];
    nums[nums.length - 1 - i] = tmp;
}
```

Most markdown compilers also support specifying the language (for language-specific syntax highlighting) by following the first three backticks with the language name, i.e.:

```java
String aString = "abc123";
```

Nearly every programming language features at least one open-source library for converting Markdown to HTML. In addition to being faster to write than HTML, and avoiding the necessity to write boilerplate code, Markdown offers some security benefits. Because it generates only a limited set of HTML elements, which specifically excludes some most commonly employed in web-based exploits (like using <script> elements for script injection attacks), it is often safer to allow users to contribute markdown-based content than HTML-based content. Note: this protection is dependent on the settings provided to your HTML generator - most markdown converters can be configured to allow or escape HTML elements in the markdown text.

In fact, both the Codio guides in this course, as well as the website used to store the project milestones, was written using Markdown. Codio includes its own Markdown converter, whereas the website was converted to HTML using the Hugo framework, a static website generator built using the Go programming language.

Additionally, chat servers like RocketChat and Discord support using markdown in posts! Try it out sometime!

GitHub even incorporates a markdown compiler into its repository displays. If your file ends in a .md extension, GitHub will evaluate it as Markdown and display it as HTML when you navigate your repository. If your repository contains a README.md file at the top level of your project, it will also be displayed as the front page of your repository. GitHub uses an expanded list of annotations known as GitHub-flavored markdown that adds support for tables, task item lists, strikethroughs, and others. You can also use Markdown in GitHub pull requests, comments, and more!

XML

Extensible Markup Language (XML) is a close relative of HTML - they share the same ancestor, Standard Generalized Markup Language (SGML). It allows developers to develop their own custom markup languages based on the XML approach, i.e. the use of elements expressed via tags and attributes. XML-based languages are usually used as a data serialization format. For example, this snippet represents a serialized fictional student:

<student>
    <firstName>Willie</firstName>
    <lastName>Wildcat</lastName>
    <wid>8888888</wid>
    <degreeProgram>BCS</degreeProgram>
</student>

While XML is most known for representing data, it can also be used to create documentation, most notably in the Microsoft .NET ecosystem.

Code Comments

Of course, one of the most important ways that developers can add documentation to their software is through the use of code comments. A code comment is simply extra text added to the source code of a program which is ignored by the compiler or interpreter - it is only visible within the source code itself. Nearly every programming language supports the inclusion of code comments to help describe or explain how the code works, and it is a vital way for developers to make notes, share information, and make sure anyone else reading the code can truly understand what it does.

Writing Useful Comments

Unfortunately, there is not a well established rule for what constitutes a useful code comment, or even how many comments should be included in code. Various developers have proposed ideas such as Literate Programming, which involves writing complete explanations of the program’s logic, all the way down to Self-Documenting Code, which proposes the idea that using properly named variables and well structured code will eliminate the need for any documentation at all, and everything in between. There are numerous articles and books written about how to document code properly that can be found through a simple online search.

For the purposes of this course, we recommend writing useful code comments anytime the code contains something interesting or unique, or something that required a bit of thinking and effort to create or understand. In that way, the next time a developer looks at the code, we can reduce the amount of time that developer spends trying to understand what the code is doing.

In short, we should write comments that help us understand our code better, but we shouldn’t focus on commenting every single line or expression, especially when it is pretty obvious what it does. To help with that, we can use properly named variables that accurately describe the data being manipulated, and use simple expressions that are easy to follow instead of complex ones.

Comment Formats

Each programming language defines its own specification for comments. Here is the basic information for both Java and Python.

// Single line Java comments are prefixed by two slashes.

int x = 5; // Comments can be placed at the end of a line.

/*
 * This is an example of a block comment.
 *
 * It begins with a slash and an asterisk, and ends
 * with an asterisk and a slash.
 *
 * By convention, each line is prefixed with an asterisk
 * that is aligned with the starting asterisk, but this is not
 * strictly required.
 */
 
/**
 * This is an example of a documentation comment.
 *
 * It begins with a slash and a two asterisks, and ends
 * with an asterisk and a slash.
 *
 * By convention, each line is prefixed with an asterisk
 * that is aligned with the starting asterisk, but this is not
 * strictly required.
 *
 * These blocks are processed by Javadoc to create documentation.
 */

# Single line Python comments are prefixed by a hash symbol

x = 5 # comments can be placed at the end of a line

""" Python does not directly support block comments.

However, a bare string literal, surrounded by three double-quotes
can be used to create a longer comment. 

Python refers to these comments as docstrings when used
to document elements such as functions or classes
"""

Formal Code Documentation

In addition to comments within the source code, we can also include formal documentation comments about classes and methods in our code. These comments help describe the functionality of parts of our code, and can be parsed to create generated documentation. On the next two pages, we’ll introduce the documentation standard for both Java and Python. Feel free to only read about the language you are learning, but it might be interesting to see how other languages handle the same idea in different ways.

Javadoc

Video Materials

The Java software development kit (SDK) includes a tool called Javadoc, which can create documentation based on the documentation comments included in the code. Both the Javadoc Documentation and the Google Style Guide include information about how those documentation comments should be structured and the information each should contain. This page will serve as a quick guide for the most common use cases, but you may wish to refer to the documentation linked above for more specific examples and information. The Checkstyle tool is also a great way to check that the documentation comments are properly structured.

General Structure

A properly structured Javadoc comment includes a few parts:

1. A summary fragment. This is the first part of the comment, ending with the first period. It should concisely describe the object being commented, but doesn’t have to be a complete sentence.
2. Additional Paragraphs. Following the summary fragment, additional paragraphs may be included to further describe the object. The paragraphs should start with the <p> tag. However, unlike HTML, notice that there is no matching </p> closing tag required.
3. Tags. Javadoc supports many tags. Here are the most common tags, listed in the order in which they should appear:
   • @author (classes and interfaces only)
   • @version (classes and interfaces only)
   • @param (methods and constructors only)
   • @return (methods only)
   • @throws
   • @see

When including multiple @author, @param or @throws tags, there are some rules governing the ordering of the tags as well. You can find much more information about the tags and how they can be used in the Javadoc Documentation.

Class Comment

Let’s begin by looking at the Javadoc comment for a class. Here’s an example:

/**
 * Represents a chessboard and moves chess pieces.
 *
 * <p>This class stores a chessboard in a 2D array and includes
 * methods to move various chess pieces across the board. Squares
 * are labelled using algebraic chess notation.
 *
 * @author Russell Feldhausen russfeld@ksu.edu
 * @version 0.1
 */
public class Chessboard {

This comment includes a summary fragment, and additional paragraph, and the two required tags for a class comment, @author and @version. At a minimum, each class we develop should include this information directly above the class declaration.

This comment provides enough information for us to understand what the class is used for and a bit about how it works, even without seeing the code.

Method Comment

Here’s another example Javadoc comment, this time for a method:

/**
 * Moves a knight from one square to another
 *
 * <p>If a knight is present on <code>source</code> and 
 * can make a legal move to <code>destination</code>, the method 
 * will perform the move. 
 *
 * @param source       the source square in algebraic chess notation
 * @param destination  the destination square in algebraic chess notation
 * @return             <code>true</code> if a piece was captured; 
 *                     <code>false</code> otherwise
 * @throws IllegalArgumentException     if a knight is not present on 
 *                                      <code>source</code> or if that knight 
 *                                      cannot move to <code>destination</code>
 */
public boolean moveKnight(String source, String destination) {

Similar to the comment above, this comment includes enough information for us to understand exactly what the method does. It tells us about the parameters it accepts and the format it expects, the return value, and any exceptions that could be thrown by this code. With this comment alone, we could probably write the code for the method itself!

Other Comments

The two examples above cover most places where we would use Javadoc comments in our code. The only other example would be for any public attributes of a class, as in this example:

/** The Student's Wildcat ID */
public int wid;

However, as we discussed in a previous module, if we follow the concepts of encapsulation and information hiding we shouldn’t have any publicly-accessible attributes, only public accessor methods such as getters and setters, which can be documented as methods. So, we probably won’t end up using this much in our own code.

Python Docstrings

Video Materials

Many Python developers have standardized on the use of docstrings as documentation comments. Both PEP 257 and the Google Style Guide include information about how those documentation comments should be structured and the information each should contain. This page will serve as a quick guide for the most common use cases, but you may wish to refer to the documentation linked above for more specific examples and information. The flake8 tool along with the flake8-docstrings plugin is also a great way to check that the documentation comments are properly structured.

General Structure

A properly structured docstring comment includes a few parts:

1. A summary line. This is the first part of the comment, ending with the first period. It should concisely describe the object being commented, but doesn’t have to be a complete sentence.
2. Additional Paragraphs. Following the summary fragment, additional paragraphs may be included to further describe the object. The paragraphs should start at the same indentation as the first quotation mark. Paragraphs are separated by blank lines.
3. Optional Sections. While not explicitly required by the standard, there are several optional sections that could be included as part of a docstring. For this course, we’ll use the following sections:
   • Author (files only)
   • Version (files only)
   • Attributes (classes with public attributes only)
   • Args (methods and constructors only)
   • Returns (methods only)
   • Raises

You can find more information about the structure of docstrings in the Google Style Guide.

File Comment

Let’s begin by looking at the docstring comment for a file. Here’s an example:

"""Implements a simple chessboard.

This file contains a class to represent a chessboard.

Author: Russell Feldhausen russfeld@ksu.edu
Version: 0.1
"""

The file docstring gives information about the contents of the file. For object-oriented programs where each file contains a single class, this can be a bit redundant, but it is useful information nonetheless. For other Python files, this may be the only comment included in the file.

While the Python documentation format does not require listing the author or the version, it is a nice convention from the Javadoc format that we can carry over into our Python docstrings as well.

Class Comment

Next, let’s look at the docstring comment for a class. Here’s an example:

class Chessboard:
    """Represents a chessboard and moves chess pieces.
    
    This class stores a chessboard in a 2D array and includes
    methods to move various chess pieces across the board. Squares
    are labelled using algebraic chess notation.
    """

This comment includes a summary fragment, and an additional paragraph. Since the class doesn’t include any public attributes, we omit that section. Instead, we’ll document the accessor methods, or getters and setters, as part of the Python property that is used to access or modify private attributes.

This comment provides enough information for us to understand what the class is used for and a bit about how it works, even without seeing the code.

Method Comment

Here’s another example docstring comment, this time for a method:

def move_knight(self, source: str, destination: str) -> bool:
    """Moves a knight from one square to another
    
    If a knight is present on source and 
    can make a legal move to destination, the method 
    will perform the move. 
    
    Args:
        source: the source square in algebraic chess notation
        destination: the destination square in algebraic chess notation
        
    Returns:
        True if a piece was captured; False otherwise
 
    Raises:
        ValueError: if a knight is not present on source or 
          if that knight cannot move to destination
    """

Similar to the comment above, this comment includes enough information for us to understand exactly what the method does. It tells us about the parameters it accepts and the format it expects, the return value, and any exceptions that could be thrown by this code. With this comment alone, we could probably write the code for the method itself!

Generated Documentation

One of the biggest innovations in documenting software was the development of documentation generation tools. These were programs that would read source code files, and combine information parsed from the code itself and information contained in code comments to generate documentation in an easy-to-distribute form (often HTML).

This approach meant that the language of the documentation was embedded within the source code itself, making it far easier to update the documentation as the source code was refactored. Then, every time a release of the software was built, the documentation could be regenerated from the updated comments and source code. This made it far more likely developer documentation would be kept up-to-date.

So, once we have properly documented our code using documentation comments, we can then use tools such as Javadoc for Java or pdoc3 for Python to automatically generate documentation for developers. That documentation contains all of the contents of our documentation comments, and serves as a handy reference for any developers who wish to use our code.

In the Java ecosystem, this is best represented by the Java API itself, which is generated using Javadoc directly from the source code of the Java SDK itself.

For Python, there are many documentation generators available, but we’ve chosen to use pdoc3. An example of its output is the pdoc3 Documentation.

In either case, the use of these tools, combined with up to date documentation comments in our code, means that we can easily generate documentation quickly and easily.

Summary

In this chapter, we examined the need for software documentation aimed at both end-users and developers (user documentation and developer documentation, respectively). We also examined some formats this documentation can be presented in: HTML, Markdown, and XML. We also discussed documentation generation tools, which generate developer documentation from specially-formatted comments in our code files.

We examined the both the Java and Python approach to documentation comments, helping other developers understand our code. For this reason, as well as the ability to produce HTML-based documentation using a documentation generator tool, it is best practice to use documentation comments in all our programs.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.