Chapter 17

REST and Forms

HTML Form Data and Sensible URL Schemes!

Subsections of REST and Forms

Introduction

Content Note

Much of the content in this page was adapted from Nathan Bean’s CIS 400 course at K-State, with the author’s permission. That content is licensed under a Creative Commons BY-NC-SA license.

Now that we have explored some ideas about getting data from the web server, let’s turn our attention to sending data to the web server. One of the earliest approaches for doing so is to use a HTML form sent as a HTTP Request, which we’ll take a look at in this chapter.

Key Terms

Some key terms to learn in this chapter are:

  • Form
  • Encoding

HTML Forms

Content Note

Much of the content in this page was adapted from Nathan Bean’s CIS 400 course at K-State, with the author’s permission. That content is licensed under a Creative Commons BY-NC-SA license.

YouTube Video

Video Materials

One of the earliest (and still widely used) mechanisms for transferring data from a browser (client) to the server is a form. The <form> is a specific HTML element that contains input fields and buttons the user can interact with.

The <input> Element

Perhaps the most important - and versatile - of these is the <input> element. By setting its type attribute, we can represent a wide range of possible inputs, as is demonstrated by this table adapted from a similar one on the MDN Web Docs:

Type Description Basic Examples
button A push button with no default behavior displaying the value of the value attribute, empty by default. <input type="button" name="button" value="Button" />
checkbox A check box allowing single values to be selected/deselected. <input type="checkbox" name="checkbox" />
<label for="checkbox" style="display: inline">Checkbox</label>
color A control for specifying a color; opening a color picker when active in supporting browsers. <input type="color" name="color" style="width: 40px; height: 40px;" />
date A control for entering a date (year, month, and day, with no time). Opens a date picker or numeric wheels for year, month, day when active in supporting browsers. <input type="date" name="date"/>
datetime-local A control for entering a date and time, with no time zone. Opens a date picker or numeric wheels for date- and time-components when active in supporting browsers. <input type="datetime-local" name="datetime-local"/>
email A field for editing an email address. Looks like a text input, but has validation parameters and relevant keyboard in supporting browsers and devices with dynamic keyboards. <input type="email" name="email"/>
file A control that lets the user select a file. Use the accept attribute to define the types of files that the control can select. <input type="file" accept="image/*, text/*" name="file"/>
hidden A control that is not displayed but whose value is submitted to the server. There is an example in the next column, but it’s hidden! <input id="hidden_id" name="hidden_id" type="hidden" value="f0e1d2c3b4">
← It’s here!
image A graphical submit button. Displays an image defined by the src attribute. The alt attribute displays if the image src is missing. <input type="image" name="image" style="height: 40px;" src="..." alt="Submit"/>
month A control for entering a month and year, with no time zone. <input type="month" name="month"/>
number A control for entering a number. Displays a spinner and adds default validation when supported. Displays a numeric keypad in some devices with dynamic keypads. <input type="number" name="number"/>
password A single-line text field whose value is obscured. Will alert user if site is not secure. <input type="password" name="password"/>
radio A radio button, allowing a single value to be selected out of multiple choices with the same name value. <input type="radio" name="radio"/>
<label style="display: inline" for="radio">Radio</label>
range A control for entering a number whose exact value is not important. Displays as a range widget defaulting to the middle value. Used in conjunction with min and max to define the range of acceptable values. <input type="range" name="range" min="0" max="25"/>
reset A button that resets the contents of the form to default values. Not recommended. <input type="reset" name="reset"/>
search A single-line text field for entering search strings. Line-breaks are automatically removed from the input value. May include a delete icon in supporting browsers that can be used to clear the field. Displays a search icon instead of enter key on some devices with dynamic keypads. <input type="search" name="search"/>
submit A button that submits the form. <input type="submit" name="submit"/>
tel A control for entering a telephone number. Displays a telephone keypad in some devices with dynamic keypads. <input type="tel" name="tel"/>
text The default value. A single-line text field. Line-breaks are automatically removed from the input value. <input type="text" name="text"/>
time A control for entering a time value with no time zone. <input type="time" name="time"/>
url A field for entering a URL. Looks like a text input, but has validation parameters and relevant keyboard in supporting browsers and devices with dynamic keyboards. <input type="url" name="url"/>
week A control for entering a date consisting of a week-year number and a week number with no time zone. <input type="week" name="week"/>

Regardless of the type, the <input> element also has a name and value property. The name is similar to a variable name, in that it is used to identify the input’s value when we serialize the form (more about that later), and the value is the value the input currently is (this starts as the value you specify in the HTML, but it changes when the user edits it).

The <textarea> Element

The <textarea> element represents a multi-line text input. Similar to terminal programs, this is represented by columns and rows, the numbers of which are set by the cols and rows attributes, respectively. Thus:

<textarea cols=40 rows=5></textarea>

Would look like:

As with inputs, a <textarea> has a name and value attribute.

The <select> Element

The <select> element, along with <option> and <optgroup> make drop-down selection boxes. The <select> takes a name attribute, while each <option> provides a different value. The <options> can further be nested in <optgroup>s with their own labels. The <select> also has a multiple attribute (to allow selecting multiple options), and size which determines how many options should be displayed at once (with scrolling if more are available).

For example:

<select id="dino-select">
    <optgroup label="Theropods">
        <option>Tyrannosaurus</option>
        <option>Velociraptor</option>
        <option>Deinonychus</option>
    </optgroup>
    <optgroup label="Sauropods">
        <option>Diplodocus</option>
        <option>Saltasaurus</option>
        <option>Apatosaurus</option>
    </optgroup>
</select>

Displays as:

The <label> Element

A <label> element represents a caption for an element in the form. It can be tied to a specific input using its for attribute, by setting its value to the id attribute of the associated input. This allows screen readers to identify the label as belonging to the input, and also allows browsers to give focus or activate the input element when the label is clicked.

For example, if you create a checkbox with a label:

<fieldset style="display:flex; align-items:center;">
  <input type="checkbox" id="example"/>
  <label for="example">Is Checked</label>
</fieldset>

Clicking the label will toggle the checkbox!

The <fieldset> Element

The <fieldset> element is used to group related form parts together, which can be captioned with a <legend>. It also has a for attribute which can be set to the id of a form on the page to associate with, so that the fieldset will be serialized with the form (this is not necessary if the fieldset is inside the form). Setting the fieldset’s disabled attribute will also disable all elements inside of it.

For example:

<fieldset>
  <legend>Who is your favorite muppet?</legend>
  <input type="radio" name="muppet" id="kermit">
    <label for="kermit">Kermit</label>
  </input>
  <input type="radio" name="muppet" id="animal">
    <label for="animal">Animal</label>
  </input>
  <input type="radio" name="muppet" id="piggy">
    <label for="piggy">Miss Piggy</label>
  </input>
  <input type="radio" name="muppet" id="gonzo">
    <label for="gonzo">Gonzo</label>
  </input>
</fieldset>

Would render:

Who is your favorite muppet?

The <form> Element

Finally, the <form> element wraps around all the <input>, <textarea>, and <select> elements, and gathers them along with any contained within associated <fieldset>s to submit in a serialized form. This is done when an <input type="submit"> is clicked within the form, when the enter key is pressed and the form has focus, or by calling the submit() method on the form with JavaScript.

There are a couple of special attributes we should know for the <form> element:

  • action - the URL this form should be submitted to. Defaults to the URL the form was served from.
  • enctype - the encoding strategy used, discussed in the next section. Possible values are:
    • application/x-www-form-urlencoded - the default
    • multipart/form-data - must be used to submit files
    • text/plain - useful for debugging
  • method - the HTTP method to submit the form using, most often GET or POST

When the form is submitted, the form is serialized using the enctype, and submitted using the HTTP method to the URL specified by the action attribute. Let’s take a deeper look at this process next.

Subsections of HTML Forms

Form Data

Content Note

Much of the content in this page was adapted from Nathan Bean’s CIS 400 course at K-State, with the author’s permission. That content is licensed under a Creative Commons BY-NC-SA license.

Form data is simply serialized key/value pairs pulled from a form and encoded using one of the three possible encoding strategies. Let’s look at each in turn.

x-www-form-urlencoded

The default encoding method is application/x-www-form-urlencoded, which encodes the form data as a string consisting of key/value pairs. Each pair is joined by a = symbol, and pairs are in turn joined by & symbols. The key and value strings are further encoded using percent encoding (URL encoding), which replaces special characters with a code beginning with a percent sign (i.e. & is encoded to %26). This prevents misinterpretations of the key and value as additional pairs, etc. Percent encoding is also used to encode URL segments (hence the name URL encoding).

Thus, the form:

<form>
    <input type="text" name="Name" value="Grover"/>
    <select name="Color">
        <option value="Red">Red</option>
        <option selected="true" value="Blue">Blue</option>
        <option value="Green">Green</option>
    </select>
    <input type="number" name="Age" value="36"/>
</form>

Would be encoded as:

Name=Grover&Color=Blue&Age=36

URL-Encoded form data can be submitted with either a GET or POST request. With a GET request, the form data is included in the URL’s query (search) string, i.e. our form above might be sent to:

www.sesamestreet.com/muppets/find?Name=Grover&Color=Blue&Age=36

Which helps explain why the entire serialized form data needs to be URL encoded - it is included as part of the url!

When submitted as a post request, the string of form data is the body of the request.

multipart/form-data

The encoding for multipart/form-data is a bit more involved, as it needs to deal with encoding both regular form values and binary file data. It deals with this challenge by separating each key/value pair by a sequence of bytes known as a boundary, which does not appear in any of the files. This boundary can then be used to split the body back into its constituent parts when parsing. Each part of the body consists of its own head and body sections, with the body of most elements simply their value, while the body of file inputs is the file data encoded in base64. Thus, the form:

<form>
    <input type="text" name="Name" value="Grover"/>
    <select name="Color">
        <option value="Red">Red</option>
        <option selected="true" value="Blue">Blue</option>
        <option value="Green">Green</option>
    </select>
    <input type="number" name="Age" value="36"/>
    <input type="file" name="Image" value="Grover.jpg" />
</form>

Would be encoded into a POST request as:

POST /test HTTP/1.1 
Host: foo.example
Content-Type: multipart/form-data;boundary="boundary" 

--boundary 
Content-Disposition: form-data; name="Name" 

Grover
--boundary 
Content-Disposition: form-data; name="Color" 

Blue
--boundary 
Content-Disposition: form-data; name="Age" 

36
--boundary 
Content-Disposition: form-data; name="Image"; filename="Grover.jpg" 

/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI...
--boundary--

Files can only be submitted using multipart/form-data encoding. If you attempt to use application/x-www-form-urlencoded, only the file name will be submitted as the value. Also, as multipart/form-data is always submitted as the body of the request, it can only be submitted as part of a POST request, never a GET. So a form containing a file input should always specify:

<form enctype="multipart/form-data" method="POST">

text/plain

The HTML 5 specification also includes a new form encoding strategy called text/plain. This strategy is exactly what it sounds like - it provides no encoding for the form data, and is meant to be more human readable, but may not be as reliably formatted for computers to interpret. So, it is not recommended for use in most web applications.

Spring and Form Data

Once we’ve built a website that can send form data using an HTTP POST request to our web application, we need some way to access that information in our controller. Let’s look at how we would accomplish this in Spring.

Spring Path Variables

In a previous example, we saw how we can create a route that includes variables directly in the path itself:

@GetMapping("/greeting/{name}")
public String greetingWithName(@PathVariable String name, Model model) {
    model.addAttribute("name", name);
    return "greeting";
}

In this example, we include the annotation @PathVariable before one of the parameters in our method. Spring will automatically match the name of that parameter to the name of one of the path variables in the route, and fill in the value when it calls the function.

Spring Request Parameters

When dealing with data sent in a POST request via an HTML form, we can use a similar method to add those variables to our method. In this case, we’ll use the @RequestParam annotation, which includes some options we can configure as well:

@PostMapping("/advancedsearch")
public String advancedSearchResults(
        @RequestParam(name = "text", required = true, defaultValue = "") String text,
        @RequestParam(name = "checkbox", defaultValue = "false") boolean checkbox,
        @RequestParam(name = "value", required = true, defaultValue = "-1") double value,
        Model model) {
    model.addAttribute("text", text);
    model.addAttribute("checkbox", checkbox);
    model.addAttribute("value", value);
}

The example above shows three different types of request parameters: String values from text entry fields, boolean values from checkboxes, and numerical values from number input fields. Spring will automatically convert the data to the requested type if possible, making it easy to use.

One thing to note is that the value of a checkbox will only be included along with the form if it is checked. If the checkbox is unchecked, that value will not be present in the form data. So, for boolean values, we don’t want to list them as required but should always include a default value of "false" in case they are not included in the form data.

Filling In Form Data

Spring also includes some handy methods for filling out form data based on the values in the template model. This is really helpful for times when we want to allow a user to submit a form but immediately redirect the user back to the same page with the form already completed, as well as some additional data. In addition, as we’ll see in a later chapter, if we have any form validation issues, we can help the user by making it easy to fix the error without having to restart filling out the form.

<input type="text" name="text" placeholder="Enter text here..." th:value="${text}">
<input type="checkbox" name="checkbox" th:checked="${checkbox}">
<input type="number" name="value" placeholder="Number" step="0.1" min="0" max="10" th:value="${value}">

In our HTML templates, we can use the th:value attribute in our input tags to fill the form input based on the given value. For checkboxes, we can use a special th:checked attribute, which will set the checked attribute on the checkbox if the value is present in the model and set to true.

Flask and Form Data

Once we’ve built a website that can send form data using an HTTP POST request to our web application, we need some way to access that information in our controller. Let’s look at how we would accomplish this in Flask.

Flask Path Variables

In a previous example, we saw how we can create a route that includes variables directly in the path itself:

@route('/greeting/<name>/')
def greeting_with_name(self, name):
    """Display greeting with name."""
    return render_template("greeting.html", name=name)

In this example, we simply include the path variable <name in the route, and a corresponding parameter in our controller function. Flask will automatically match the name of that parameter to the name of one of the path variables in the route, and fill in the value when it calls the function.

Flask Request Parameters

When dealing with data sent in a POST request via an HTML form, Flask uses a slightly different approach. Part of the Flask library is the requests object, which can be used to access information about the request sent to the server. Part of that object is a dict named form, which includes all of the form data. So, we can access the data from an HTML form by accessing the elements in the form dictionary:

@route("/advancedsearch/", methods=['POST'])
def advanced_search_results(self):
    """Search results page."""
    # don't use request.form['text'] - raises exceptions!
    text: str = request.form.get('text', None)
    checkbox: bool = bool(request.form.get('checkbox', False))
    try:
        value: float = float(request.form.get('value', "-1"))
    except ValueError:
        value = -1
    return render_template(
        "advanced_search.html",
        text=text,
        checkbox=checkbox,
        value=value)

The example above shows three different types of request parameters: String values from text entry fields, boolean values from checkboxes, and numerical values from number input fields. Since they are all sent as text, we have to use the various methods in Python to convert them to the data type we need.

However, there are a few important things to note in this code. First, instead of directly accessing the elements in the form dictionary, as in request.form['text'], we are using the get() method as described in the Python documentation. This is because directly accessing the elements will raise an exception if they are not present, which we’ll have to handle. Instead, we can use the get method to access them if they are present. If not, we can provide a second parameter which will be the “default” value used if no value is present. This makes it much easier to handle situations where we can’t guarantee that all values would be present in the form.

Likewise, for some numerical values, we may still need to use a try-except statement to safely convert them, as shown in the example above. We are using a default value of -1 in the case that the value is not provided, but also in the except clause if the value provided cannot be properly converted to a numerical value.

Finally, one thing to note is that the value of a checkbox will only be included along with the form if it is checked. If the checkbox is unchecked, that value will not be present in the form data. So, for boolean values, we should always include a default value of "false" in case they are not included in the form data.

Filling In Form Data

Flask also includes some handy methods for filling out form data based on the values in the template model. This is really helpful for times when we want to allow a user to submit a form but immediately redirect the user back to the same page with the form already completed, as well as some additional data. In addition, as we’ll see in a later chapter, if we have any form validation issues, we can help the user by making it easy to fix the error without having to restart filling out the form.

<input type="text" name="text" placeholder="Enter text here..." value="{{ text }}}">
<input type="checkbox" name="checkbox" {{ "checked" if checkbox else "" }}>
<input type="number" name="value" placeholder="Number" step="0.1" min="0" max="10" value="{{ value }}">

In our HTML templates, we can use the value attribute in our input tags to fill the form input based on the given value. For checkboxes, we can use a short Python ternary if statement, which will set the checked attribute on the checkbox if the value is present in the model and set to true.

RESTful Routes

Content Note

Much of the content in this page was adapted from Nathan Bean’s CIS 400 course at K-State, with the author’s permission. That content is licensed under a Creative Commons BY-NC-SA license.

YouTube Video

Video Materials

Many web applications deal with some kind of resource, i.e. people, widgets, records. Much like in object-orientation we have organized the program around objects, many web applications are organized around resources. And as we have specialized ways to construct, access, and destroy objects, web applications need to create, read, update, and destroy resource records (we call these CRUD operations).

In his 2000 PhD. dissertation, Roy Fielding defined Representational State Transfer (REST), a way of mapping HTTP routes to the CRUD operations for a specific resource. This practice came to be known as RESTful routing, and has become one common strategy for structuring a web application’s routes. Consider the case where we have an online directory of students. The students would be our resource, and we would define routes to create, read, update and destroy them by a combination of HTTP action and route:

CRUD Operation HTTP Action Route
Create POST /students
Read (all) GET /students
Read (one) GET /students/[ID]
Update PUT or POST /students/[ID]
Destroy DELETE /students/[ID]

Here the [ID] is a unique identifier for the individual student. Note too that we have two routes for reading - one for getting a list of all students, and one for getting the details of an individual student.

REST is a remarkably straightforward implementation of very common functionality, no doubt driving its wide adoption. Many web application frameworks support REST, either explicitly through special code structures or shortcuts, or implicitly through the use of route parameters.

When we use a RESTful route to create or update new resources, we often want to take an additional step - validating the supplied data.

Subsections of RESTful Routes

Validation

Content Note

Much of the content in this page was adapted from Nathan Bean’s CIS 400 course at K-State, with the author’s permission. That content is licensed under a Creative Commons BY-NC-SA license.

Validation refers to the process of making sure the submitted data matches our expectations. Validation can be done client-side or server-side. For example, we can use the built-in HTML form validation properties to enforce rules, like a number that must be positive:

<input type="number" min="0" name="Age" required>

If a user attempts to submit a form containing this input is submitted, and the value is less than 0, the browser will display an error message instead of submitting. In addition, the psuedo-css class :invalid will be applied to the element.

We can also mark inputs as required using the required attribute. The browser will refuse to submit the form until all required inputs are completed. Inputs with a required attribute also receive the :required pseudo-class, allowing you to assign specific styles to them.

You can read more about HTML Form validation on MDN.

Client-side validation is a good idea, because is minimizes invalid requests against our web application. However, we cannot always depend on it, so we also need to implement server-side validation. We can write custom logic for doing this, but many web application frameworks also have built-in support for validation.

Summary

In this chapter we looked at how data is handled in web applications. We saw how forms can be used to submit data to our server, and examined several common encoding strategies. We also saw how we can retrieve this data in our web application by examining the routes or the form data submitted. We also explored the concept of RESTful routes. Finally, we discussed validating submitted values, on both the client and server side of a HTTP request.

You should now be able to handle creating web forms and processing the submitted data.

Web APIs

Not all web applications are built to be viewed in a browser. Many are built to be used by other programs. We call these web applications APIs (Application Programming Interfaces). These also make HTTP or HTTPS requests against our applications, but usually instead of serving HTML, we serve some form of serialized data instead - most commonly XML or JSON.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.

Quizdown quiz omitted from print view.