Introduction

Now that we have explored some ideas about getting data from the web server, let’s turn our attention to sending data to the web server. One of the earliest approaches for doing so is to use a HTML form sent as a HTTP Request, which we’ll take a look at in this chapter.

Key Terms

Some key terms to learn in this chapter are:

• Form
• Encoding

HTML Forms

Video Materials

One of the earliest (and still widely used) mechanisms for transferring data from a browser (client) to the server is a form. The <form> is a specific HTML element that contains input fields and buttons the user can interact with.

The <input> Element

Perhaps the most important - and versatile - of these is the <input> element. By setting its type attribute, we can represent a wide range of possible inputs, as is demonstrated by this table adapted from a similar one on the MDN Web Docs:

Regardless of the type, the <input> element also has a name and value property. The name is similar to a variable name, in that it is used to identify the input’s value when we serialize the form (more about that later), and the value is the value the input currently is (this starts as the value you specify in the HTML, but it changes when the user edits it).

The <textarea> Element

The <textarea> element represents a multi-line text input. Similar to terminal programs, this is represented by columns and rows, the numbers of which are set by the cols and rows attributes, respectively. Thus:

<textarea cols=40 rows=5></textarea>

Would look like:

As with inputs, a <textarea> has a name and value attribute.

The <select> Element

The <select> element, along with <option> and <optgroup> make drop-down selection boxes. The <select> takes a name attribute, while each <option> provides a different value. The <options> can further be nested in <optgroup>s with their own labels. The <select> also has a multiple attribute (to allow selecting multiple options), and size which determines how many options should be displayed at once (with scrolling if more are available).

For example:

<select id="dino-select">
    <optgroup label="Theropods">
        <option>Tyrannosaurus</option>
        <option>Velociraptor</option>
        <option>Deinonychus</option>
    </optgroup>
    <optgroup label="Sauropods">
        <option>Diplodocus</option>
        <option>Saltasaurus</option>
        <option>Apatosaurus</option>
    </optgroup>
</select>

Displays as:

The <label> Element

A <label> element represents a caption for an element in the form. It can be tied to a specific input using its for attribute, by setting its value to the id attribute of the associated input. This allows screen readers to identify the label as belonging to the input, and also allows browsers to give focus or activate the input element when the label is clicked.

For example, if you create a checkbox with a label:

<fieldset style="display:flex; align-items:center;">
  <input type="checkbox" id="example"/>
  <label for="example">Is Checked</label>
</fieldset>

Clicking the label will toggle the checkbox!

The <fieldset> Element

The <fieldset> element is used to group related form parts together, which can be captioned with a <legend>. It also has a for attribute which can be set to the id of a form on the page to associate with, so that the fieldset will be serialized with the form (this is not necessary if the fieldset is inside the form). Setting the fieldset’s disabled attribute will also disable all elements inside of it.

For example:

<fieldset>
  <legend>Who is your favorite muppet?</legend>
  <input type="radio" name="muppet" id="kermit">
    <label for="kermit">Kermit</label>
  </input>
  <input type="radio" name="muppet" id="animal">
    <label for="animal">Animal</label>
  </input>
  <input type="radio" name="muppet" id="piggy">
    <label for="piggy">Miss Piggy</label>
  </input>
  <input type="radio" name="muppet" id="gonzo">
    <label for="gonzo">Gonzo</label>
  </input>
</fieldset>

Would render:

The <form> Element

Finally, the <form> element wraps around all the <input>, <textarea>, and <select> elements, and gathers them along with any contained within associated <fieldset>s to submit in a serialized form. This is done when an <input type="submit"> is clicked within the form, when the enter key is pressed and the form has focus, or by calling the submit() method on the form with JavaScript.

There are a couple of special attributes we should know for the <form> element:

• action - the URL this form should be submitted to. Defaults to the URL the form was served from.
• enctype - the encoding strategy used, discussed in the next section. Possible values are:
  • application/x-www-form-urlencoded - the default
  • multipart/form-data - must be used to submit files
  • text/plain - useful for debugging
• method - the HTTP method to submit the form using, most often GET or POST

When the form is submitted, the form is serialized using the enctype, and submitted using the HTTP method to the URL specified by the action attribute. Let’s take a deeper look at this process next.

Form Data

Form data is simply serialized key/value pairs pulled from a form and encoded using one of the three possible encoding strategies. Let’s look at each in turn.

x-www-form-urlencoded

The default encoding method is application/x-www-form-urlencoded, which encodes the form data as a string consisting of key/value pairs. Each pair is joined by a = symbol, and pairs are in turn joined by & symbols. The key and value strings are further encoded using percent encoding (URL encoding), which replaces special characters with a code beginning with a percent sign (i.e. & is encoded to %26). This prevents misinterpretations of the key and value as additional pairs, etc. Percent encoding is also used to encode URL segments (hence the name URL encoding).

Thus, the form:

<form>
    <input type="text" name="Name" value="Grover"/>
    <select name="Color">
        <option value="Red">Red</option>
        <option selected="true" value="Blue">Blue</option>
        <option value="Green">Green</option>
    </select>
    <input type="number" name="Age" value="36"/>
</form>

Would be encoded as:

Name=Grover&Color=Blue&Age=36

URL-Encoded form data can be submitted with either a GET or POST request. With a GET request, the form data is included in the URL’s query (search) string, i.e. our form above might be sent to:

www.sesamestreet.com/muppets/find?Name=Grover&Color=Blue&Age=36

Which helps explain why the entire serialized form data needs to be URL encoded - it is included as part of the url!

When submitted as a post request, the string of form data is the body of the request.

multipart/form-data

The encoding for multipart/form-data is a bit more involved, as it needs to deal with encoding both regular form values and binary file data. It deals with this challenge by separating each key/value pair by a sequence of bytes known as a boundary, which does not appear in any of the files. This boundary can then be used to split the body back into its constituent parts when parsing. Each part of the body consists of its own head and body sections, with the body of most elements simply their value, while the body of file inputs is the file data encoded in base64. Thus, the form:

<form>
    <input type="text" name="Name" value="Grover"/>
    <select name="Color">
        <option value="Red">Red</option>
        <option selected="true" value="Blue">Blue</option>
        <option value="Green">Green</option>
    </select>
    <input type="number" name="Age" value="36"/>
    <input type="file" name="Image" value="Grover.jpg" />
</form>

Would be encoded into a POST request as:

POST /test HTTP/1.1 
Host: foo.example
Content-Type: multipart/form-data;boundary="boundary" 

--boundary 
Content-Disposition: form-data; name="Name" 

Grover
--boundary 
Content-Disposition: form-data; name="Color" 

Blue
--boundary 
Content-Disposition: form-data; name="Age" 

36
--boundary 
Content-Disposition: form-data; name="Image"; filename="Grover.jpg" 

/9j/4AAQSkZJRgABAQEAYABgAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjI...
--boundary--

Files can only be submitted using multipart/form-data encoding. If you attempt to use application/x-www-form-urlencoded, only the file name will be submitted as the value. Also, as multipart/form-data is always submitted as the body of the request, it can only be submitted as part of a POST request, never a GET. So a form containing a file input should always specify:

<form enctype="multipart/form-data" method="POST">

text/plain

The HTML 5 specification also includes a new form encoding strategy called text/plain. This strategy is exactly what it sounds like - it provides no encoding for the form data, and is meant to be more human readable, but may not be as reliably formatted for computers to interpret. So, it is not recommended for use in most web applications.

Spring and Form Data

Once we’ve built a website that can send form data using an HTTP POST request to our web application, we need some way to access that information in our controller. Let’s look at how we would accomplish this in Spring.

Spring Path Variables

In a previous example, we saw how we can create a route that includes variables directly in the path itself:

@GetMapping("/greeting/{name}")
public String greetingWithName(@PathVariable String name, Model model) {
    model.addAttribute("name", name);
    return "greeting";
}

In this example, we include the annotation @PathVariable before one of the parameters in our method. Spring will automatically match the name of that parameter to the name of one of the path variables in the route, and fill in the value when it calls the function.

Spring Request Parameters

When dealing with data sent in a POST request via an HTML form, we can use a similar method to add those variables to our method. In this case, we’ll use the @RequestParam annotation, which includes some options we can configure as well:

@PostMapping("/advancedsearch")
public String advancedSearchResults(
        @RequestParam(name = "text", required = true, defaultValue = "") String text,
        @RequestParam(name = "checkbox", defaultValue = "false") boolean checkbox,
        @RequestParam(name = "value", required = true, defaultValue = "-1") double value,
        Model model) {
    model.addAttribute("text", text);
    model.addAttribute("checkbox", checkbox);
    model.addAttribute("value", value);
}

The example above shows three different types of request parameters: String values from text entry fields, boolean values from checkboxes, and numerical values from number input fields. Spring will automatically convert the data to the requested type if possible, making it easy to use.

One thing to note is that the value of a checkbox will only be included along with the form if it is checked. If the checkbox is unchecked, that value will not be present in the form data. So, for boolean values, we don’t want to list them as required but should always include a default value of "false" in case they are not included in the form data.

Filling In Form Data

Spring also includes some handy methods for filling out form data based on the values in the template model. This is really helpful for times when we want to allow a user to submit a form but immediately redirect the user back to the same page with the form already completed, as well as some additional data. In addition, as we’ll see in a later chapter, if we have any form validation issues, we can help the user by making it easy to fix the error without having to restart filling out the form.

<input type="text" name="text" placeholder="Enter text here..." th:value="${text}">
<input type="checkbox" name="checkbox" th:checked="${checkbox}">
<input type="number" name="value" placeholder="Number" step="0.1" min="0" max="10" th:value="${value}">

In our HTML templates, we can use the th:value attribute in our input tags to fill the form input based on the given value. For checkboxes, we can use a special th:checked attribute, which will set the checked attribute on the checkbox if the value is present in the model and set to true.

Flask and Form Data

Once we’ve built a website that can send form data using an HTTP POST request to our web application, we need some way to access that information in our controller. Let’s look at how we would accomplish this in Flask.

Flask Path Variables

In a previous example, we saw how we can create a route that includes variables directly in the path itself:

@route('/greeting/<name>/')
def greeting_with_name(self, name):
    """Display greeting with name."""
    return render_template("greeting.html", name=name)

In this example, we simply include the path variable <name in the route, and a corresponding parameter in our controller function. Flask will automatically match the name of that parameter to the name of one of the path variables in the route, and fill in the value when it calls the function.

Flask Request Parameters

When dealing with data sent in a POST request via an HTML form, Flask uses a slightly different approach. Part of the Flask library is the requests object, which can be used to access information about the request sent to the server. Part of that object is a dict named form, which includes all of the form data. So, we can access the data from an HTML form by accessing the elements in the form dictionary:

@route("/advancedsearch/", methods=['POST'])
def advanced_search_results(self):
    """Search results page."""
    # don't use request.form['text'] - raises exceptions!
    text: str = request.form.get('text', None)
    checkbox: bool = bool(request.form.get('checkbox', False))
    try:
        value: float = float(request.form.get('value', "-1"))
    except ValueError:
        value = -1
    return render_template(
        "advanced_search.html",
        text=text,
        checkbox=checkbox,
        value=value)

The example above shows three different types of request parameters: String values from text entry fields, boolean values from checkboxes, and numerical values from number input fields. Since they are all sent as text, we have to use the various methods in Python to convert them to the data type we need.

However, there are a few important things to note in this code. First, instead of directly accessing the elements in the form dictionary, as in request.form['text'], we are using the get() method as described in the Python documentation. This is because directly accessing the elements will raise an exception if they are not present, which we’ll have to handle. Instead, we can use the get method to access them if they are present. If not, we can provide a second parameter which will be the “default” value used if no value is present. This makes it much easier to handle situations where we can’t guarantee that all values would be present in the form.

Likewise, for some numerical values, we may still need to use a try-except statement to safely convert them, as shown in the example above. We are using a default value of -1 in the case that the value is not provided, but also in the except clause if the value provided cannot be properly converted to a numerical value.

Finally, one thing to note is that the value of a checkbox will only be included along with the form if it is checked. If the checkbox is unchecked, that value will not be present in the form data. So, for boolean values, we should always include a default value of "false" in case they are not included in the form data.

Filling In Form Data

Flask also includes some handy methods for filling out form data based on the values in the template model. This is really helpful for times when we want to allow a user to submit a form but immediately redirect the user back to the same page with the form already completed, as well as some additional data. In addition, as we’ll see in a later chapter, if we have any form validation issues, we can help the user by making it easy to fix the error without having to restart filling out the form.

<input type="text" name="text" placeholder="Enter text here..." value="{{ text }}}">
<input type="checkbox" name="checkbox" {{ "checked" if checkbox else "" }}>
<input type="number" name="value" placeholder="Number" step="0.1" min="0" max="10" value="{{ value }}">

In our HTML templates, we can use the value attribute in our input tags to fill the form input based on the given value. For checkboxes, we can use a short Python ternary if statement, which will set the checked attribute on the checkbox if the value is present in the model and set to true.

RESTful Routes

Video Materials

Many web applications deal with some kind of resource, i.e. people, widgets, records. Much like in object-orientation we have organized the program around objects, many web applications are organized around resources. And as we have specialized ways to construct, access, and destroy objects, web applications need to create, read, update, and destroy resource records (we call these CRUD operations).

In his 2000 PhD. dissertation, Roy Fielding defined Representational State Transfer (REST), a way of mapping HTTP routes to the CRUD operations for a specific resource. This practice came to be known as RESTful routing, and has become one common strategy for structuring a web application’s routes. Consider the case where we have an online directory of students. The students would be our resource, and we would define routes to create, read, update and destroy them by a combination of HTTP action and route:

Here the [ID] is a unique identifier for the individual student. Note too that we have two routes for reading - one for getting a list of all students, and one for getting the details of an individual student.

REST is a remarkably straightforward implementation of very common functionality, no doubt driving its wide adoption. Many web application frameworks support REST, either explicitly through special code structures or shortcuts, or implicitly through the use of route parameters.

When we use a RESTful route to create or update new resources, we often want to take an additional step - validating the supplied data.

Validation

Validation refers to the process of making sure the submitted data matches our expectations. Validation can be done client-side or server-side. For example, we can use the built-in HTML form validation properties to enforce rules, like a number that must be positive:

<input type="number" min="0" name="Age" required>

If a user attempts to submit a form containing this input is submitted, and the value is less than 0, the browser will display an error message instead of submitting. In addition, the psuedo-css class :invalid will be applied to the element.

We can also mark inputs as required using the required attribute. The browser will refuse to submit the form until all required inputs are completed. Inputs with a required attribute also receive the :required pseudo-class, allowing you to assign specific styles to them.

You can read more about HTML Form validation on MDN.

Client-side validation is a good idea, because is minimizes invalid requests against our web application. However, we cannot always depend on it, so we also need to implement server-side validation. We can write custom logic for doing this, but many web application frameworks also have built-in support for validation.

Summary

In this chapter we looked at how data is handled in web applications. We saw how forms can be used to submit data to our server, and examined several common encoding strategies. We also saw how we can retrieve this data in our web application by examining the routes or the form data submitted. We also explored the concept of RESTful routes. Finally, we discussed validating submitted values, on both the client and server side of a HTTP request.

You should now be able to handle creating web forms and processing the submitted data.

Review Quiz

Check your understanding of the new content introduced in this chapter below - this quiz is not graded and you can retake it as many times as you want.