Web Data

Now that we’ve seen how to dynamically create the content of a Razor Page on the server, we should turn our attention to how we can do so based on our user’s needs. Consider the example application we have been developing - it exposes a database of movie information to the user. How might a user want to use this database? They might want to find the details for a specific movie - who directed it, when was it released, etc. They might be looking for a movie they want to watch in a favorite genre. They might be looking to find all the movies directed by a favorite director… there are a lot of possibilities.

As software developers, we need to anticipate how our users will likely want to use the software we are developing. And with that in mind, we need to develop user interfaces that allow the user to communicate those needs to the software we are writing as concisely as possible. Remember that in HTTP, all communication between the client (and its user) and the server (our web application), is mediated through the request-response mechanism:

Request-Response Mechanism

Thus, any information the user wants to supply the server is typically communicated through a request. HTML provides a mechanism for enabling this communication in a structured way - forms. A <form> element, when submitted, serializes the values of all <input> elements contained within it and submits them to the server in the request.

The values in the form are serialized (converted into text) and submitted with the request. Exactly how this serialization happens depends on the kind of request. For GET requests the serialized data is added to the url as the query string. For POST requests, the serialized data is sent as the body of the request.

You can usually use either form (except when uploading files, which must be POSTed). However, you might think about the semantics of each action. A GET request is for retrieving something from the server, so any data your sending is about modifying that request. Thus, search and filter requests make sense as a GET request. A POST request is about sending something to the server, thus account creation, blog posts, and so on make sense as a POST request.

Info

When sent using secure HTTP (HTTPS), both the query string portion of the URL and the request body are encrypted, so there is no difference in the in-transit security. However, browsers typically store the full URL in the history, so if sensitive information is being sent, you might want to use POST requests.

Adding Search to the Movie Site

Let’s add a search form and functionality to our movie website. We’ll add this to our Index.cshtml page, just above the <h1> element:

    <form>
        <input type="text" name="SearchTerms"/>
        <input type="submit" value="Search">
    </form>
    <h1>Movie Results</h1>

We’ll also change the <h1> contents to “Movie Results”.

Try typing a search term into the search box, and click the search button. Has anything changed?

The Request Object

When you click the search button, the browser serializes your form, and makes a request against your server including the search terms. By default this request is a GET request, and the contents of the form are serialized using urlencoding (aka percent encoding), a special string format. This string is then appended to the requested url as the query string (aka search string) - a series of key-value pairs proceeded by the question mark symbol(?) and separated by the ampersand (&).

This data is made available to us in our PageModel Index.cshtml.cs by ASP.NET. Let’s take a look at it now. Notice the method public void OnGet()? This method is invoked every time the page is requested using a GET request. Thus, if we need to do some initialization and/or processing, this would be the place to do it.

Inside the PageModel, we can access the request data using the Request object. The exact string can be accessed with Request.QueryString, or the parsed and deserialized results can be accessed from Request.Query. Let’s use the latter to pull out the search terms:

    public void OnGet() 
    {
        String terms = Request.Query["SearchTerms"];
    }

We can store that value, and make it available to the page itself, by creating a public property. Let’s create one named SearchTerms:

    public string SearchTerms { get; set; }

And we’ll refactor our OnGet() to store the search terms coming in from the request:

    public void OnGet() 
    {
        SearchTerms = Request.Query["SearchTerms"];
    }

Now we can refactor our input element to use that public property from our model as its default value:

    <input type="text" name="SearchTerms" value="@Model.SearchTerms"/>

The first time we visit the index page, the SearchTerms value will be null, so our input would have value="". The browser interprets this as empty. If we add a search term and click the search button, we’ll see the page reload. And since @Model.SearchTerms has a value this time, we’ll see that string appear in search box!

Now we just need to search for those terms…

Adding Search to the Database

We’ll start by defining a new static method in our MovieDatabase.cs file to search for movies using the search terms:

    /// <summary>
    /// Searches the database for matching movies
    /// </summary>
    /// <param name="terms">The terms to search for</param>
    /// <returns>A collection of movies</returns>
    public static IEnumerable<Movie> Search(string terms)
    {
        // TODO: Search database
    }

We’ll need a collection of results that implements the IEnumerable<T> interface. Let’s use the familiar List<T>:

    List<Movie> results = new List<Movie>();

Now, there is a chance that the search terms we recieve are null. If that’s the case, we would either 1) return all the movies, or 2) return no movies. You can choose either option, but for now, I’ll return all movies

    // Return all movies if there are no search terms
    if(terms == null) return All;

If we do have search terms, we need to add any movies from our database that include those terms in the title. This requires us to check each movie in our database:

    // return each movie in the database containing the terms substring
    foreach(Movie movie in All)
    {
        if(movie.Title.Contains(terms, StringComparison.InvariantCultureIgnoreCase)) 
        {
            results.Add(movie);
        }
    }   

We’ll use String.Contains() to determine if our terms are a substring within the title, ignoring case differences. If we find it, we’ll add the movie to our results list.

Finally, we’ll return that list:

    return results;

Now, we can refactor our Index.cshtml.cs to use this new search method:

    /// <summary>
    /// The movies to display on the index page 
    /// </summary>
    public IEnumerable<Movie> Movies { get; protected set; }

    /// <summary>
    /// The current search terms 
    /// </summary>
    public string SearchTerms { get; set; }

    /// <summary>
    /// Gets the search results for display on the page
    /// </summary>
    public void OnGet() 
    {
        SearchTerms = Request.Query["SearchTerms"];
        Movies = MovieDatabase.Search(SearchTerms);
    }

We’ll also need ot refactor our Index.cshtml.cs to use the search results, instead of the entire database:

<ul class="movie-list">
    @foreach(Movie movie in @Model.Movies)
    {
        <li>
            <div class="details">
                <h3 class="title">@movie.Title</h3>
                <div class="mpaa">@movie.MPAARating</div>
                <div class="genre">@movie.MajorGenre</div>
            </div>
            <div class="ratings">
                @if(movie.IMDBRating != null)
                {
                    <div class="imdb">
                        @movie.IMDBRating
                    </div>
                }
                @if(movie.RottenTomatoesRating != null)
                {
                    <div class="rotten-tomatoes">
                        @movie.RottenTomatoesRating
                    </div>
                }
            </div>
        </li>
    }
</ul>

If we try running the project again, and searching for the term “Love”… it crashes? What is going on?

The Encountered Exception

Notice that the error is a NullReferenceException, and occurs in our if statement checking the title.

Bad Data

If we think about what variables are involved in the line if(movie.Title.Contains(terms, StringComparison.InvariantCultureIgnoreCase)), we have:

  • movie
  • movie.Title
  • terms

Which of these three values can be null? We know for certain terms is not, as we test for the null value and return if it exists just before this portion of our code. Similarly, movie cannot be null, as it is an entry in the list provided by All, and if it were null, our page would have crashed before we added searching. That leaves movie.Title as a possibility.

If we comb through the data in movies.json, we find on line 54957 a movie with null for a title:

  {
    "Title": null,
    "USGross": 26403,
    "WorldwideGross": 3080493,
    "USDVDSales": null,
    "ProductionBudget": 3700000,
    "ReleaseDate": "Nov 03 2006",
    "MPAARating": "Not Rated",
    "RunningTime": 85,
    "Distributor": "IFC Films",
    "Source": "Original Screenplay",
    "MajorGenre": "Thriller/Suspense",
    "CreativeType": "Contemporary Fiction",
    "Director": null,
    "RottenTomatoesRating": 39,
    "IMDBRating": 6.6,
    "IMDBVotes": 11986
  },

Working from the provided metadata, we can eventually identify the film as one titled Unknown. It would seem that whomever wrote the script to create this JSON file interpreted “Unknown” to mean the title was unknown (hence null), rather than the literal word “Unknown”.

If we dig deeper into the JSON file, we can find other issues. For example, the JSON identifies the controversial film Birth of a Nation as being released in 2015, when it was actually the first full-length theatrical film ever released, in 1915! Most likely the original database from which these entries were derived only used two digits for the year, i.e. 15, and the scripter who converted it to JSON chose a threshold date to determine if it was released in the 20 or 21st century, i.e.:

if(date < 28) 
{
    date += 2000;
}
else {
    date += 1900;
}

The earliest movie release date in the JSON is 1928, for “The Broadway Melody”, which suggests that all the movies released between 1915 and 1928 have been mislabeled as being released in the 21st century!

Unfortunately, these kinds of errors are rampant in databases, so as software developers we must be aware that our data may well be dirty - containing erroneous values, and anticipate these errors much like we do with user input. It is a good idea to clean up and fix these errors in our database so that it will be more reliable, but we also need to check for potential errors in our own code, as the database could be updated with more junk data in the future. Thus, we’ll add a null check to our if statement in MovieDatabase.cs:

if(movie.Title != null && movie.Title.Contains(terms, StringComparison.InvariantCultureIgnoreCase)) 

This will protect us against a NullReferenceException when our movie titles are null. Now if you try the search again, you should see the results:

The Search Results for &ldquo;Love&rdquo;

Adding Categorical Filters to the Movie Site

Let’s add some filters to the page as well. We’ll start with categorical filters, i.e. filtering by category. Let’s start by filtering for MAA Rating. We know that there are only a handful of ratings issued by the Motion Picture Association of America - G, PG, PG-13, R, and NC-17. We might be tempted to use an enumeration to represent these values, but in C# an enumeration cannot have strings for values. Nor can we use the hyphen (-) in an enumeration name.

Defining the MPAA Ratings

So let’s define a string array with our MPAA values, and make it accessible from our MovieDatabase class:

    /// <summary>
    /// Gets the possible MPAARatings
    /// </summary>
    public static string[] MPAARatings
    {
        get => new string[]
        {
            "G",
            "PG",
            "PG-13",
            "R",
            "NC-17"
        };
    }

Now in our <form> in Index.cshtml we can add a checkbox for each of these possible values:

<form>
    @foreach  (String rating in MovieDatabase.MPAARating) 
    {
        <label>
            <input type="checkbox" name="MPAARatings" value="@rating"/>
            @rating
        </label>
    }
    
    <input type="text" name="SearchTerms" value="@Model.SearchTerms"/>
    <input type="submit" value="Search">
</form>

If you try running the project now, and check a few boxes, you’ll see the query string results look something like:

?SearchTerms=&MPAARatings=G&MPAARatings=PG-13

Notice how the key MPAARatings is repeated twice? What would that look like in our PageModel? We can find out; declare a var to hold the value in the OnGet() method of Index.cshtml.cs:

    var MPAARatings = Request.Query["MPAARatings"];

If we add a breakpoint on this line, and run our code, then check several boxes (you’ll have to continue the first time you hit the breakpoint), then step over the line, we’ll see that the var MPAA rating is set to a string collection. We could therefore store it in an array property in Index.cshtml.cs, much like we did with our SearchTerms:

    /// <summary>
    /// The filtered MPAA Ratings
    /// </summary>
    public string[] MPAARatings { get; set; }

And we can refactor the line we just added to OnGet() to use this new property:

    MPAARatings = Request.Query["MPAARatings"];

Then, in our Index.cshtml.cs Razor Page, we can refactor the checkbox to be checked if we filtered against this rating in our last request:

    <input type="checkbox" name="MPAARating" value="@rating" checked="@Model.MPAARatings.Contains(rating)"/>

Now our filters stick around when we submit the search request. That just leaves making the filters actually work.

Applying MPAA Rating Filters

Let’s add another method to our MovieDatabase class, FilterByMPAARating():

    /// <summary>
    /// Filters the provided collection of movies
    /// </summary>
    /// <param name="movies">The collection of movies to filter</param>
    /// <param name="ratings">The ratings to include</param>
    /// <returns>A collection containing only movies that match the filter</returns>
    public static IEnumerable<Movie> FilterByMPAARating(IEnumerable<Movie> movies, IEnumerable<string> ratings)
    {
        // TODO: Filter the list

    }

Notice that in this method, we accept an IEnumerable<Movie> parameter. This is the list of movies we want to filter. We use this, instead of the All() we did in the Search() method, as we would want to filter the results of a search.

Let’s do a null/empty check, and just return this shortlist if no filters are specified:

    // If no filter is specified, just return the provided collection
    if (ratings == null || ratings.Count() == 0) return movies;

Otherwise, we’ll use the same process we did before. Start with an empty list of movies, and iterate over the collection seeing if any match. However, as we have two collections (the movies and the ratings), we’ll see if the ratings collection contains the supplied movie’s rating.

    // Filter the supplied collection of movies
    List<Movie> results = new List<Movie>();
    foreach(Movie movie in movies)
    {
        if(movie.MPAARating != null && ratings.Contains(movie.MPAARating))
        {
            results.Add(movie);
        }
    }

Finally, we’ll return our results:

    return results;

Now, back in our PageModel Index.cshtml.cs, we’ll apply our filter to the results of our search. The refactored OnGet() should then be:

    public void OnGet()
    {
        SearchTerms = Request.Query["SearchTerms"];
        MPAARatings = Request.Query["MPAARatings"];
        Movies = MovieDatabase.Search(SearchTerms);
        Movies = MovieDatabase.FilterByMPAARating(Movies, MPAARatings);        
    }

Now we can run a search with filters applied. For example, searching for the word “Love” and movies that are PG or PG-13 yields:

Filtered Search Results

You might be wondering why Cloverfield is listed. But remember, we’re searching by substring, and C LOVE rfield contains love!

Filtering by Genre

Let’s add filters for genre next. But what genres should be included? This is not as clear-cut as our MPAA rating, as there is no standards organization that says “these are the only offical genres that exist.” In fact, new genres emerge from time to time. So a better source of this info might just be to see what Genres are defined in our data, i.e.:

    HashSet<string> genres = new HashSet<string>();
    foreach(Movie movie in All) {
        if(movie.MajorGenre != null) 
        {
            genres.Add(movie.MajorGenre);
        }
    }

Here we use a HashSet instead of a list, as it only adds each unique item once. Duplicates are ignored.

But where would this code go? We could place it in a getter for MovieDatabase.Genres:

    public IEnumerable<String> Genres 
    {
        get 
        {
            HashSet<string> genres = new HashSet<string>();
            foreach(Movie movie in All) {
                if(movie.MajorGenre != null) 
                {
                    genres.Add(movie.MajorGenre);
                }
            }
        }
    }

But this means that every time we want to access it, we’ll search through all the movies… This is an O(n) operation, and will make our website slower.

Instead, let’s create a private static variable in the MovieDatabase class to cache this collection as an array of strings:

    // The genres represented in the database
    private static string[] genres;

And expose it with a public static property:

    /// <summary>
    /// Gets the movie genres represented in the database 
    /// </summary>
    public static string[] Genres => genres;

And finally, we’ll populate this array in the static constructor of MovieDatabase, after the JSON file has been processed:

    HashSet<string> genreSet = new HashSet<string>();
    foreach(Movie movie in movies) {
        if(movie.MajorGenre != null) 
        {
            genreSet.Add(movie.MajorGenre);
        }
    }
    genres = genreSet.ToArray();

This approach means the finding of genres only happens once, and getting the Genre property is a constant-time O(1) operation.

Implementing the filters follows the same process as we used for the MPAA filters; I’ll leave that as an exercise for the reader.

Numerical Filters

Let’s tackle one of the critics ratings next. While we could create categories and use checkboxes, this doesn’t capture the incremental values (i.e. 4.3), and it would be a lot of checkboxes for Rotten Tomatoes ratings! Instead, we’ll use a numerical filter, which limits our possible results to a range - between a minimum and maximum value.

Moreover, let’s clean up our Index page, as it is getting difficult to determine what filter(s) go together, and and are adding more.

Refactoring the Index Page

Let’s move the filters to a column on the left, leave the search bar above, and show our results on the right. This will require refactoring our Index.cshtml file:

    <form id="movie-database">

        <div id="search">
            <input type="text" name="SearchTerms" value="@Model.SearchTerms" />
            <input type="submit" value="Search">
        </div>

        <div id="filters">

            <h4>MPAA Rating</h4>
            @foreach (String rating in MovieDatabase.MPAARating)
            {
                <label>
                    <input type="checkbox" name="MPAARatings" value="@rating" checked="@Model.MPAARatings.Contains(rating)" />
                    @rating
                </label>
            }

            <h4>Genre</h4>
            @foreach (String genre in MovieDatabase.Genres)
            {
                <label>
                    <input type="checkbox" name="Genres" value="@genre" />
                    @genre
                </label>
            }

            <h4>IMDB Rating</h4>
            <div>
                Between
                <input name="IMDBMin" type="number" min="0" max="10" step="0.1" placeholder="min"/>
                and
                <input name="IMDBMax" type="number" min="0" max="10" step="0.1" placeholder="max"/>
            </div>

        </div>

        <div id="results">
            <h1>Movie Results</h1>

            <ul class="movie-list">
                @foreach (Movie movie in @Model.Movies)
                {
                    <li>
                        <div class="details">
                            <h3 class="title">@movie.Title</h3>
                            <div class="mpaa">@movie.MPAARating</div>
                            <div class="genre">@movie.MajorGenre</div>
                        </div>
                        <div class="ratings">
                            @if (movie.IMDBRating != null)
                            {
                                <div class="imdb">
                                    @movie.IMDBRating
                                </div>
                            }
                            @if (movie.RottenTomatoesRating != null)
                            {
                                <div class="rotten-tomatoes">
                                    @movie.RottenTomatoesRating
                                </div>
                            }
                        </div>
                    </li>
                }
            </ul>
        </div>

    </form>

Most of this is simply moving elements around the page, but note that we are using inputs of type=number to represent our range of IMDB values. We can specify a minimum and maximum for this range, as well as an allowable increment. Also, we use the placeholder attribute to put text into the input until a value is added.

Adding More Styles

Now we’ll need to add some rules to our wwwroot/css/styles.css. First, we’ll use a grid for the layout of the form:

form#movie-database {
    display: grid;
    grid-template-columns: 1fr 3fr;
    grid-template-rows: auto auto;
}

The right column will be three times as big as the right.

We can make our search bar span both columns with grid-column-start and grid-column-end:

#search {
    grid-column-start: 1;
    grid-column-end: 3;
    text-align: center;
}

Notice too that for CSS, we start counting at 1, not 0. The filters and the results will fall in the next row automatically, each taking up their own respective grid cell. You can read more about the grid layout in A Complete Guide to Grid.

Let’s go ahead and use flexbox to lay out our filters in a column:

#filters {
    display: flex;
    flex-direction: column;
}

And make our number inputs smaller:

#filters input[type=number] {
    width: 3rem;
}

Notice the use of square brackets in our CSS Selector to only apply to inputs with type number.

Also, let’s remove most of the margin below our <h4> elements:

#filters h4 {
    margin-bottom: 0.2rem;
}

The resulting page looks much cleaner:

The Styled Page

Capturing the Filter Values

Now we need to get the filter values from our GET request query string. We could do this like we’ve been doing before, with:

    Request.Query["IMDBMin"];

But the returned value would be a string, so we’d need to parse it:

    IMDBMin = double.Parse(Request.Query["IMDBMin"]);

If the query was null, then this would evaluate to NAN, which we wouldn’t want to set our <input> to…

Instead, we’ll look at some options built into the PageModel.

Parameter Binding

The first of these options is <em>Parameter Binding</em>. In this approach, we define parameters to our OnGet() method to be parsed out of the request automatically, i.e.:

    /// <summary>
    /// Gets the search results for display on the page
    /// </summary>
    OnGet(string SearchTerms, string[] MPAARatings, string[] Genre, double? IMDBMin, double? IMDBMax) {
        this.SearchTerms = SearchTerms;
        this.MPAARatings = MPAARatings;
        this.Genre = Genre;
        this.IMDBMin = IMDBMin;
        this.IMDBMax = IMDBMax;
        Movies = MovieDatabase.Search(SearchTerms);
        Movies = MovieDatabase.FilterByMPAARating(Movies, MPAARatings);
        Movies = MovieDatabase.FilterByGenre(Movies, Genres);
        Movies = MovieDatabase.FilterByIMDBRating(Movies, IMDBMin, IMDBMax);
    }

The benefit of this approach is that as long as C# knows a conversion into the type we specify, the conversion is done automatically. Note that the parameter name matches the name property of the corresponding <input> - this must be the case for the Razor Page to bind the parameter to the corresponding input value.

Note that we still need to assign these parameter values to the corresponding properties of our PageModel. If we don’t, then those properties will all be null, and the <inputs> rendered on our page will always be blank.

Model Binding

A second option is to use <em>Model Binding</em>. Model binding also automatically converts incoming form data, but in this case it binds directly to the properties of our PageModel. We indicate this form of binding with a [BindProperty] attribute, i.e.:


public class IndexModel : PageModel {

    [BindProperty(SupportsGet=true)]
    string SearchTerms {get; set;}

    [BindProperty(SupportsGet=true)]
    string[] MPAARatings {get; set;}

    [BindProperty(SupportsGet=true)]
    string[] Genre {get; set;}

    [BindProperty(SupportsGet=true)]
    double? IMDBMin {get; set;}

    [BindProperty(SupportsGet=true)]
    double? IMDBMax {get; set;}

    /// <summary>
    /// Gets the search results for display on the page
    /// </summary>
    OnGet() {
        Movies = MovieDatabase.Search(SearchTerms);
        Movies = MovieDatabase.FilterByMPAARating(Movies, MPAARatings);
        Movies = MovieDatabase.FilterByGenre(Movies, Genres);
        Movies = MovieDatabase.FilterByIMDBRating(Movies, IMDBMin, IMDBMax);
    }
}

Note that with this approach, the incoming data is directly bound to the properties, so we don’t need to do any special assignments within our OnGet() method. Also, note that we have to use SupportsGet=true in order for this binding to occur on GET requests (by default, model binding only happens with POST requests).

Note

You only need to do one binding approach per property in a PageModel. I.e. you can just use the property decorator:

public class SomePageModel : PageModel
{
    [BindProperty(SupportsGet=true)]
    public float SomeProperty { get; set; }

    public void OnGet() {
        DoSomething(SomeProperty);
    }
}

or you might use parameter binding:

public class SomePageModel : PageModel
{
    public void OnGet(float SomeProperty) {
        DoSomething(SomeProperty);
    }
}

or you can parse it from the request:

class SomePageModel : PageModel
{
    public void OnGet() {
        var someProperty = float.Parse(Request.Query["SomeProperty"]);
        DoSomething(someProperty);
    }
}

These are all different means of accessing the same data from the incoming request.

Now all we need to do is implement the actual filter.

Implementing the IMDB Rating Filter

We’ll define the new filter in our MovieDatabase class as another static method:

    /// <summary>
    /// Filters the provided collection of movies
    /// to those with IMDB ratings falling within
    /// the specified range
    /// </summary>
    /// <param name="movies">The collection of movies to filter</param>
    /// <param name="min">The minimum range value</param>
    /// <param name="max">The maximum range value</param>
    /// <returns>The filtered movie collection</returns>
    public static IEnumerable<Movie> FilterByIMDBRating(IEnumerable<Movie> movies, double? min, double? max)
    {
        // TODO: Filter movies
    }

Notice that here too we use the nullable double value. So our first step is probably to do a null check:

    if (min == null && max == null) return movies;

But what if only one is null? Should we filter for that part of the range? It wouldn’t be hard to do:

    var results = new List<Movie>();

    // only a maximum specified
    if(min == null)
    {
        foreach(Movie movie in movies)
        {
            if (movie.IMDBRating <= max) results.Add(movie);
        }
        return results;
    }

And the minimum would mirror that:

    // only a minimum specified
    if(max == null)
    {
        foreach(Movie movie in movies)
        {
            if (movie.IMDBRating >= min) results.Add(movie);
        }
        return results;
    }

Finally, we could handle the case where we have both a min and max value to our range:

    // Both minimum and maximum specified
    foreach(Movie movie in movies)
    {
        if(movie.IMDBRating >= min && movie.IMDBRating <= max)
        {
            results.Add(movie);
        }
    }
    return results;

Notice too, that in each of these cases we’re treating the range as inclusive (including the specified minimum and maximum). This is the behavior most casual internet users will expect. If the database and user expectations are different for your audience, you’d want your code to match that expectation.

Now we can filter by IMDB rating:

Filtering By IMDB

Finishing Up

Since we’re displaying the Rotten Tomatoes rating, we should probably also have a filter for it. This will work almost exactly like the IMDB rating - but with the range from 0 to 100. I’ll leave this as an exercise for the reader.

Using LINQ

In our movie website, we’ve been using a custom “database” object, MovieDatabase to provide the movie data and search and filter functionality. In professional practice, we would probably replace this with an external database program. These programs have been developed specifically for persisting and accessing data, and leverage specialized data structures like B+ trees that allow them to do searches and filters much more efficiently.

Some of the most widely adopted database programs are relational databases like MySQL, MsSQL, Postgres, and SQLLite. These predate object-orientation, and accept requests for data in the form of SQL (Structured Query Language). You will learn more about these databases and how to use them in CIS560.

LINQ

Microsoft, inspired by SQL, introduced Language Integrated Query (LINQ) in 2007 to bring the concept of a query language into .NET. LINQ allows you to construct queries in either chained method calls or in a syntax similar to SQL that can operate on collections or relational databases.

For relational databases, LINQ queries are converted internally to SQL queries, and dispatched to the database. But their real power is that they can also be used with any collection that supports either the IEnumerable or IEnumerable<T> interfaces. LINQ can be used to replace the kinds of filter and search functions we wrote for our MovieDatabase. Let’s try it out with our movie website.

LINQ operates through extension methods, so we need to be sure to add the LINQ namespace to our CS files where we are going to employ them. We’ll be adding our LINQ commands to our page model, so sure that is the case with your Pages/Index.cshtml.cs:

using System.Linq;

Adding this using statement will bring in all the extension methods for LINQ, and will also prompt Visual Studio to recognize the query-style syntax.

Searching with LINQ

One of the most useful extension methods in LINQ is Where(). It can be invoked on any IEnumerable and accepts a Func (a delegate type). This delegate takes a single argument - the current item being enumerated over, and its body should return a boolean value - true if the item should be included in the results, and false if it should not.

We can thus use our existing SearchTerms property as the basis for our test: movie => movie.Title != null && movie.Title.Contains(SearchTerms) (note that with lambda expressions, if the result is single expression we can omit the {} and the return is implied). Also note that we incorporate the null check directly into the expression using the boolean and && operator.

We can replace our current search with a Where() call invoking this method:

Movies = MovieDatabase.All.Where(movie => movie.Title != null && movie.Title.Contains(SearchTerms, String));

However if we run the program we will encounter a null exception. On our first GET request, the value of SearchTerms is null, which is not a legal argument for String.Contains().

Making Search Conditional

If the SearchTerms is null, that indicates that the user did not enter a search term - accordingly, we probably don’t want to search. We can wrap our filter in an if test to prevent applying the filter when there is no search term. Thus, we would refactor our filtering expression to:

Movies = MovieDatabase.All;
// Search movie titles for the SearchTerms
if(SearchTerms != null) {
    Movies = Movies.Where(movie => movie.Title != null && movie.Title.Contains(SearchTerms, StringComparison.InvariantCultureIgnoreCase));
}

Now if we run the program, and search for a specific term, we’ll see our results is modified to contain only those movies with the terms in their title!

We can also use the query syntax instead of the extension method syntax, which is similar to SQL:

Movies = MovieDatabase.All;
// Search movie titles for the SearchTerms
if(SerchTerms != null)
{
    Movies = from movie in Movies
        where movie.Title != null && movie.Title.Contains(SearchTerms, StringComparison.InvariantCultureIgnoreCase)
        select movie;
}

This syntax is converted by the C# compiler to use the extension methods as part of the compilation process.

Info

The search approach listed above will only find the exact search terms, i.e. searching for “Clear Present Danger” will not match the film “Clear and Present Danger”. How might you tweak the search approach so that terms would be found individually?

You can use either form for writing your queries, though it is best to stay consistent within a single program.

Filtering with LINQ

Filtering is also accomplished through the Where(). We can add additional tests within our search Where(), but it is more legible to add additional Where calls:

Movies = MovieDatabase.All
    // Search movie titles for the SearchTerms
    if(SearchTerms != null) {
        Movies = Movies.Where(movie => movie.Title != null && movie.Title.Contains(SearchTerms, StringComparison.InvariantCultureIgnoreCase));
    }
    // Filter by MPAA Rating
    if(MPAARatings != null && MPAARatings.Length != 0)
    {
        Movies = Movies.Where(movie =>
            movie.MPAARating != null &&
            MPAARatings.Contains(movie.MPAARating)
            );
    }

The numerical filters are handled the same way - with additional Where clauses. I leave this as an exercise for the reader to complete.

Benefits of LINQ

While LINQ is a bit less code for us to write, there is another big benefit - the actual filtering is only applied when we start iterating through the results. This means that LINQ queries with multiple Where() invocations can combine them into a single iteration.

Consider our movie website example. We have four filters - the search, the MPAA Ratings, the IMDB Ratings, and the Rotten Tomato Rating. In the worst case, each movie in the database would pass each filter, so that our list never got smaller. With the filter functions we wrote in MovieDatabase, we would have to iterate over that full list four times. In terms of complexity, that’s $O(4n)$.

In contrast, because LINQ doesn’t actually run the filters until we start iterating, it can combine all the While tests into a single boolean expression. The result is it only has to iterate through the list once. Hence, its complexity becomes $O(n)$. There is a little additional overhead for holding onto the query information that way, but it’s small compared to the benefit.

Also, we could have done the same kind of optimization ourselves, but it takes a lot of work to set up, and may not be worth it for a single web app. But LINQ provides us that benefit for free.