Introduction

You have now learned about and built examples of several early implementations for web applications - file servers and server pages. These kinds of web applications dominated the Internet in the 90’s, and continue to play a major role today. However, the needs of web applications grew increasingly more dynamic as the web continued to evolve, leading to the development of truly dynamic web servers built around a stack of technologies. You’ve also built several of these as well… and discovered just how long it can take to build one from scratch.

Consider the simple case of an online store - it must authenticate users, keep track of their shopping carts, and in processing orders, send emails to the user and instructions to inventory/warehouse systems. These are not trivial needs, nor are the programs built to meet them simple. These needs heralded the introduction of full-stack web development approaches; the stack being a combination of technologies that had to be used together effectively to host a web application.

In order to be a web developer, therefore, a programmer needed familiarity with a number of technologies, a basic grounding in IT, and proficiency with a programming languages. Of course, there were a limited number of professionals with these capacities available, and demand for them has never stopped growing.

Moreover, the applications themselves began to grow increasingly complex. It is very likely that you’ve already found yourself struggling to understand the various parts of our examples in this class, and how they fit together. This was also true for the developers tackling these problems in the past, especially as many did not have deep enough training. This situation paralleled the software crisis of the 70’s, where increasingly sophisticated programs were being developed by undertrained programmers, and software development was plagued with cost overruns, missed deadlines, and failed projects.

One major outcome of the software crisis was the development of software engineering techniques, and changes to programming languages to help programmers make less errors in writing code, and find their errors more quickly. A similar process occurred with web development in the 2000’s, leading to the development of “web frameworks”. These grew from the realization that most dynamic web servers need the same core features - the ability to interact with persistent data sources, to authenticate users and keep track of their actions on the site, and to interact with other servers. A web development framework seeks to provide those features, allowing the developer to quickly get started building the more unique features of the server.

Web Frameworks

Web frameworks were designed meet these challenges, and to help programmers develop web applications more quickly and with less errors. They do so through providing a standardized approach to building a web application, including imposing a software architecture, providing commonly-needed functionality, libraries for database communication, and generators to create boilerplate code. These frameworks built upon existing Web Stacks, allowing the programmer to focus almost exclusively on the scripting aspect.

A slew of frameworks for various scripting languages emerged in 2005:

• Symfony using PHP
• Django using Python
• Ruby on Rails using Ruby
• Grails using Groovy (a Java Virtual Machine Language)

And new ones continue to emerge relatively often:

• Express using [JavaScript (via Node)] in 2010
• Laravel using PHP in 2011
• Phoenix using Elixir in 2014
• ASP.NET MVC using .NET languages 2015

These represent only some of the best known server-side frameworks. A newer trend are client-side frameworks, which are built on JavaScript (or one of its derivatives) and run in the browser communicating with a web API. Some of the best known of these include:

• Ember using JavaScript in 2011
• Meteor using JavaScript in 2012
• React using JavaScript or JSX in 2013
• Vue.js using JavaScript in 2014
• Angular using TypeScript in 2016
• Next.JS using JavaScript or JSX in 2016

You can see there is a large variety represented and available to us - and these are only some of the best known!

Web Framework Characteristics

Web frameworks can be classified in a number of ways, but there are several that you will see pop up in discussions. For example, an opinionated web framework is one that was developed with a strict software architecture that developers are expected to follow. While it may be possible to deviate from this expected structure, to do so often causes significant headaches and coding challenges. Ruby on Rails is a good example of an opinionated framework - there is a Rails Way of writing a Ruby on Rails application, and deviating from that way will require a lot more work on your part.

Other frameworks do not try to impose a structure - merely offer a suite of tools to use as you see fit. While this comes with a great degree of freedom, it also means that you can find wildly divergent example code written using the framework. Express is a great example of this kind of framework - it is essentially a router that has the ability to add a template library (of your choice) and a database library (of your choice), and does not expect you to lay out your files in any specific way.

Another aspect to frameworks is what kinds of architectural patterns they adopt. Ruby on Rails, Django, and Microsoft’s MVC, all use a Model-View-Controller architecture. Express and Phoenix adopt a Pipeline architecture. Microsoft’s Razor Pages, while built on Microsoft MVC, have gone back to a page-based architecture similar to the server pages we spoke of previously, as does Next.JS.

A third distinction is if the framework is server-side (meaning it runs on the server) or client-side (meaning it consists of a JavaScript program that runs in the browser), or a hybrid of the two. Ruby on Rails, Django, Microsoft’s MVC, Express, and Phoenix are all server-side frameworks - they do the bulk of the work of creating the HTML being served on the server. React, Vue, and Angular are all client-side frameworks that create their HTML dynamically in the browser using JavaScript, typically through making requests against a web API. Meteor and NextJS are hybrids that provide both client-side and server-side libraries.

Single Page Apps

Client-side frameworks often focus on creating single page apps. In this pattern, the entire website consists of a single HTML page with very little content, mostly the <script> elements to download a large client-side JavaScript library. Once downloaded, this library populates the page with HTML content by directly manipulating the DOM.

Consider a drawing application for creating pixel art. It is entirely possible to write a single-page application that only needs a webserver to serve its static HTML and JavaScript files. Once downloaded into the browser, you could draw with it and download the resulting images directly out of the browser - no further server interaction is needed! Thus, you can host such an app on a simple static hosting service at very low cost.

On the other hand, if you did need server functionality (say you want to be able to save drawings to the server and re-open them on a different machine), you can combine your client-side app with a server-side API. It would provide authentication and persistent storage through a few endpoints, most likely communicating through sending JSON as requests and responses.

A good example of this kind of client-side application is MIT Media Labs’ Scratch, a block-based programming environment for teaching programming. The Scratch Development Environment is a client-side app that provides the programming environment, a vm for running the Scratch program, and the user interface code for displaying the result of running that program. All of the computation of running the Scratch program is therefore done in the browser’s JavaScript environment, not on the server. The server provides a minimal API for saving Scratch projects and their resources, as well as publishing them - but the actual heavy computation is always done on the client.

This approach - offloading heavy computation to the browser instead of the server means servers don’t need to work as hard. The benefit is they can be less powerful machines and serve more users while consuming less power, and generating less waste heat. This is why Google Docs, Microsoft 360, Facebook, and other big players have been pushing as much of their site’s computational needs to the client for years.

Progressive Web Apps

A next step in the evolution of single-page apps is the progressive web application (PWA). While these are web applications, they also provide a native-like feel when running in a phone or tablet, and can usually have a shortcut saved to the desktop. It is built around several new web standards that provide specific functionality:

Secure Context

A PWA is always served with https. Also, many features that a PWA might want to use (geolocation, the camera) are only available to a site served over http.

Service Worker

A service worker is a JavaScript script (much like the web workers we learned about earlier) that manages communication between the app running in the browser and the network. Most specifically, it is used to cache information for when the network is unavailable, which can allow your app to run offline.

Manifest File

The manifest is a JSON file that describes the application (much like the Node package.json) that provides the details necessary to load the app and “install” it on mobile devices. Note that installing essentially means running a locally cached copy of the website.