Web Application Trends

Revised 2015 Nov 20¹

TO DO

Add WebSockets
- WHATWG Specification
Add Server-Sent Events
See Tornado - the best Python web framework for descriptions of these.

Introduction

Review limitations of CGI
- Per-request process startup overhead, especially significant for interpreted languages where the starting the interpreter may take more time than running the script
- Low-level API
- Send request, then wait for server response
Reactions:
- Efficiency:
  - Java servlets
  - FastCGI and kin
  - Apache mod_python, mod_php, mod_perl, etc.
  - Python WSGI
- Higher-level APIs: web frameworks
- Ajax and other technologies to make web pages more interactive and responsive

Avoiding the Inefficiency of CGI

Servlets

The World Wide Web was born in 1990, at first with only static web pages; CGI came along in 1993.

Java (1995) gained popularity initially through applets, Java programs running in the web browser (client side). Servlets (1997) are the server-side counterpart.

With servlets, a Java interpreter runs within the web server (usually a special purpose web server such as Tomcat).

Advantages:

The Java interpreter starts when the web server starts: no process start-up for each http request.
Each servlet runs for a long period of time, gaining speed from Java’s incremental (JIT) compilation to native code.

FastCGI

Developed ca. 1995.
The web server and FastCGI server are two separate processes, communicating through sockets.
FastCGI server is long running, avoiding the process start-up overhead.

Advantages:

FastCGI process starts only once.
FastCGI process can be restarted independently of the web server, since it is a separate process.

References:

mod_python, etc.

mod_python (ca. 2000) was developed for the Apache web server to avoid the cost of starting up the Python interpreter (starting a new process) for each CGI script run. Similar modules exist for some other languages, e.g., mod_php, mod_perl.

These Apache web server components (mod_python, mod_php, mod_perl); are similar to servlets, in that they run an interpreter in the web server.

Advantage:

Interpreter starts once.

Disadvantage:

Only for the Apache web server (at least mod_php, etc, per se, though it’s possible other servers have something similar).

WSGI

WSGI (Python Web Server Gateway Interface) is a newer interface which makes mod_python generally obsolete. There is a mod_wsgi for Apache httpd (which is officially Python 2 but maybe also supports Python 3). There are probably implementations for other web servers.

PEP 333 was the original definition for WSGI, now superseded by PEP 3333 for Python 3. (PEP = Python Enhancement Proposal)
The Python standard library contains a WSGI-enabled HTTP server.
There were some difficulties in porting WSGI to Python 3: the rigid distinction between bytes and str(ing) objects seems pre-eminent. In general, HTTP deals with bytes; web pages are delivered as bytes encoding strings (using some encoding), and strings in Python 3 are Unicode strings, which are not bytes and cannot be directly combined with bytes (for example, b'abc' + 'def' is an error).
WSGI middleware means “[WSGI-based software] components that that look like an application to their containing server, while acting as a server for their contained application”. Such middleware can perform services for their contained applications such as preprocessing requests and postprocessing responses.

Example: we can use the Python 3 standard library’s WSGI-enabled HTTP server to run Listing 11–1 (p. 184) easily:

wsgi_app.py (restricted access)

Web Frameworks

So far, we have considered alternatives to CGI that are designed to make more efficient use of the computer hardware. Programmers are another scarce resource that must be used efficiently. Web frameworks provide higher-level APIs, which can help programmers to work more productively. However, there is a start-up cost: the programmer must learn the web framework.

Web frameworks are also called “web application frameworks”, “web server frameworks”, etc.

WSGI provides a low-level API, so it is rarely used directly by web developers (as distinguished from web framework developers).

The Python web frameworks nowadays are built over WSGI, and since WSGI is a “standard” interface to web servers, it makes the web frameworks easier to use with a variety of servers.

A major goal of WSGI is to make any Python web framework usable with any web server.
Web frameworks tend to be large and complex; entire books could be written about Django and Zope, and we would need another course to consider them in serious detail.

Bottle

(We will skip the details of bottle in order to focus on Tornado, a slightly larger web framework for Python. Tornado will have its own page(s) of notes.)

However, bottle is a small (“micro”) web framework which we can use to demonstrate the most basic ideas.

Written for Python 2, but easily converted to 3.
Examples from the Bottle tutorial
Bottle has its own server, for development and testing; so we can easily run the examples. To run these you also need bottle.py. On merlin, just check out the darcs bottle examples repository and run as follows:
```
$ darcs get /home/info/share/i320/bottle
$ cd bottle
$ python3 tut01.py
```
Off merlin, you should be able to check out the repository as follows (if you have darcs installed on your computer):
```
darcs get USER@merlin.iue.edu:/home/info/share/i320/bottle
```
substituting your IU user id for USER.
Small example on p. 188
- @ notation for “decorators”. Decorators are functions applied to functions, returning functions.
- Files from the textbook (restricted access):
  - bottle_app.py
  - bottle_template.html

More on Web Frameworks

Services typically provided by web frameworks
- Routing or dispatching of URLs to software components that will generate responses to the requested URLs. The easiest form of routing is that each URL corresponds to a file: either a static web page or a script that will generate a response. But sometimes you need to do something more flexible, more sophisticated.
- Easy access to request parameters (e.g., form input values).
- Templates, which are, roughly speaking, web pages with spaces or blanks to be filled in. More precisely, the spaces are variables or snippets of code. Templates achieve separation of model from view: the actual data presented, from the way it is presented.
- Authentication (user identification, login, etc.)
- Persistence: storing the data long term (typically in some form of database).
Requests and responses
- In HTTP, they are streams of bytes.
- Any decent framework will present them to the application as objects, with methods to query requests (headers, parameters, etc.) and set attributes of responses (headers, content, etc.).
Other (non-Python) web “frameworks”:
- Java servlets and JSP
- ASP.NET
- Ruby on Rails, “favoring convention over configuration”
- PHP, a language, not a framework, but one primarily designed for embedding in HTML
- Haskell: Happstack, Yesod
Choices: how to mix script code with HTML
- Scriptlet in HTML (like PHP, JSP) or HTML in script (like Java servlets)?
- Print statements to write HTML or objects or functions that represent HTML elements?
  - Example: servlets or JSP
- Examples:
  - BlazeHtml
  - What is PHP?
  - Other?

Towards Greater Interactivity

Java and JavaScript enable programs to run in the web browser, avoiding the need to send a request to a server and wait for the server to respond.

Ajax (Asynchronous JavaScript and XML) allows the browser to exchange data with the server “in the background”; Comet enables the web server to push streaming data.

References

Top 10 web application security risks for the year 2013 (PDF)
HOWTO Use Python in the web provides a nice overview of past and present methods of using Python on the web server side, including CGI, mod_python, FastCGI, WSGI, middleware, MVC, templates, data persistence (through databases or otherwise), and frameworks.
PEPs for WSGI:
- PEP 333 – Python Web Server Gateway Interface v1.0 is superseded by PEP 3333.
- PEP 444 – Python Web3 Interface, a related proposal.
- PEP 3333 – Python Web Server Gateway Interface v1.0.1 replaces 333.
Bottle tutorial
Some web frameworks make extensive use of decorators. Decorators are higher-order functions: a function applied to a function, returning a (modified) function.
- PEP 318 explains the arguments about the syntax (@ was victorious), but little about the meaning
- Python decorators gives a little more explanation about the meaning
- Annotations are a related concept from Java
- Decorator pattern in Wikipedia
Happstack Philosophy contains links to some articles about “scaling” web services to very high performance levels
- Happstack is a Haskell web framework.
- Much of the articles on scalability come from High Scalability
  - See Start here <!– This post has gone missing:
- Proxying Happstack describes how to “proxy” a Happstack server through an Apache web server, but the ideas could be extended to Python servers (e.g., bottle or the reference WSGI server) as well. –>

Revisions:
- 2015 Nov 20. Add “TO DO” section.
- Version 2.2.3, 2015 Oct 26. Remove obsolete reference.
- Version 2.2.2, 2014 Oct 30. De-emphasized bottle.
- Version 2.2.1, 2013 Nov 30. Updated Top 10 security link.
- Version 2.2, 2013 Oct 31.
- Version 2.1, 2011 Nov 26. Added links to top 10 security risks and “HOWTO Use Python in the web”.
- Version 2, 2011 Nov 17.
- Version 1, 2011 Nov 10.
↩