CherryPy for CGI programmers

Document options Document options requiring JavaScript are not displayed Sample code Rate this page Help us improve this content

Level: Introductory

Leonard Richardson (leonardr@segfault.org), Software Engineer, CollabNet

16 Aug 2005

The CherryPy application framework for Python makes Web applications easier to write than plain Common Gateway Interface (CGI). At the same time, it's simple -- not full of little-used features -- and easy to learn. This introduction shows everything needed to write Web applications with CherryPy.

For more than 10 years, Web programmers have used the Common Gateway Interface (CGI) to connect applications to Web servers and the Web browsers on the other side. CGI has much to recommend it: It can be used with any programming language, and it's supported nearly universally across Web servers and hosting services. Unfortunately, CGI also has serious drawbacks. The interface between the Web server and a CGI script is somewhat convoluted. In addition, Web servers spawn a separate process for every CGI request, which means poor performance and no persistence across requests.

Over the years, dissatisfied hackers have created alternate ways to bridge this gap between Web servers and application code. In recent years, some popular ways of doing this have included Java™ servlets, the Ruby on Rails framework, and the Apache modules mod_perl and mod_python.

These bridges are so numerous that picking one can be difficult, and the problem of overabundance is especially acute in the Python world. Some of the server application bridges are full-fledged application frameworks, with their own templating systems, authentication services, object-relational mappers, and other such features. With so many choices offering so many features to learn, it's not surprising that programmers with no free time stick to what they already know.

This article introduces CherryPy, a simple but very usable Web framework for Python. All it does is connect the Web server to your Python code with as little fuss as possible. It doesn't make decisions about what other tools to use, so you're free to pick a templating system, database mapper, or other tool on its own terms. I'll show you how to write applications that use CherryPy. The article assumes you have some knowledge of Python and of how HTTP requests and responses work.

A CherryPy request

Instead of relying on Apache or another Web server, CherryPy runs its own small Python-based Web server. A traditional Web server creates a Web space out of a tree of directories disk, but the CherryPy server creates its Web space out of a tree of Python objects.

Consider a request for the URL http://localhost:8080/hello/. In a traditional Web server, this URL corresponds to the hello/ directory beneath the root of the Web space. When you access it with a Web browser, the Web server reads the hello/ directory's index.html file or invokes that directory's index.cgi script as a CGI and sends you the output.

Instead of serving a Web space rooted in a directory on disk, the CherryPy Web server serves a Web space rooted at a particular Python object, cpg.root. Each method and member of that object is accessible by tacking its name onto the root URL. Therefore, the hello/ URL corresponds in CherryPy to the hello member of cpg.root.

Serve a request by defining a method

If cpg.root.hello is a method, CherryPy calls that method, and the output of the method is sent to the Web browser. The following code defines an object that exposes a hello method:

#!/usr/bin/env python
from cherrypy import cpg

class Application:
    @cpg.expose
    def hello(self):
        return "Hello, world!"
    
cpg.root = Application()
cpg.server.start()

Run this script through Python, and the CherryPy Web server will start up. As long as you keep the script running, you can access http://localhost:8080/hello/ and be served the string Hello, world!. (Of course, this assumes you don't already have a server running on port 8080.) If you visit any other URL, including the Web server root, you'll get a CherryPy error; /hello/ is the only URL this application knows how to serve.

Serve a request by defining an object tree

If cpg.root.hello is an object instead of a method, when a user accesses /hello/, CherryPy calls the hello object's index() method. This code serves the /hello/ URL the same way as the previous example:

#!/usr/bin/env python
from cherrypy import cpg

class HelloWorld:
    @cpg.expose
    def index(self):
        return "Hello, world!"

class Application:
    hello = HelloWorld()
    
cpg.root = Application()
cpg.server.start()

When you visit the /hello/ URL, your request is mapped to the object cpg.root.hello, and its default method (index()) is called to handle the request.

Back to top

Expose objects and methods

What's the point of the @cpg.expose decorator on the hello and index methods? It tells CherryPy that it's OK to call that method in response to a Web request.

In a Web site that serves static files, it's assumed that every file and directory in the Web space is intended for public consumption. There are exceptions, though. For instance, Apache won't serve hidden files, such as .htaccess, on UNIX® systems.

When you expose an object tree as a CherryPy URL space, the assumption goes the other way: Unless you explicitly label a method as exposed, it isn't served to outside users. Think of the distinction between public and private class members in many programming languages (and enforced unofficially by the _method() convention in Python). A well-designed CherryPy class will probably have only a few public methods exposed to Web clients, but it may have many internal methods a client shouldn't be allowed to access directly.

I think the decorator is the best way to expose a method to CherryPy, but you can also expose a method by setting its exposed member to True:

class HelloWorld:
    def index(self):
        return "Hello, world!"
    index.exposed = True

Decorators don't exist in versions of Python prior to version 2.4, so the exposed trick may be the only way for you to expose methods to CherryPy.

Back to top

Gather user input

When a user submits a form or otherwise provides information to a CGI script, the Web browser gathers that information and passes it to the script through environment variables. It's the script's responsibility to parse and make sense of this information, even though a set of conventions governs the proper format and use of this information. CherryPy uses these informal conventions to eliminate several steps of this process, providing user input as the arguments to the Python method it decides to call.

Turn a query string into keyword arguments

Consider a URL that has query arguments, like http://localhost:8080/hello/?what=hello&who=world. The user may have clicked this URL or submitted it as the result of filling out an HTML form. A traditional CGI-based Web server would pass what=hello&who=world to the CGI script in the QUERY_STRING environment variable. The CGI script would be responsible for fetching that variable and parsing the string. Python's cgi module does the parsing for you. But with CherryPy, you don't have to do anything. The CherryPy Web server automatically transforms a URL's query string into a set of keyword arguments.

class Application:
    @cpg.expose
    def hello(self, what='Hello', who='world'):
        return '%s, %s!' % (what, who)

When a user hits the /hello/ URL, CherryPy turns any what and the who in the query string into arguments to the hello() method. The transition from URL to Python method call is totally transparent to you. It doesn't even matter whether the original HTTP request came in through the GET or the POST method.

Turn extra path portions into positional arguments

The other source of user input for CGI scripts is the PATH_INFO environment variable. It's commonly used to make a Web application's URLs look more like real Web pages. For instance, consider the URL http://localhost:8080/hello/world/. If /hello/ designates a CGI script, then visiting /hello/world/ invokes that script with the PATH_INFO environment variable set to /world/.

In a CGI environment, it's the CGI script's responsibility to parse the extra path information. But as with query string arguments, CherryPy takes care of the parsing for you based on typical usage conventions. Whereas query string key-value pairs become keyword arguments in a CherryPy application, extra path info arguments become positional arguments to an object's default() method:

class Hello:
    @cpg.expose
    def default(self, who):
        return 'Hello, %s!' % who

class Application:
    hello = Hello()

cpg.root = Application()
cpg.server.start()

The hello/ portion of the /hello/world/ URL gets mapped to cpg.root.hello, an instance of Hello. The Hello object has no method or member object called world, so the world portion of the URL is passed in as a positional argument to that object's default() method.

Back to top

Read headers from the request object

CherryPy takes care of parsing the URL the user requested and dispatching to a Python method with the appropriate arguments, but an HTTP request is more than just a URL. What about the incoming HTTP headers?

Your CherryPy methods have access to an object called cgp.request, which contains a lot of information about the user's HTTP request. The most interesting member of this object is requestMap, which contains all the incoming HTTP headers associated with a Web request:

class Application:
    @cpg.expose
    def index(self):
        items = [x + ': ' + y for x,y in cpg.request.headerMap.items()]
        return "<br />".join(items)

Run this application and visit http://localhost:8080/, and you'll see a listing of all the HTTP headers your browser sent along with its request.

Back to top

Write to the response object

As with the HTTP request, so with the response. A CherryPy application method typically returns the body of the response as a string, but sometimes you need to set additional HTTP headers, do a redirect, or change the HTTP response code. You can do all of these things with the cpg.response object made available to each method.

cpg.response.headerMap is a map of outgoing HTTP headers, just as cpg.request.requestMap is a map of incoming headers:

#!/usr/bin/env python
from cherrypy import cpg

class Application:
    @cpg.expose
    def setHeader(self, header, value):
        """Hit the '/setHeader?header=Value&foo=bar' URL to get a
	response in which the HTTP header "foo" has a value of
	"bar"."""
        cpg.response.headerMap[header] = value
        return 'Set HTTP response header "%s" to "%s"' % (header, value)

The HTTP status code is just an HTTP header called Status, so you can set it to 404, 503, or whatever other status you need:

    @cpg.expose
    def forbidden(self):
        "Hit the '/forbidden' URL to be denied access."
        cpg.response.headerMap['Status'] = '503 Forbidden'
        return "You don't have permission to access this resource."
        

To do an HTTP redirect, you can manually set the Status and Location headers, or you can use the redirect method of CherryPy's httputils helper library, which does the same thing:

    @cpg.expose
    def redirect(self):
        "Hit the '/redirect' URL to be redirected."
        from cherrypy.lib import httptools
        httptools.redirect('./destination')

    @cpg.expose
    def destination(self):
        "This is where you end up if you hit the '/redirect' URL."
        from cherrypy.lib import httptools
        cpg.response.headerMap['Content-Type'] = 'text/plain'
        return 'Here is some plain text.'

cpg.root = Application()
cpg.server.start()

Back to top

Keep persistent information in a session

Consider an application in which I can hit the URL /name/set?name=leonardr to set a bit of data. I can then hit the URL /name/show and be told, Your previously set name is leonardr. Because the second request used information from the first, both requests must have formed part of a single session. The string leonardr I sent on my first request was stored on the server somewhere. When I made my second request, I was somehow identified as the same person who made the first request, and the information I sent along with the first request was retrieved.

CherryPy hides most of this complexity and makes it easy to set up and use cookie-based sessions -- an important feature, but not one supported out of the box by CGI. You can store any Python object in the cpg.request.sessionMap map, and it will be there the next time the same user hits one of your pages. The following example works just like the application just described:

#!/usr/bin/env python
from cherrypy import cpg

class Application:
    @cpg.expose
    def set(self, name):
        cpg.request.sessionMap['name'] = name
        return 'Set name to %s' % name

    @cpg.expose
    def show(self):
        return 'Your previously set name is %s.' % \
               cpg.request.sessionMap.get('name', '[none]')
               

Pretty simple so far. However, for this code to work, the CherryPy server needs to be set up to associate a session cookie with each HTTP response. Otherwise, CherryPy will never be able to associate two requests with the same session. You can do this configuration in a CherryPy server configuration file (see Resources), but it's easier to demonstrate it as part of the Python code that starts the CherryPy Web server:

cpg.root = Application()
cpg.server.start(configMap={'sessionStorageType' : 'ram',
                            'sessionCookieName' : 'CherryPySessionCookie',
                            'sessionTimeout' : 60}) #Session expires in an hour

Back to top

Conclusion

CherryPy uses the same concepts as CGI to bind a Web server to a Web application, but it improves performance and gains persistence across requests by handling all its requests within a single process. Because it binds only to Python code, it doesn't need the often obscure information-passing techniques of CGI. This leads to fewer lines of more understandable code. CherryPy makes an excellent replacement for CGI and a good base on which to build Python Web applications.

Back to top

Download

	Information about download methods

Resources

• The Web Framework Shootout compares many popular Python Web frameworks, including CherryPy.
  
  
• The CherryPy Web site has a basic tutorial, a number of more advanced recipes, and a list of third-party add-ons.
  
  
• Unlike frameworks like mod_python that make you mount applications onto URLs served by Apache, CherryPy connects to Apache through rewrite rules. In the default setup, both the CherryPy server and Apache are running on your machine at once, and Apache requests are proxied to CherryPy requests. This sounds more complicated, but it's actually easier to set up than mod_python. To learn how to serve CherryPy requests through Apache, visit the CherryPy Wiki.
  
  
• The simple session example given in the article should work as is, but if you're doing more complex operations with a user's session, you should lock the session to make sure two simultaneous requests from the same user don't put the session into an inconsistent state. The "Locking sessions" section of this Wiki will help you with this.
  
  
• The format of the CherryPy Web server config file and the options you can put into it are described on this Wiki.
  
  
• Get started with Python with the developerWorks article "Python 101."
  
  
• See "Charming Python: Review of Python IDEs" to learn more about Python tools.
  
  
• "Discover Python, Part 1: Python's built-in numerical types" is the first in a series written for Java developers.
  

About the author

		Leonard Richardson is the author of many Python applications and libraries, including NewsBruiser and Beautiful Soup. He is a co-author of the new tome Beginning Python, from Wrox.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Back to top