Python Web frameworks, Part 2: Web development with TurboGears and Python

Document options Document options requiring JavaScript are not displayed Rate this page Help us improve this content

Level: Intermediate

Ian Maurer (ian@itmaurer.com), Senior Consultant, Brulant, Inc.

11 Jul 2006

In this second article of a two-part series, we demonstrate TurboGears, another open source MVC-style Web application framework based on Python. Where the first article was an introduction to the Django framework, this article shows how to use TurboGears to create a Web-based shopping application and concludes with a comparison between Turbogears and Django.

The TurboGears developers call this project a "megaframework," because it is made up of several pre-existing subprojects. TurboGears helps glue together a number of main components:

• MochiKit: JavaScript library
• Kid: Templating language
• CherryPy: Base Web framework
• SQLObject: Object-relational mapper (ORM)

Installing TurboGears

The first step in working with TurboGears is ensuring that you have Python installed. The latest version of TurboGears requires Python 2.4. See the Resources section for a link to the Python home page.

Setuptools is a new project from the Python community that makes installing and updating Python software much easier. Software is packaged in an archive file called an Egg, similar to a Java™ JAR file or a Ruby GEM file, and is downloaded and installed by a tool called easy_install.

TurboGears is one of the first large Python projects to use setuptools for distribution and installation. See Resources for links to the setuptools page and the TurboGears download page.

This article uses the latest development version at the time of this writing (0.9a5), retrieved through the preview download page:

easy_install -f http://www.turbogears.org/preview/download/index.html TurboGears

To retrieve the latest stable version of TurboGears (currently 0.8.9), simply remove the -f option and the URL to the preview site:

easy_install TurboGears

Checking for the TurboGears admin tool

After installing TurboGears, you should have the admin tool, tg-admin, available to you on your path. Check by typing tg-admin by itself on the command line:

~/dev$ tg-admin

TurboGears 0.9a5 command line interface

Usage: /usr/bin/tg-admin [command] [options]

Available commands:
      i18n  Manage i18n data
      info  Show version info
quickstart  Create a new TurboGears project
     shell  Start a Python prompt with your database available
       sql  Run the SQLObject manager
   toolbox  Launch the TurboGears Toolbox
    update  Update an existing turbogears project

Back to top

Quick start

To begin a TurboGears project, use the tg-admin quickstart function. You'll be asked to provide both a project and directory name. For this article, I create a simple shopping cart example called TG Commerce, which has a package name of tgcommerce:

~/dev$ tg-admin quickstart
Enter project name: TG Commerce
Enter package name [tgcommerce]:
Selected and implied templates:
  turbogears#turbogears  web framework

Variables:
  package:  tgcommerce
  project:  TG-Commerce
Creating template turbogears
  Creating ./TG-Commerce/

    ... (output snipped) ...

TurboGears creates a TG-Commerce project directory with a tgcommerce package below it. You can now start the test server that comes with your project:

~/dev/TG-Commerce$ python start-tgcommerce.py

... (output snipped) ...

05/Mar/2006:11:31:54 HTTP INFO Serving HTTP on http://localhost:8080/

View the test page at the URL provided, then kill the server using Ctrl-C.

Back to top

Creating a model

SQLObject is the object-relational mapper (ORM) library that allows you to develop database-persistent Python objects. You define a Python class, add the desired attributes (fields), and let SQLObject handle the generation of the SQL needed to create tables, insert new records, and find, update, or delete existing records.

SQLObject supports several databases, including MySQL, PostgreSQL, Firebird, and others. You can find more information about SQLObject by following the link in Resources.

In this example, I use SQLite as the database backend. SQLite is a lightweight database that requires no configuration and resides on disk as a simple file. To use SQLite, install the pysqlite library with setuptools:

easy_install pysqlite

To configure the TurboGears database, specify the sqlobject.dburi in the dev.cfg file. For SQLite, specify the path where you want the database file to reside:

sqlobject.dburi="notrans_sqlite:///path/to/devdir/TG-Commerce/tgcommerce.database"
server.environment="development"
autoreload.package="tgcommerce"

The TurboGears quickstart creates and pre-populates a model.py file with boilerplate code. This is where SQLObject classes should be placed. At the top is the section that sets up the database connection hub:

from sqlobject import *
from turbogears.database import PackageHub

hub = PackageHub("tgcommerce")
__connection__ = hub

Next are the model classes. Each one represents a table in the database, defined with class-level attributes that map to database columns. These attributes are instances of SQLObject column types, which include basic data types, such as StringCol and CurrencyCol, and relationship types, such as ForeignKey and MultipleJoin.

For this shopping cart, there is a hierarchical Category class and a simple Product class. The category hierarchy is defined by the parent ForeignKey and the subcategories MultipleJoin.

class Category(SQLObject):
    name = StringCol(length=64)
    parent = ForeignKey('Category', default=None)
    subcategories = MultipleJoin('Category', joinColumn='parent_id')
    products = MultipleJoin('Product')

class Product(SQLObject):
    name = StringCol(length=64)
    sku = StringCol(length=64)
    price = CurrencyCol(notNone=True, default=0.0)
    category = ForeignKey('Category')

To verify the model, use the tg-admin sql sql command to display the SQL code that will create the tables needed. Note that SQLObject creates an id column for each table. This happens when no primary key is defined. See Resources for links to more documentation on SQLObject.

~/dev/TG-Commerce$ tg-admin sql sql
Using database URI sqlite:///home/ubuntu/dev/TG-Commerce/tgcommerce.db
CREATE TABLE category (
    id INTEGER PRIMARY KEY,
    name VARCHAR(64),
    parent_id INT
);

CREATE TABLE product (
    id INTEGER PRIMARY KEY,
    name VARCHAR(64),
    sku VARCHAR(64),
    price DECIMAL(10, 2) NOT NULL,
    category_id INT
);

Below are the Order and OrderItem classes that are used for storing the shopping cart and its items. Since order is a SQL keyword, the table name for the Order class is redefined to orders using the sqlmeta subclass's table attribute:

class Order(SQLObject):
    items = MultipleJoin('OrderItem', joinColumn='order_id')

    class sqlmeta:
        table = 'orders'

class OrderItem(SQLObject):
    quantity = IntCol(notNone=True)
    price = CurrencyCol(notNone=True)
    total = CurrencyCol(notNone=True)
    order = ForeignKey('Order')
    product = ForeignKey('Product')

Use the tg-admin sql create command to create the database tables:

~/dev$ tg-admin sql create
Using database URI sqlite:///home/ubuntu/dev/TG-Commerce/tgcommerce.db

You can find more commands using tg-admin sql help, including a command to drop tables. Use extreme care when using this command, since it will delete all of your tables along with their data. For this reason alone, the tg-admin command should be locked down in a production environment.

Back to top

Manipulating the model with CatWalk

Starting with version 0.9, TurboGears comes with a set of tools called Toolbox, which includes a model browser called CatWalk. CatWalk is designed for developers who want to quickly create, update, and delete data for their model using a GUI tool.

Toolbox starts as a separate server and is run using the tg-admin command:

~/dev/TG-Commerce$ tg-admin toolbox
... (snip) ...
05/Mar/2006:15:01:33 HTTP INFO Serving HTTP on http://localhost:7654/

If a browser doesn't open up automatically, navigate to the URL (http://localhost:7654/) specified by the Toolbox server, and click the CatWalk link to open CatWalk.

Toolbox is geared toward developers, not end-users, and is best used for aiding in data modelling and bootstrapping your application with data. You can shut down the toolbox server with Ctrl-C. You won't use it in this walk-through.

Back to top

Creating a view

The default way of creating views in TurboGears is by using the Kid XML templating language. Since Kid uses XML, all the templates must be well formed, or they will throw an exception during rendering. Kid also supports template inheritance, in which derived templates can extend from base templates so that common code can be created and maintained in one location.

In TurboGears, Kid files are located in the templates directory with .kid extensions. By default, there is a master.kid file and a welcome.kid file, where master.kid is the base template file, and welcome.kid inherits from it using the py:extends attribute on the <html> tag.

To create a new template, I recommend that you copy or rename the welcome.kid file and use that as your starting point. The category template was first created for this example, and it displays the following information about a given category:

• Category name (title and breadcrumb)
• Links to ancestors (breadcrumbs)
• Links to subcategories (list)
• Links to products (list)

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
    xmlns:py="http://purl.org/kid/ns#"
    py:extends="master.kid">

<head>
    <title>Category: ${category.name}</title>
</head>

<body>
	<div id="breadcrumbs">
		<ul>
			<li py:for="ancestor in list(category.ancestors)[::-1]">
				>
				<a href="/category/${ancestor.id}"
						py:content="ancestor.name"></a>
			</li>
			<li id="active">
				> <span py:replace="category.name" />
			</li>
		</ul>
	</div>

	<div id="subcategories" py:if="category.subcategories">
		<div class="header">Subcategories</div>
		<ul>
			<li py:for="subcategory in category.subcategories">
				<a href="/category/${subcategory.id}"
						py:content="subcategory.name"></a>
			</li>
		</ul>
	</div>

	<div id="subcategories" py:if="category.products">
		<div class="header">Products</div>
		<ul>
			<li py:for="product in category.products">
				<a href="/product/${product.id}"
						py:content="product.name"></a>
			</li>
		</ul>
	</div>
</body>
</html>

Listing 11 show some of the key Kid functionality:

• Inheriting from master.kid using py:extends
• Expression substitution in the title using ${category.name}
• Looping through ancestors, subcategories, and products using py:for
• Using slicing logic to reverse the ancestor display in the breadcrumbs
• Filling a link's text using py:content
• Replacing a span tag completely with py:replace

For the above referenced ancestors attribute to be valid, the Category model class needs to be modified to include a _get_ancestors method:

class Category(SQLObject):
    name = StringCol(length=64)
    parent = ForeignKey('Category', default=None)
    subcategories = MultipleJoin('Category', joinColumn='parent_id')
    products = MultipleJoin('Product')

    def _get_ancestors(self):
	ancestor = self.parent
	while ancestor:
	    yield ancestor
	    ancestor = ancestor.parent

Back to top

Creating a controller

TurboGears quickstart provides a project with a controllers.py module that is the location of the Root controller class. This is the main entry point for your application and the place to add new controller methods.

Below are two sample controller methods associated with the category HTML template through the TurboGears expose decorator. Controller methods return dictionaries, which are used as a namespace, or context, during the rendering of the specified Kid template.

from turbogears import controllers, expose

class Root(controllers.Root):

    @expose("tgcommerce.templates.category")
    def index(self):
	from model import Category
	category = Category.selectBy(parent=None)[0]
	return dict(category=category)

    @expose("tgcommerce.templates.category")
    def category(self, categoryID):
	from model import Category
	category = Category.get(categoryID)
	return dict(category=category)

In TurboGears, URLs cleanly map to the methods and are contained within the Root controller. The root URL / maps to a special method called index. By adding a method called category to Root, it can be accessed through the URL /category. Any URL submitted that doesn't match a given method causes a 404 error, unless a default method is defined.

Below are some possible URL scenarios and their results:

• /: displays the first category without a parent id
• /category?categoryID=2: displays the category with an id of 2.
• /category/1: displays the category with an id of 1 (format available as of TG 0.9)
• /category: throws a 500 error, due to the missing category id.
• /xyz: throws a 404 error

Figure 2 shows the category display page:

Back to top

Product display

For the product display page, a product controller is created that retrieves a product from the database and passes it to the product Kid template for rendering.

@expose("tgcommerce.templates.product")
def product(self, productID):
    from model import Product
    product = Product.get(productID)
    return dict(product=product)

The product.kid template has a table of product information. Notice the use of the Python string formatting to display a price with two significant digits:

<table id="product-info">
  <tr>
    <td class="fld">Name:</td>
    <td class="val"><span py:replace="product.name" /></td>
  </tr>
  <tr>
    <td class="fld">SKU:</td>
    <td class="val"><span py:replace="product.sku" /></td>
  </tr>
  <tr>
    <td class="fld">Price:</td>
    <td class="val">$<span py:replace="'%.2f' % product.price" /></td>
  </tr>
</table>

Figure 3 shows the product display page:

Back to top

Error handling

The one thing the controller methods haven't taken into account is the SQLObjectNotFound thrown by the SQLObject get methods. Listing 16 shows a refactoring that catches this exception and rethrows it as a NotFound exception, which sends a basic HTTP 404 error:

from model import Category, Product
from sqlobject import SQLObjectNotFound
from cherrypy import NotFound
from turbogears import controllers, expose, url

class Root(controllers.Root):

    @expose("tgcommerce.templates.category")
    def category(self, categoryID):
	try:
	    category = Category.get(categoryID)
	except SQLObjectNotFound:
	    raise NotFound()
	return dict(category=category)

    @expose("tgcommerce.templates.product")
    def product(self, productID):
	try:
	    product = Product.get(productID)
	except SQLObjectNotFound:
	    raise NotFound()
	return dict(product=product)

Another strategy for handling missing objects, rather than sending a 404 error, is to redirect instead. This is done using the turbogears.redirect(...) method:

from turbogears import redirect
try:
    object = ClassName.get(objectID)
except SQLObjectNotFound:
    raise redirect("/path_to_redirect")

Back to top

Shopping cart

In order to have a working shopping cart, you need to have sessions enabled to maintain state across requests. Sessions are enabled in the configuration file by setting the session_filter.on value to True.

session_filter.on = True
sqlobject.dburi="notrans_sqlite:///path/to/devdir/TG-Commerce/tgcommerce.database"
server.environment="development"
autoreload.package="tgcommerce"

To display the current shopping cart on each page, you can place the HTML in the master template file, rather than copying and pasting it to each page:

<html>
  <body>
  ...
	<div id="shopping-cart">
		Shopping Cart:
		<span id="cart-qty" py:content="cart.total_quantity" /> items
		($<span id="cart-amt" py:content="'%.2f' % cart.total_price" />)
	</div>
  </body>
</html>

The above template change, by itself, will fail, since there isn't a cart value in the return dictionaries from the controller methods. Listing 20 shows the change to the products method that will include the cart currently stored in the session.

@expose("tgcommerce.templates.product")
def product(self, productID):
    try:
	product = Product.get(productID)
    except SQLObjectNotFound:
	raise NotFound()
    return self.results(product=product)

def results(self, cart=None, **kw):
    if not cart:
	cart = self.get_cart()
    kw['cart'] = cart
    return kw

def get_cart(self):
    cartID = session.get("cartID", None)

    cart = None
    if cartID:
	try:
	    cart = Order.get(cartID)
	except SQLObjectNotFound:
	    pass

    if cart is None:
	cart = Order()
	session["cartID"] = cart.id

    return cart

The results method calls get_cart, which will retrieve the current Order object or create a new one, if one doesn't exist or cannot be found. This change should be in the index and category methods as well.

Back to top

Ajax add to cart

To make this shopping cart example buzzword compliant, the product add-to-cart function is done with an Ajax call without refreshing the page. For the uninitiated, Ajax is a Web implementation pattern that stands for Asynchronous JavaScript + XML. The article that initially defined the Ajax term is listed in Resources.

This example uses JavaScript Object Notation (JSON), rather than XML, as the transport format. JSON is more lightweight and supported natively by both TurboGears and MochiKit. Even though using asynchronous JavaScript with JSON is gaining popularity, its use will likely retain the Ajax moniker for the simple fact that the acronym Ajaj is not as catchy.

Below is the controller method add_to_cart, which gets a specified product and adds it to the cart currently in the session. The method returns the cart object along with the total price and quantity of the items in the cart:

@expose()
def add_to_cart(self, productID):
    cart = self.get_cart()

    product = Product.get(productID)
    cart.add_product(product, 1)

    return self.results(cart=cart,
	       price=cart.total_price,
	       quantity=cart.total_quantity)

Notice that the expose method has not been supplied with a template name. This is because the method is not going to be called directly from a browser and rendered in HTML. If you do view the add_to_cart page (http://localhost:8080/add_to_cart/1) directly in the browser or with a tool like curl, you will be given the raw JSON data: {"tg_flash":null, "price":24, "cart":{"id":24}, "quantity":1}.

The JavaScript call from the client (browser) to the add_to_cart controller method on the server is very simple. MochiKit provides a function called loadJSONDoc that takes a URL and returns what is known as a deferred object. You can use this deferred object for defining a callback method when the asynchronous call returns a response.

function addToCart(productID) {
	var deferred = loadJSONDoc('/add_to_cart/' + productID);
	deferred.addCallback(updateCart);
}

function updateCart(results) {
	$("cart-amt").innerHTML = numberFormatter("#.00")(results["price"]);
	$("cart-qty").innerHTML = results["quantity"];
}

The updateCart function is called upon the return, and the JSON values are supplied in an associative array variable called results. The function updates the cart's amount and quantity using the ID and the innerHTML attribute. On top of that, the handy MochiKit numberFormatter function is used to specify the significant digits of the price.

The last step is then to add a link that calls the above addToCart function, passing the current product ID. You also need to add the MochiKit.js and your cart.js files as script tags to your product template:

<head>
    ...
    <script src="/tg_js/MochiKit.js" type="text/javascript" />
    <script src="/static/javascript/cart.js" type="text/javascript" />
</head>
<body>
    ...
    <table>
	...
		<tr>
			<td colspan="2" align="right">
				<a href="#" onclick="addToCart(${product.id})">Add To Cart</a>
			</td>
		</tr>
	</table>
</body>

Back to top

Conclusion: Comparing TurboGears and Django

Django and TurboGears are both MVC-style frameworks that allow for the agile and rapid development of Web sites using the Python language. To choose the best one for your needs, consider these differences:

• Background:

  Both projects, like Ruby on Rails, were extracted from existing applications and released to the open source community. Django has been around longer and originally came from an online newspaper that serves millions of page views per day. TurboGears was pulled from a rich-client, RSS News Reader application that is still under development. TurboGears is more community-driven than Django because it was built with pre-existing, open source components.

  The different backgrounds of each project have led to different project priorities. The Django team, coming from the high-demand, fast-paced world of online journalism, have focused on a framework that allows content-based applications to be constructed quickly and modified easily. The TurboGears team, with its consumer-product foundation, has geared itself toward rich client applications and a pluggable architecture.

• URLs:

  TurboGears' request dispatch mechanism is routed via controller class and method names. Once you add a new controller class or method, it becomes automatically available. If you need to change the path that executes a given controller, you need to reconfigure your code structure. Conversely, Django uses a separate regular expression-based configuration file to map URLs to the underlying code, decoupling the URL path structure from the actual implementation.

  The TurboGears system is quicker to set up than Django, since it requires only an expose decorator to make new pages available. However, the Django configuration system allows for maximum control and flexibility. Django URLs can be easily remapped onto an application after a major refactoring. This helps prevent "link rot" caused by old bookmarks or cached search engine results. "Link rot" severely hurts the traffic levels and usability of content-based Web sites that Django was designed to create.

• Code reuse:

  The TurboGears team calls their project a megaframework to clearly express how TG is a project made up of existing components. The TurboGears team selects and incorporates the best available open source code, rather than writing it from scratch. A side benefit of the TurboGears framework is that it is a megaproject with a megacommunity. TG has become a powerful, central force, driving interest and involvement into the core components that make up TurboGears. It is the tide that raises all ships.

  Django, on the other hand, was created back in 2003 when the state of existing Python components wasn't as solid as it is today. The Django Web stack was created from scratch, and the end result is a stable framework that has been used to create several Web sites that handle millions of hits per day. Some have postulated, however, that the Django project may suffer from the NIH (Not Invented Here) Syndrome due to its lack of code reuse. The position of the Django team is that the work needed to create a framework in Python from scratch is not any more difficult than gluing existing components together and that the end result is a more uniform and cohesive framework.

• JavaScript:

  TurboGears has given MochiKit, a JavaScript library, a first-class position in its framework. The team has also created a widget library that makes extensive use of JavaScript to create "rich" form elements. This shows how important rich-client (Ajax) development is in the TurboGears world. The Django team has not chosen a JavaScript library to include with their framework by default, but has discussed the possibility. Neither project restricts you, in any way, from using any JavaScript libraries.

• Administrative tools:

  Both projects have admin interfaces. The Django admin tool's target audience is the end user who needs a well polished data entry tool so that custom tools are not needed every time new functionality is added to an application. The TurboGears admin tool, on the other hand, focuses on developers by giving them a set of design tools along with a basic database viewer and editor.

• License:

  Because Django was created from scratch, the entire project is under one open source license (the BSD license). TurboGears, which is made up of multiple projects, has several licenses. SQLObject, the ORM tool, is protected by the LGPL (Lesser General Public License), which states that if the source code of SQLObject library itself is changed and distributed, those changes need to be made available. The license does not require applications that use it to be open sourced. Some companies block the use of software licensed under the LGPL, regardless. In those cases, you may consider using SQLAlchemy, another ORM tool with strong support in the TG community.

• Real world examples:

  See the Resources section for links to pages that list known Django- and TurboGears-powered sites. These live applications demonstrate what can be done with each tool.

Resources

• "Develop for the Web with Django and Python" (developerWorks, June 2006), Part 1 of Ian's two-part series, introduces Django.
  
  
• The Python Tutorial will get you started with Python.
  
  
• The SQLObject home page provides more information on the default TurboGears object-relational mapping (ORM) tool.
  
  
• Read Ajax: A New Approach to Web Applications, the article that kicked off the Ajax craze.
  
  
• Learn more about Ajax on developerWorks.
  
  
• Building and Distributing Packages with setuptools shows how to install setuptools, along with easy_install, part of the Python Eggs package.
  
  
• Trac lists Web sites using TurboGears.
  
  
• The Django project lists Web sites using Django.
  
  
• In the developerWorks Linux zone, find more resources for Linux developers.
  
  
• Stay current with developerWorks technical events and Webcasts.

• Python.org is the home of the Python programming language, where you can find links for downloading the Python interpreter and standard libraries.
  
  
• TurboGears.com is the home page for the TurboGears framework.
  
  
• The TurboGears preview site is the current home for the 0.9 alpha release.
  
  
• Kid is an XML-based template language for TurboGears.
  
  
• CherryPy is a pythonic, object-oriented Web framework that provides the base for the TurboGears framework.
  
  
• MochiKit is a well-tested JavaScript library that provides DOM and Ajax support.
  
  
• JSON (JavaScript Object Notation) is a lightweight data-interchange format used by MochiKit and TurboGears in asynchronous messaging.
  
  
• Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®.
  
  
• With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.
  
  

• Check out developerWorks blogs and get involved in the developerWorks community.
  
  

About the author

		Ian Maurer is a senior consultant for Brulant, Inc., where he specializes in developing integrated e-commerce solutions using open source and IBM WebSphere technologies for various industries including consumer goods and retail. Ian resides in northeastern Ohio and is a member of the Cleveland Area Python Interest Group.

Rate this page

Please take a moment to complete this form to help us better serve you. Did the information help you to achieve your goal? Yes No Don't know   Please provide us with comments to help improve this page:   How useful is the information? 1 2 3 4 5 Not useful Extremely useful

Back to top