 | Level: Intermediate Peter Seebach (developerworks@seebs.plethora.net), Freelance writer, Plethora.net
27 Mar 2007 The open source release of the Second Life viewer program by
developer Linden Lab offers a rare opportunity to peer into the comparative
strengths of closed and open source development models. This article, the first in a
series gives an overview of some of the differences between these development
styles, and talks about what's involved in setting up your own build environment.
Second Life is a virtual world, maintained through a combination of client
software and hosting servers. It is somewhat similar to various "massively
multiplayer" online games (MMOs) but has the unusual quality that nearly all of
the content is user-provided. It is also unusual in that Linden Labs recently
announced the release of its client software as open source. This is something
that is rarely, if ever, done in commercial MMO apps. The specific license chosen
is the GPL, version 2, with an exception for FLOSS software.
|
More in this series
-
Part 2: Hacking Second Life:
What happens when a company releases proprietary software to the
open source community? Find out here. We cover the build process and some of
the stepping stones and stumbling blocks on the way to hacking Second Life.
-
Part 3: Adding simple translation to Second Life:
The open source Second Life client is amenable to being modified. Add a
service that lets you translate other languages without leaving the comfort of
Second Life.
|
|
In this series, I introduce the client (or "viewer" in Linden terminology) and
explore the development environment, documentation, and more. Developers who are
used to an open source environment are sometimes a little put off by things that
might be done differently in a commercial environment, and this project offers a
number of opportunities to explore some of the tradeoffs. Of course, the best way
to explore a program is to do something with it, so this series gets into the code
to make a few changes.
Second Life system requirements
The Second Life client is available for Microsoft® Windows, Mac® OS
X, and Linux® systems (and may run on others, now). I'll show how to build
and develop on Linux, and eventually cross-compile so that the client is available
for users of other platforms as well. To run the client, your system needs
reasonable 3D hardware and a moderately fast CPU; of course, faster is always
better. For Linux builds, the program requires direct rendering support in
hardware; most nVidia or ATI graphics cards are supported by some version or
another of the drivers.
Initial testing by Linden was done on Debian, Fedora Core 4, and Knoppix; I
tested on SUSE 10.1, but you are encouraged to use the distro of your choice and
can report problems to the Linden developer support forums (see
Resources for a link).
Building the Second Life client is not exactly easy or painless right now, but
that is pretty normal when a package that has historically been closed gets opened
up. And compared to, for instance, the nightmarish experience of trying to get
early Mozilla releases built, it's a walk in the park.
As an alpha release, it is also subject to change. You will find the most
up-to-date list of packages, and instructions for building, at the
Compiling the viewer (Linux)
page at the Second Life Wiki.
The instructions suggest a build time of "a couple of hours." I ran the build
several times under different CPUs and Linux installations and encountered build
times ranging from an hour and 20 minutes to just 25 minutes -- not one of which
was entirely glitch-free. Here are some of the more interesting things I
encountered:
-
Dependence on older packages. The client is known not to work with gcc 4.
You will need gcc 3.4.
-
Use of proprietary packages like fmod sound toolkit; it's "free of
charge," but it's closed source. Similarly, the client was initially only really
usable with the Kakadu JPEG implementation instead of OpenJPEG (see
Resources for more on this).
-
Reliance on non-standard packages like SCons. A replacement for GNU make,
it is available with many Linux installations but is rarely installed by
default. SCons is a cakewalk to build, but it's one more thing to do before
getting started.
-
Special or additional build requirements: For instance, the ELFIO normal
build produces a static library, and the Second Life viewer requires a shared
library. Second Life needs xmlrpc-epi, but you have to apply special patches to
it.
-
Conflicts: Speaking of xmlrpc-epi, it conflicts with xmlrpc-c. Although
it's hardly the client's fault, the accelerated 3D graphics driver I needed
broke compilation of OpenGL programs, leading to a fairly circuitous workaround.
If you're used to an open source development environment, a commercial build
process can seem pretty bad. Why are there all these dependancies? Why do I have
to set up all these things before using the software?
Part of the problem is the (justifiable, for a commercial application)
assumption that you will only try to build the viewer on machines that are
configured correctly for the build. When you control the development systems, it's
easy to ensure that every development workstation or build machine has all the
libraries you need. The thing that takes most of your time in building the OpenGL
client for the first time is setting up the environment.
You could set up the entire environment by hand, but it's a fair amount of extra
work. The existence of the local patches (in the xmlrpc-epi code) and the variety
of packages many systems don't ship with means that you're going to be doing a
fair amount of compiling of other things before the Second Life code will even
compile, let alone run. This isn't totally unheard of in building open source
projects; what's unusual is that it doesn't all get caught by the "configure"
phase, because there isn't one. That saves time per build if your system is set up
correctly, but it can cost you a lot of time if it isn't.
Once the correct environment is in place, the essential process and most common
paths are automated, so you can just push the button and walk away. The source
downloads page (see Resources) has a table showing all
the downloads for each version; the prebuilt libraries for each supported platform
are offered as separate downloads listed in the same table row.
The setup work isn't entirely automated, but it doesn't have to happen very
often, comparatively. For that matter, Linden Labs may well have an automated
system that creates a viable Second Life build environment.
The difference is that a commercial developer can usually assume complete
control over the build environment, while open source software typically has to be
able to build on a variety of platforms, reasonably easily, to achieve adoption:
When I built gcc 3.4 so I could build the Second Life client, I just ran
configure, make, and make install.
I couldn't just run configure, make, and make install to build the Second Life
client -- because it doesn't use make at all.
Meet SCons
The program used to build the Second Life viewer is called SCons, a program
intended to replace make.
Until I started working with Second Life, I'd never heard of SCons. My initial,
gut, reaction, was "oh, for crying out loud, why did they use some weird,
proprietary program instead of just using make?" That response lasted a good five,
maybe ten, minutes. By the time I'd tracked down the Web page for the project, I
realized that my gut reaction was just plain wrong. First off, SCons is not
"proprietary." It's not make, nor is it compatible with make, but you can't
possibly call a free product distributed in source with open source licensing
"proprietary."
That still leaves the question of why you'd use a program that many users don't
have (my Linux install was a "full install" and didn't have it, although I found
the RPM file on the install media) instead of a program pretty much everyone has.
There are a number of answers; frankly, SCons is technically superior to make.
Now, technical superiority isn't everything; there are a lot of advantages to
"widely available" and "well known to lots of people." However, these advantages
are much stronger in a traditional open source environment, where you want random
strangers to be able to pick up your project and hack on it, and much weaker in a
traditional corporate environment, where a week or so of training is easily
justified if it lets people do their jobs faster. For a large build (and the
Second Life viewer qualifies), the benefits of the parallel and distributed
building alone are probably sufficient to justify the learning curve.
The main thing you need to know if you're going to start messing around with the
build process is that the project file (called "SConstruct") used by SCons is a
Python program. Because SConstruct files are Python programs, they can do a lot of
internal configuration checking themselves. You don't need to run configure to
build these files; part of this is because the SConstruct program sets a lot of
build parameters correctly itself.
|
Architecture-specific optimizations
One question that might come to mind is, why isn't there code to offload 3D
rendering to, for instance, the SPEs on a Cell BE chip? In practice, the answer is
simple; there are very few user-targetable systems that have both coprocessors to
offload to, and an accessible GPU to handle the 3D rendering. For instance, Linux
on the PS3 doesn't get access to the system's GPU. For most users, the time it
would take to develop the features exceeds the performance benefits. That said,
although it's probably not commercially viable, it might make an interesting
project for a hobbyist to pursue. |
|
Reading the SConstruct file gives you a lot of insights into the build process.
The options passed on the command line are explained at least in part by comments
in the file. Of particular interest is that the suggested invocation found in the
file header builds versions for every supported platform, not just the host
platform.
The default behavior is to do a distributed build using a list of hosts embedded
in the SConstruct file. This is an interesting option to have and is probably one
of the reasons SCons was used; most versions of make don't support distributed
compilation, but if you had a team of programmers working on a project that takes
over an hour to build, you might consider distributed compilation a significant
feature.
Build
The Linden build process relies on copying files from all the many disparate
places that many Linux packages store their include files, to a unified location
in the Linden build tree. The instructions are found on the developer Wiki (see
Resources). Their sample shell commands for preparing for
the build, even if you use the prebuilt libraries, look like this:
Listing 1. Copying include files
cp -a /usr/include/atk-1.0 ${SLSRC}/libraries/i686-linux/include/
cp -a /usr/include/gtk-2.0 ${SLSRC}/libraries/i686-linux/include/
cp -a /usr/lib/gtk-2.0/include/* ${SLSRC}/libraries/i686-linux/include/gtk-2.0/
cp -a /usr/include/glib-2.0 ${SLSRC}/libraries/i686-linux/include/
cp -a /usr/lib/glib-2.0/include/* ${SLSRC}/libraries/i686-linux/include/glib-2.0/
cp -a /usr/include/pango-1.0 ${SLSRC}/libraries/i686-linux/include/
|
This assumes that ${SLSRC} refers to the location in which you've unpacked the
second life client tree. In fact, the above didn't work for me, because on my
system, all of the /usr/ paths had to be replaced with /opt/gnome/; for instance,
the gtk include files are found in /opt/gnome/include/gtk-2.0. Easy enough to fix,
but it's an example of the sort of thing that makes people write giant configure
scripts. It took a couple more tries to get it running.
There are a lot more similar instructions to follow on the build process page,
but the downloaded prebuilt libs packages lets you skip ahead to actually running
the build. Simply invoke SCons with a couple of additional options:
Listing 2: Invoking SCons
$ cd indra
$ scons DISTCC=no BTARGET=client BUILD=release
|
The additional options specify whether or not to do a distributed compile (no,
unless you want to do a lot of network preparation), which target to build (the
client is the only one you have source for), and what kind of build to do (for
instance, you could build a debug version). If you want to run more than one job
at once, just add -j 3 to the
scons command line, same as for GNU make.
The directory tree
The Second Life viewer source tree has two main directories: the "libraries"
directory, which holds support code and header files, and the "indra" directory,
which contains the source for the viewer itself, as well as Linden-provided
support libraries.
The .o files are placed in /tmp; specifically, SCons builds a directory tree in
/tmp/username in which it stores output files so as to avoid altering the build
tree more than absolutely necessary. By contrast, the binary itself is stored in
the newview/ subdirectory of the indra/ directory. The Linux binary is about
131MB, unstripped. Stripped, it's a svelte 26MB.
The documentation
As you might expect for a project being released as open source, Second Life has
a Wiki. As is usually the case, the Wiki isn't perfect, but it's a very good
starting point for documentation; I'll explore it more fully in the next article.
Conclusion
A good way to get a feel for an implementation is to see how easy or hard it is
to change it. I'll also do that in Part 2.
While there are a number of interesting quirks to the Second Life client's
design that aren't totally compatible with the average open source environment
(for instance, the Second Life client provides essentially its own GUI library,
complete with buttons and menus, instead of using an existing library -- not
unusual for game development, but definitely unusual for open source development),
the Second Life implementation seems to be fairly well organized. That speaks to a
good development model and one well suited to an open source release. Modules are
modular; that's a good thing, as I'll talk about next time.
Resources Learn
-
Read more Second Life articles on developerWorks.
- Completely new to Second Life? The
Wikipedia is often "a good
place to start," though in this case their entry will probably make more sense after you've
become a little familiar with it.
- IBM has launched a
Second Life Business group
to take advantage of the potential for virtual training, conferences and commerce,
and has worked on projects like
Virtual Sears
and
a virtual online replica of China's Forbidden City.
Some people
make a real-life living in Second Life.
-
Experience a movie trailer, instead of just watching one: the
Virtual Worlds 2007 outlines many
business opportunities in virtual reality gaming, focusing in large part on
advertising and entertainment.
-
OpenMetaverse is a
project that's been working on open source material related to Second Life.
- The
Second Life Open Source Portal
includes guides for
obtaining and compiling source for MS Windows, Mac OSX, Linux and FreeBSD
as well as
common compilation problems,
and more. You must log on (with your Second Life user name and passwords) to even
view Linden's
forums for Second Life users and developers.
- Developers will be interested in learning
more about SCons; in the
OpenSL bugtracker and
the
stepby step guide for how to add different categories of features
-- and perhaps even in
bounties -- in which
people are willing to pay for desired improvements to code.
- How does open sourced Second Life help the OSS
community? As an example, OpenJPEG is now
more usable
for everyone thanks to the community that is developing around work on the Second
Life client.
- For creating things inside Second Life, see
Using the Linden Scripting Language
(Dr. Dobb's Portal).
- What kind of tradeoffs are made in a commercial
vs. an open development environment? The biggest difference is simply that, in a
commercial environment, the build system is adapted to the needs of the program,
and in an open source environment, the program generally adapts to the needs of a
variety of host systems. Many of Peter's thoughts on the engineering tradeoffs
that tend to influence the development path of open source software were
influenced by Eric Raymond's book,
The Art of UNIX Programming
(Addison Wesley). As well, Peter gained much insight from a conversation with
Linden's
"Open Source Busybody,"
Rob Lanphier.
- In the
developerWorks Linux zone,
find more resources for Linux developers.
- Stay current with
developerWorks technical events and Webcasts.
Get products and technologies
-
Order the SEK for Linux,
a two-DVD set containing the latest IBM trial software for Linux from DB2®,
Lotus®, Rational®, Tivoli®, and WebSphere®.
- With
IBM trial software,
available for download directly from developerWorks, build your next development
project on Linux.
Discuss
About the author | 
| | Peter Seebach has been using computers for years and is gradually becoming acclimated. He still doesn't know why mice need to be cleaned so often, though. |
Rate this page
| |
