Skip to main content

skip to main content

developerWorks  >  Linux  >

Yellow Dog Linux on Power Mac G5: A Linux on Power developer's workstation

Take an early look at 64-bit Linux on a PowerPC 970 platform

developerWorks
Document options

Document options requiring JavaScript are not displayed

Discuss


Learn and share!

Exchange know-how with your peers -- try our new Pass It Along beta app


Rate this page

Help us improve this content


Level: Introductory

David Mertz, Ph.D. (mertz@gnosis.cx), Developer, Gnosis Software, Inc.

03 Aug 2004

The IBM PowerPC® 970 CPUs are well-designed, high-performance chips that ship in millions of end-user systems under Apple Computers' Power Macintosh G5 moniker. These CPUs greatly lower the bar for 64-bit computing on the desktop and on small servers. Currently, Terra Soft's beta Y-HPC is one of only two 64-bit Linuxes that run on G5s. As their names imply, the G5-enabled betas -- both 32- and 64-bit versions -- are for evaluation only. This article is an early look at the promise of Linux™ on a G5 and is intended for developers interested in trying out this combination in anticipation of production-ready releases to come.

For many Linux users, the best reason to buy an Apple Power Macintosh G5 machine will be, quite simply, the well engineered, high-performance, and reasonably priced machines available from Apple. Many enterprises, hosting companies, schools, and research facilities have a mixture of x86 and PowerPC systems that they provide to users. When you want to assure a uniform system/user interface across these machines, Linux is the best choice for an operating system. PowerPC 970-based machines are comparable in performance to AMD's Athlon64 and Opteron and to the Intel™ Pentium™ 4EE and Xeon.

Additionally, cross-compiling within the family of Power Architecture™ CPUs will provide relatively consistent performance characteristics across portions of an application. So, for example, profiling an application on a G5 gives you a better guide to the bottlenecks you might see on a POWER5™ machine than would profiling on a Pentium 4.

My previous developerWorks article, Linux on Mac: a POWER programmer's primer, explored some of the general motivations for running Linux on Apple hardware, especially given that every Apple already ships with a generally excellent UNIX®-family operating system called OSX. I don't want to repeat the whole thing here, but basically the arguments in favor of Linux on a G5 amount to "same as for the G4, but a way zippier chip." In addition to the installation tips mentioned in that article, I'll go into several more here, some general, others specific to Terra Soft Solutions' G5-enabled versions of Linux or to the G5.

Drop by the Developer's corner

If you're an individual developer, corporate programmer, or ISV looking for a thorough resource for getting started with Linux on Power Architecture, the best place to start is the Linux on Power Architecture Developer's corner. There you'll find technical articles, downloads, links to testing and porting facilities, an online discussion forum, and more.

This article explores how to set up a dual-boot environment with Yellow Dog Linux/Y-HPC and OSX on G5 systems, including issues to watch for during installation and configuration. Note that the name of the higher performance distribution we are using is Y-HPC; this refers currently to the beta 64-bit product and will later be used for Terra Soft's production-ready 64-bit product. The company's 32-bit product is known as Yellow Dog Linux (YDL) and, for use on G5 machines, is also considered to be in beta. It is currently at version 3.0.1 but will emerge from beta as version 4.0. We first install the 32-bit YDL, then upgrade to a 64-bit Y-HPC kernel. Note that, being beta, neither distribution should be considered ready to run in production development environments. Please also note that, once both versions emerge from beta in several weeks' time, the install procedures should be much simpler than what is outlined here.

I'll address some general features of the Yellow Dog distribution, such as configuration and tool details. To give a sense of the kind of performance we might expect from 64-bit Linux kernels and from the G5 chipset, I've also performed benchmarks using LMBench under several scenarios. I'll touch briefly on processor-specialized compilation options using gcc, and also on cross-compilation to other POWER-family chips.

Installing Yellow Dog Linux

The first thing you need to know is that you cannot boot a G5 from the publicly available YDL 3.0.1 ISOs. This is because the "hybrid" versions of 3.0.1 for the G5 are in beta; once they are production-ready, they will be posted to the public site. Meanwhile, to access the 32-bit YDL ISOs with G5 compatibility, you need to pay for a membership subscription to "YDL.net Enhanced" (see Resources).

Once you have subscribed, take care to download the correct ISOs. The general ISOs are numbered 3.0.1 and live in the /enhanced/iso/ FTP directory; these are not the ones you want. The experimental G5 ISOs are also numbered 3.0.1, but they live further down, in the /enhanced/iso/PowerMacG5/ directory. Given that they are still in beta, the exact file dates are likely to change; I used those named *-20040204-*.iso.

Once it is out of beta, the G5-compatible 32-bit YDL will be promoted to version 4.0 and will be available without a membership. The 64-bit Y-HPC, however, will only be available to "YDL.net Pro" (not Enhanced) members. The only other way to access the 64-bit kernels from Terra Soft is to wait until they become available in the TSS online store, or to buy them, pre-configured, on Terra Soft pre-built G5 systems. This is done at the buyer's request and at extra cost, so if it's what you want, you need to specify it.

In any case, other PPC Linux distributors -- such as Mandrake, Debian, Gentoo, and SUSE -- are also readying their own G5-ready Linux systems. Of those, Gentoo is the only one I know of that is also working actively on 64-bit G5 kernels; the others are sticking to 32-bit versions for now. However, much of what this article covers, from performance characteristics to Macintosh-related HowTos, should apply equally to all of these anticipated future distributions.

Another thing worth mentioning, particularly if you are used to Linux on x86, is that there is not yet any good way to non-destructively repartition existing HFS+ partitions. Of course, this is true for any Linux-on-Mac distribution that you would want to install, since Power Mac G5s come partitioned as one large partition. You will need to wipe out and reinstall OSX to get your dual-boot system configured. For dedicated server use, you probably only need one OS anyway, but for developers and most other users, multi-boot is a useful configuration.

The easiest way to set up a dual- or multi-boot Power Mac G5 system is to reinstall OSX from the installation media that came with the machine. At the start of the OSX installation, select "Disk Utility" from the menu, create a smaller partition as HFS+, and leave the remainder as free space. On my 160 GB test system HDD, I allocated 30 GB for OSX (I can always create more HFS+ partitions later if needed).

Once you have reinstalled and left some free disk space, simply put in the G5-compatible YDL CD that you obtained according to the above instructions. As with all Macintosh system, hold down the "c" key during reboot to boot from CD rather than HDD. Yellow Dog's installer is very friendly and is based on Red Hat's Anaconda installer. Everything except the sound card was automatically recognized correctly by the installer -- even my exact monitor model and capabilities, which OSX failed to detect. My DHCP router and Ethernet network connected seamlessly. A minor annoyance was that the installer ran with a 60 Hz monitor refresh, which produces a distracting flicker under florescent lights. But once installed, Yellow Dog uses a healthy (and configurable) 70+ Hz. Note that you want to reinstall OSX first, and then install Linux; doing it the other way around leads to more work.

During installation, YDL presents you with partitioning options, performed with the user-friendly tool "Disk Druid." Journalling ext3 is its preferred filesystem (and what I used). If you select "Automatic," Disk Druid will claim all the free space for Linux; I preferred to manually configure another 30 GB ext3 partition (and leave the default swap partition at the end of the disk). This left about 100 GB free for me to install additional OS or data partitions, using whichever filesystems I needed. A little bit later in the installation process, Yellow Dog installs the boot manager Yaboot, which works mostly the same as lilo does on x86 systems (including an /etc/yaboot.conf/ configuration file).

Once you select which packages you wish to install -- perhaps by choosing a general option such as "Desktop," "Server," or "Developer Workstation" -- the installation finishes and reboots into the Yaboot boot manager. Assuming you choose "Linux" rather than "OSX" during boot, Yellow Dog scrolls through all the textual messages about drivers and daemons loading that Linux users are accustomed to. After about 40 seconds (on a dual-1.8GHz G5 machine), you get to the Yellow Dog welcome screen where you can log in as a user, reboot, choose the desktop environment, and the like. For what it's worth, OSX boots in a very impressive 15 seconds on the same machine. KDE is YDL's default environment, but Gnome is also included on the installation CDs (and you can build whatever window manager you like from source).



Back to top


Working with your G5 Linux system

YDL is a lot like other modern Linux distributions. It comes with GCC (for the current G5 betas, it's v3.3.3) as well as recent versions of Python, Perl, Ruby, and other programming languages. The KDE launch menu contains office software such as OpenOffice, GIMP, Mozilla, and several development environments such as KDevelop, arranged in logical hierarchies. You can use the default scheme and interface configuration as they are, or you can use KDE Control Center or miscellaneous right-clicks to change behavior as you like.

Let me mention a few issues specific to Macintosh and Power Mac G5 configurations that x86 Linux users might not be familiar with. Early on, you should consider making your OSX HFS+ partition available to read data from. My understanding is that the driver to handle journalling in HFS+ is still experimental, so you are better off treating HFS+ as read-only. To access HFS+, you'll do something like the following (you'll need to log in as root or su):


Listing 1. Mounting an HFS+ partition under YDL
% mkdir /mnt/osx
% modprobe hfsplus
% parted /dev/sda  # press "p" for partition list
% mount /dev/sda3 /mnt/osx -thfsplus
% ls /mnt/osx

The parted command lets you check partition numbers and filesystems. Take a look within the tool to see which HFS+ partitions you have. If you like using Konqueror and the KDE desktop to navigate files, you may want to drag /mnt/osx from a Konqueror window to the desktop.

Unlike on x86 systems, Macintoshes do not come with an "eject" button on their CD drives. Under OSX, you open the drive with a special button on the keyboard. Under YDL, you need to issue the command eject /dev/cdrom from a shell prompt (or attach the action to an icon). Once you insert a new CD in the drive, run mount /dev/cdrom or configure automount appropriately.

Unfortunately, Mac-on-Linux (MoL) does not yet work with the hybrid 2.6.4 kernel that Terra Soft put together in order to get the beta to work on the G5. For those of you who are new to Linux, you should know that at times like this, downloading and building the latest source code of a dysfunctional application can often get it to work. This time, however, it did not, so we will have to wait for the Mac-on-Linux project to put out a new module before we can expect MoL to work on the G5.



Back to top


"Upgrading" 32-bit YDL to 64-bit Y-HPC

According to Terra Soft, 64-bit installation will become much simpler once it is production-ready. In the meantime, if you want to try the 64-bit kernel before Y-HPC officially ships, first install the 32-bit YDL as I did. Then, go to the Yellow Dog Linux Y-HPC page (see Resources) and download the vmlinux* and System* files to /boot/. From there, I ran:


Listing 2. Unpacking the 64-bit Linux kernel
% gunzip vmlinux-2.6.1-1.64.ydl.1.1280.gz
% gunzip System.map-2.6.1-1.64.ydl.1.1280.gz
% chmod u+x vmlinux-2.6.1-1.64.ydl.1.1280

I grabbed the kernel modules too, downloaded to /root/, and ran:


Listing 3. Unpacking 64-bit kernel modules
% bzip2 -d /root/modules-2.6.1-1.64.ydl.1.1280.tar.bz2
% cd /
% tar xvf /root/modules-2.6.1-1.64.ydl.1.1280.tar

This last step creates a large number of files underneath /lib/modules/2.6.1-1.64.ydl.1.1280/. The next step is to add the 64-bit kernel to the boot manager. First edit /etc/yaboot.conf to include:


Listing 4. Adding 64-bit Y-HPC kernel to Yaboot
image=/boot/vmlinux-2.6.1-1.64.ydl.1.1280
      label=linux-64bit
      root=/dev/sda4
      read-only
      append="hda=ide-cd"

Then run ybin (as root), and you are ready to choose kernels at the next restart. As you will see, performance is minimally affected by using the 64-bit kernel; but choosing 64-bit lets you compile 64-bit applications that might benefit from 64-bit-ness.



Back to top


GCC options and cross-compilation

I ran some benchmarks for LMBench (which I will summarize at the end of this article) using variations on kernel versions and compilation options. Although this benchmark was not dramatically affected by the options I tried, the source for LMBench still provided a helpful project to test compiler options on. For example, in one configuration under the 64-bit kernel, I tried configuring the compiler by entering:

% export CPPFLAGS='-mcpu=970 -mtune=970'

prior to running the usual make step. A useful summary of key PowerPC 970 compiler options can be found in the document "About Compilers with VMX Support" (see Resources). The referenced document covers GCC 3.3.3, the version shipped with the hybrid YDL, and with the IBM eServer™ JS20 BladeCenter™ PowerPC 970-based machine. You can also compare compiler options available to Linux and Darwin (Mac OSX) -- largely similar, but with a few differences.

Moreover, the design of the whole POWER family architecture allows both cross-compilation to various specific targets as well as compilation to a common instruction base. The possibility of developing an application on an Apple Power Mac G5 but running it on an IBM eServer p5 is quite intriguing.

The GCC documentation, section 3.17.23, "IBM RS/6000 and PowerPC Options" (see Resources) provides some helpful information on cross-compilation options. The -mcpu flag is the main one to look at. Quoting from the reference:

-mcpu=cpu_type
Set architecture type, register usage, choice of mnemonics, and instruction scheduling parameters for machine type cpu_type. Supported values for cpu_type are 401, 403, 405, 405fp, 440, 440fp, 505, 601, 602, 603, 603e, 604, 604e, 620, 630, 740, 7400, 7450, 750, 801, 821, 823, 860, 970, common, ec603e, G3, G4, G5, power, power2, power3, power4, power5, powerpc, powerpc64, rios, rios1, rios2, rsc, and rs64a.
-mcpu=common selects a completely generic processor. Code generated under this option will run on any POWER or PowerPC processor. GCC will use only the instructions in the common subset of both architectures, and will not use the MQ register. GCC assumes a generic processor model for scheduling purposes.

Experimenting, I found that on my YDL system, I was able to compile (and run) LMBench to several different CPU targets, such as G5, G4, or powerpc. I was particularly interested in trying compilation to the common target, but my system lacked some necessary headers. I could perhaps work this out by downloading the GCC sources and other associated files. With the YDL stock GCC, I could still compile many -- just not all -- of the LMBench source files to the common target.



Back to top


Benchmarking

First, I should get a few disclaimers out of the way. The first is that Terra Soft's 32- and 64-bit systems, which are in beta, are being compared to OSX, which is shipping. So all of these numbers might change in coming months when Terra Soft releases production 32- and 64-bit distributions. However, the fact that the benchmarks focus on low-level performance does make it likely that the numbers will be at least somewhat relevant to the shipping versions. Nevertheless, these numbers do not (and are not meant to) reflect on OSX vs. YDL/Y-HPC so much as on OSX vs. Linux, 32-bit vs. 64-bit, and, to a much lesser extent, on how GCC compiler switches affect LMBench.

The second disclaimer is that I do not have sufficient benchmarking expertise to perform meaningful comparisons of a G5 system with competing processors in a similar class, such as Intel's Xeon or AMD's Athlon64.

Given that few published benchmarks are completely unbiased, I believe it is still useful (and fair) to take a look at how differences in operating system, kernel-bit-ness, and compilation options affect benchmark results on the self-same machine. One of the highly touted features of Apple's Power Mac G5 systems is their fast memory subsystem -- DDR SDRAM on a 400 MHz bus, with an advertised bandwidth of up to 6.4 GB/sec (my test system is not the very top-of-the-line, though). To look at this, I felt the LMBench 2.0.4 (the latest stable release) was a good tool: it focuses on low-level performance, especially of memory subsystems.

I ran LMBench in four different configurations. In some cases, I performed multiple runs, but the variations were small enough that I present only one example from each configuration. As a base case, I compiled and ran LMBench on Mac OSX (1.3.4). In general, it looks like Linux benches slightly better than OSX on most tests. However, a few stunningly bad results on OSX make me wonder if all parts of the suite were running accurately. For example, it is impressive and believable that 64-bit Y-HPC gets 50 percent better local TCP bandwidth than does OSX -- Linux is quite well tuned for TCP. But for OSX to experience 500 times the latency of Linux on disk page faults is rather shocking. Of course, that latter issue, like the other poor OSX results on "File & VM system latencies," has more to do with the differences between HFS+ and ext3 than with OS kernels as such.

It's always hard to extrapolate from analytic benchmarks to synthetic benchmarks, but LMBench is most likely to be a proxy for applications that use lots of IPC (like scientific applications and other number crunchers) -- high-load Web serving could be affected too.

On the Linux side, I ran one configuration with the default 32-bit kernel and no compiler options. Under the 64-bit kernel, I compiled LMBench both with no GCC options, and then with 970-specific tuning. The Linux results do not cover a huge range, but there generally appears to be a slight gain with the 64-bit kernel, and another slight gain with the application compiled using the 970 flags.

Here, then, are my LMBench results (edited slightly for layout):


Table 1. LMBench 2.0.4 summary on Power Mac G5
Basic system parameters
Host                 OS Description              MHz
--------- ------------- ----------------------- ----
Darwin    Darwin 7.2.0  powerpc-apple-darwin7.2 1800
32bit     Linux 2.6.4-1 powerpc-linux-gnu 32bit 1800
64bit     Linux 2.6.1-1 powerpc-linux-gnu 64bit 1800
64bit-970 Linux 2.6.1-1 PPC-linux -mtype=970    1800


Processor, Processes - times in microseconds - smaller is better
----------------------------------------------------------------
Host      null null      open selct sig  sig  fork exec sh
          call  I/O stat clos TCP   inst hndl proc proc proc
--------- ---- ---- ---- ---- ----- ---- ---- ---- ---- ----
Darwin    2.23 2.78 6.27 9.17  15.6 3.09 8.97 1468 3146 6796
32bit     0.30 0.42 5.13 6.08  25.5 0.72 2.67  295  990 4093
64bit     0.23 0.43 5.04 6.47  33.3 0.70 2.19  257  914 3778
64bit-970 0.24 0.44 4.98 6.49  33.2 0.70 2.25  262  955 3856


Context switching - times in microseconds - smaller is better
-------------------------------------------------------------
Host      2p/0K 2p/16K 2p/64K 8p/16K 8p/64K 16p/16K 16p/64K
          ctxsw  ctxsw  ctxsw ctxsw  ctxsw   ctxsw   ctxsw
--------- ----- ------ ------ ------ ------ ------- -------
Darwin     12.4   12.3   12.3  12.9   16.4   14.2     34.2
32bit      10.4   10.5   10.6   8.68   9.71   7.57    26.2
64bit      11.1   11.3   11.4   9.35   12.7  10.1     29.2
64bit-970  11.2   11.3    7.97  7.11   16.4   7.57    27.9


*Local* Communication latencies in microseconds - smaller is better
-------------------------------------------------------------------
Host      2p/0K  Pipe AF     UDP  RPC/   TCP  RPC/ TCP
          ctxsw       UNIX         UDP         TCP conn
--------- ----- ----- ---- ----- ----- ----- ----- ----
Darwin     12.4  40.6 40.3  58.4        74.3       153.
32bit      10.4  20.5 24.5  31.5  40.6  57.5  41.4 53.3
64bit      11.1  47.8 42.4  58.5  45.9  67.7  68.9 47.9
64bit-970  11.2  40.2 24.6  34.0  40.5  35.0  48.5 47.7


File & VM system latencies in microseconds - smaller is better
--------------------------------------------------------------
Host        0K File      10K File      Mmap    Prot    Page
          Create Delete Create Delete  Latency Fault   Fault
--------- ------ ------ ------ ------  ------- -----   -----
Darwin     108.2  136.3 1440.9  228.2    11.0K 47.4   3740.0
32bit       44.1   34.3  121.8   64.3   3894.0  0.595    7.0
64bit       46.5   30.8  118.2   59.2   3314.0  0.514    4.0
64bit-970   46.5   30.8  118.3   59.2   3256.0  0.550    4.0


*Local* Communication bandwidths in MB/s - bigger is better
-----------------------------------------------------------
Host      Pipe AF    TCP  File   Mmap  Bcopy  Bcopy  Mem   Mem
               UNIX      reread reread (libc) (hand) read write
--------- ---- ---- ---- ------ ------ ------ ------ ---- -----
Darwin     52.  346 351.  364.0 1837.4 1431.0  897.8 1840 1207.
32bit     715. 1137 428. 1037.6 1799.4  899.9  888.5 1805 1203.
64bit     634. 1189 609.  922.3 1789.5  888.5  888.5 1792 1200.
64bit-970 620. 1046 629.  915.4 1792.0  888.4  889.6 1797 1209.


Memory latencies in nanoseconds - smaller is better
---------------------------------------------------
Host        L1 $   L2 $ Main mem
---------  ----- ------ --------
Darwin     1.664 6.1040  152.3
32bit      1.666 6.1110  153.1
64bit      1.667 6.1150  153.9
64bit-970  1.666 6.1140  153.3



Back to top


Conclusion

Overall, a G5 is quite a slick machine to run Linux on -- and is now available at reasonable end-user desktop prices. Even as a beta, Terra Soft's Yellow Dog Linux and Y-HPC are mostly ready to run on G5 systems, with only relatively few, relatively minor glitches still remaining (not all discussed specifically here). I'd say this bodes well for the production releases.

I was not really sure what to expect in advance about the relative performance of OSX and Linux on a 970 machine. Both operating systems are much faster than "fast enough" as end-user desktops or development platforms, and any lingering differences tends to conflate the differences between KDE/X11 and Aqua/Quartz, rather than reflect underlying OS characteristics. However, my attempts at memory benchmarking suggest that Linux is likely to provide higher-end performance for high-traffic servers, complex compilations, and complex scientific or graphical applications than OSX does. Obviously, before reaching any specific conclusion relative to your own needs, high-level benchmarking of specific applications is a good idea, if you have a choice between underlying operating systems (which you usually do for Free Software).



Resources



About the author

David Mertz

David Mertz is owner and chief consultant for Gnosis Software, Inc. whose corporate slogan is "We Know Stuff!" (and we do). You can reach David at mertz@gnosis.cx; you can investigate all aspects of his life at his personal Web page. Check out his book, Text Processing in Python.




Rate this page


Please take a moment to complete this form to help us better serve you.



 


 


Not
useful
Extremely
useful
 


Share this....

digg Digg this story del.icio.us del.icio.us Slashdot Slashdot it!



Back to top