On Cell/B.E. machines, there are two different types of processor: The PowerPC Processing Element (PPE), and the Synergistic Processing Elements (SPEs). The PPE can run standard PowerPC applications, but the SPEs require a special compiler that supports their instruction set. This article describes how to build a toolchain that will enable you to compile programs for the SPEs.
Update: you probably don't need to build your own toolchain any more - most distributions provide pre-built packages. However, I've left this page here for reference.
If you want a system that's ready-to-go, try out the Cell SDK on the Barcelona Supercomputing Center's website.
In a nutshell, this article describes how to build a cross-toolchain for the
spu-elf architecture.
Firstly, create a directory to do your builds:
[jk@pokey ~]$ mkdir build
install paths and program prefixes
In this series of instructions, you'll be building an entire toolchain to compile applications to run on the Synergistic Processing Elements (SPEs) of the Cell Broadband Engine Processor. However, we don't want the gcc that we build to overwrite your currently-installed gcc.
There are two approaches to providing separate toolchains on one machine: by using a different install path, or giving the programs a prefix, to distinguish them from other tools on your system.
If you've built other open source applications in the past, you're probably
already familiar with the --prefix= argument to a package's
configure script, used to set the base install path of the
package. By specifying something like --prefix=/usr/local/spu, we
can install all of the toolchain's files within the
/usr/local/spu/ path, so your new gcc will be installed at
/usr/local/spu/bin/gcc, and won't conflict with your existing gcc,
which is (most likely) at /usr/bin/gcc.
However, in this example we won't be setting the prefix, so we end up with
the default of /usr/local. To prevent overwriting any existing
toolchain already installed in this path, we use the
--program-prefix=spu- setting, which adds a fixed prefix (spu-) to
all of the programs installed. For example, the compiler we build will end up
in /usr/local/bin/spu-gcc, and the linker will end up in
/usr/local/bin/spu-ld. This prefix is used for all of the
utilities installed.
If you'd like to install your toolchain to somewhere other than
/usr/local, just add a suitable --prefix= argument to
the invocation of configure in the code listings below.
binutils
Download binutils-2.18.tar.gz from any GNU mirror (eg. ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.18.tar.gz)
[jk@pokey ~]$ wget ftp://ftp.gnu.org/pub/gnu/binutils/binutils-2.18.tar.gz [jk@pokey ~]$ tar zxvf binutils-2.18.tar.gz [jk@pokey ~]$ mkdir build/binutils [jk@pokey ~]$ cd build/binutils [jk@pokey binutils]$ ../../binutils-2.18/configure --disable-werror \ --program-prefix=spu- --target=spu-elf [jk@pokey binutils]$ make [jk@pokey binutils]$ sudo make install [jk@pokey binutils]$ cd ../..
Now you should have a working bintuils installed. To do a quick test, we can try assembling a SPU object:
[jk@pokey ~]$ echo 'frest 0,1' | spu-as -
- this should complete with no error messages.
gcc
Building gcc is a little trickier - the SPU target isn't upstream yet, so we need to grab the sources from the SDK3.0 release. To generate a gcc source tree from the spu-gcc RPM:
[jk@pokey ~]$ wget http://www.bsc.es/projects/deepcomputing/linuxoncell/cellsimulator/sdk3.0/SRPMS/spu-gcc-4.1.1-107.src.rpm [jk@pokey ~]$ mkdir tmp [jk@pokey ~]$ cd tmp [jk@pokey tmp]$ rpm2cpio ../spu-gcc-4.1.1-107.src.rpm | cpio -i -v [jk@pokey tmp]$ tar jxvf gcc-r886.tar.bz2 [jk@pokey tmp]$ cat common-sysroot.diff no-timode.diff | patch -p0 [jk@pokey tmp]$ mv toolchain/gcc ../gcc-4.1.1 [jk@pokey ~]$ cd .. [jk@pokey ~]$ rm -rf tmp
Then we can do the usual configure/make/make install:
[jk@pokey ~]$ mkdir build/gcc [jk@pokey ~]$ cd build/gcc [jk@pokey gcc]$ ../../gcc-4.1.1/configure --disable-threads --disable-checking \ --with-system-zlib --enable-languages=c --disable-libssp \ --program-prefix=spu- --target=spu \ --with-as=/usr/local/bin/spu-as \ --with-ld=/usr/local/bin/spu-ld [jk@pokey gcc]$ make [jk@pokey gcc]$ sudo make install [jk@pokey gcc]$ cd ../..
If you use a different --prefix= option, remember to change the
--with-as and --with-ld options to the full paths
of the spu-as and spu-ld programs installed in the
binutils step.
Once gcc has installed, you should have a working compiler, but you won't yet be able to create spu executables, as we still need a C library. To test the installed gcc, we can try compiling a trivial C file to a SPU object:
[jk@pokey ~]$ echo 'int f(int i){return i+2;}' | spu-gcc -Wall -o/dev/null -c -xc -
- this should also complete with no error messages.
newlib
Now that you have a spu-compiler, we can compile newlib. The upstream version 1.15.0 version should work nicely.
[jk@pokey ~]$ wget ftp://sources.redhat.com/pub/newlib/newlib-1.15.0.tar.gz [jk@pokey ~]$ tar zxvf newlib-1.15.0.tar.gz [jk@pokey ~]$ mkdir build/newlib [jk@pokey ~]$ cd build/newlib [jk@pokey newlib]$ ../../newlib-1.15.0/configure --target=spu [jk@pokey newlib]$ make [jk@pokey newlib]$ sudo make install [jk@pokey newlib]$ cd ../..
To test, we can try a complete 'hello world' program:
[jk@pokey ~]$ echo -e '#include <stdio.h>\nint main(void)' \
'{printf("hello world!\\n");return 0;}' | \
spu-gcc -Wall -o/dev/null -xc -
If this works, you have a shiny new compiler for spu binaries. If not, mail me!
Happy SPU-ing.
Thanks for this post. It's been very helpful as I bring up my new PS3 cluster (well, OK, used since it's mostly a decomissioned one from CMU).
Perhaps if you're in the Research Triangle area or at a supercomputing conferences I should buy you a beer. It's much easier to build this toolchain with modern GCC/GNU + newlib than try to get the IBM Fedora/RHEL-only toolkit working on Gentoo.
Of course, I've not yet really gotten into the guts of writing the code yet, so perhaps the toolchain will still fall down cause I messed up somewhere. Next up: a basic integration on all 9 Cell procs with their SPUs.
Cheers and thanks again! -Joseph