Project

General

Profile

History

Fermi Buildroot Linux

About

Buildroot is a set of Makefiles and patches for building a Linux kernel and a custom root file system
based on Busybox and ucLibc. This project customizes Buildroot by adding packages for Fermi specific
products. The included configuration produces a Linux 2.6 kernel with RTAI patches and a minimal file
system including openssh, Kerberos 5, Acsys/FE and EE Support software installed.

Dependencies

The following programs/packages need to be installed on your development system:
  • openssl-devel
  • glibc-static
  • glibc-devel.i686 (for x86-64 systems)
  • pc (Rich Neswold's Protocol Compiler)

If you need to create a new bootable flash drive you will need to have grub on your development system.
SLF ships with a grub suitable for booting x86 boards, but you will need to do a custom build to include
your board's ethernet driver if you want to do network booting.

Git

The ees-buildroot project contains the full Buildroot source plus our customizations. There are three
main branches:

buildroot-master Tracks the master branch on the official Buildroot repository. Updates to this branch are pulled from buildroot/master every so often and then merged into ees-custom.
ees-custom Our main development branch - includes all Fermi customizations. Development of Fermi customizations should proceed on topic branches and then merged into this branch when the developer feels that they are ready. When updates to Buildroot are pulled they should be merged from buildroot-master into this branch for integration.
master For stable releases only.

Building

  1. Configure buildroot with make menuconfig. Fermi packages are under Package Selection / Fermi. Once you have configured your image and tested it, copy the .config file to a 'project-config' file and add it to git (Do not check in the .config file, add it to .gitignore). When you want to rebuild that project's image, copy the project-image file to .config.
  2. If you need to configure the kernel run make linux-menuconfig. When you're done, copy the kernel config file from output/build/linux-custom/.config into ees-buildroot/.linux-config.
  3. Run make clean if you want your build to start from scratch, but this takes a long time. If you want to rebuild just a specific package then delete it's directory from ees-buildroot/output/build and if necessary, it's source package from ees-buildroot/dl.
  4. Run make
  5. From ees-buildroot/output/images, copy bzImage to a bootable flash drive with grub installed. A sample menu.lst file can be found in ees-buildroot/fs/skeleton/boot/grub. If booting with TFTP, copy bzImage over to /fecode-bd/linux_boot/ees/project on nova.fnal.gov.

Node Configuration

The root filesystem image generated by ees-buildroot is intended to be generic and loaded into a ramfs at
bootup, but there are certain files that need to be customized for each node, e.g. the network configuration.
The solution here is to mount a node-specific compact flash or NFS file system that contains these files
and use symlinks to map them to the image that is loaded into the ramdisk. The mount point for this filesystem
is /rfs. You can specify if this is an NFS mount or a local drive in /etc/fstab.
The files replaced with symlinks are:
  • /etc/hostname -> /rfs/etc/hostname
  • /etc/hosts -> /rfs/etc/hosts
  • /etc/krb5.keytab -> /rfs/etc/krb5.keytab
  • /etc/network/interfaces -> /rfs/etc/network/interfaces
  • /etc/ssh_host_key -> /rfs/etc/ssh_host_key
  • /etc/ssh_host_key.pub -> /rfs/etc/ssh_host_key.pub
  • /etc/ssh_host_dsa_key -> /rfs/etc/ssh_host_dsa_key
  • /etc/ssh_host_dsa_key.pub -> /rfs/etc/ssh_host_dsa_key.pub
  • /etc/ssh_host_ecdsa_key -> /rfs/etc/ssh_host_ecdsa_key
  • /etc/ssh_host_ecdsa_key.pub -> /rfs/etc/ssh_host_ecdsa_key.pub
  • /etc/ssh_host_rsa_key -> /rfs/etc/ssh_host_rsa_key
  • /etc/ssh_host_rsa_key.pub -> /rfs/etc/ssh_host_rsa_key.pub
  • /usr/local/products/data -> /rfs/usr/local/products/data

Booting a New Node

This section will guide you through step-by-step how to take the image that buildroot makes for you and boot-up a new node with it.

First, create a linux partition on your flash disk and make it bootable with cfdisk (as root):

# cfdisk /dev/sdc

Format the new partition:

# mkfs.ext3 /dev/sdc1

Configure grub. If you are using TFTP to boot you will need to configure your node's IP address and gateway.

# mount /dev/sdc1 /mnt
# mkdir -p /mnt/boot/grub
# cp fs/skeleton/boot/grub/menu.lst /mnt/boot/grub
# nano /mnt/boot/grub/menu.lst

Install GRUB with grub-install (--recheck instructs grub-install to rebuild it's device map before installing):

# grub-install /dev/sdc --recheck --root-directory=/mnt

Copy your kernel image:

# cp output/images/bzImage /mnt/boot

Configure your node-specific files:

# mkdir -p /mnt/etc
# mkdir -p /mnt/etc/network
# mkdir -p /mnt/usr/local/products/data
# nano /mnt/etc/hostname
# nano /mnt/etc/hosts
# nano /mnt/etc/network/interfaces

Un-mount and boot up:

# umount /dev/sdc1

The first time you bootup, sshd will generate a set of host keys for itself. This should only happen once. Request host kerberos principals from the helpdesk for your new node. The request is usually processed the same day. One you have your key (password) from the helpdesk you can create a host principal:

# kadmin -p host/nodename.fnal.gov -q "ktadd host/nodename.fnal.gov" 
(enter password from helpdesk e-mail)
# kadmin -p ftp/nodename.fnal.gov -q "ktadd ftp/nodename.fnal.gov" 
(enter password from helpdesk e-mail)

Remote Deployment

You can transfer new builds to your target using scp. Most of our projects have a reminstall target that will take care of scp'ing an application to your target (specified with remsys) for you. However, most targets live inside the controls firewall so in order to scp you must first configure ssh to proxy through outland.fnal.gov. Add the following to your ~/.ssh/config file:

Host target.fnal.gov
      ProxyCommand ssh user@outland.fnal.gov nc %h %p

Then to copy your_program to the target:

# scp -q your_program root@target.fnal.gov:/usr/local/products/bin/your_program

Keep in mind that the root filesystem is in a ramdisk, so the new file that you copy over will disappear after you reboot unless you rebuild and deploy your kernel+ramfs with the new changes. To deploy a new kernel, you can add an entry for nova.fnal.gov to your ~/.ssh/config file and copy the kernel to the linux_boot area:

# scp -q output/images/bzImage nova.fnal.gov:/fecode-bd/linux_boot/ees/project/bzImage.dev

Remote Debugging

You can debug executables on your target using GDB's remote features. Start the application with gdbserver:

$ gdbserver :2345 /usr/local/products/bin/your_program

Then, from the host that you built ees-buildroot on, establish a secure tunnel for GDB into the controls vlan:

$ ssh -f -N -L 2345:target.fnal.gov:2345 outland

Alternatively, you could log into the controls VPN and address your target directly.
Move into the build directory and connect to the target via the tunnel:

$ cd output/build/your_program
$ gdb your_program
(gdb) target remote localhost:2345