History

Embedded Linux Software Development¶

About¶

Buildroot is a set of Makefiles and patches for building embedded Linux kernels and custom root file systems based on Busybox and ucLibc. This project customizes Buildroot by adding packages for Fermi specific products, board support for single-board computers (SBCs) and System-on-Module chips (SoM) used in Accelerator Division and tools to assist in developing embedded Linux applications on a shared development system (adlinux.fnal.gov).

Building¶

Quick Start¶

Configure buildroot with make menuconfig. Fermi packages are under Package Selection / Fermi.
If you need to configure the kernel run make linux-menuconfig. When you're done, copy the kernel config file from output/build/linux-custom/.config into ees-buildroot/.linux-config.
Run make clean if you want your build to start from scratch, but this takes a long time. If you want to rebuild just a specific package then delete it's directory from ees-buildroot/output/build and if necessary, it's source package from ees-buildroot/dl.
Run make
From ees-buildroot/output/images, copy bzImage to a bootable flash drive with grub installed. A sample menu.lst file can be found in ees-buildroot/fs/skeleton/boot/grub. If booting with TFTP, copy bzImage over to /fecode-bd/linux_boot/ees/project on nova.fnal.gov. Alternatively, a target is provided in the buildroot Makefile for installing the boot images to /fecode-bd/linux_boot/ees for you:
```
make install_target_images BOARD=<board> [VARIANT=<variant>]
```

Build Using an Extracted Tool-chain¶

Buildroot allows you the option of using an external tool-chain, which can speed up the build process considerably. The tool-chain can be vendor supplied or extracted from another Buildroot build (extracting a tool chain is discussed below) . Using an external tool-chain will also allow you to share a tool-chain between multiple developers.

To configure Buildroot to use an external tool-chain installed on adlinux:

make menuconfig
From toolchain menu:
1. Select external toolchain / pre-installed
2. Toolchain directory: /usr/local/products/elsd/<board>/<linux-X.Y_variant>/output/host/usr
3. Select Kernel header version
4. Select glibc for toolchain library (mvme8100) or uclibc (conc405x)
5. Enable RPC support, C++ support, whatever else buildroot asks for
Save configuration, exit, make clean

Node Configuration¶

The root filesystem image generated by ees-buildroot is intended to be generic and loaded into a ramfs at bootup, but there are certain files that need to be customized for each node, e.g. the network configuration. The solution here is to mount a node-specific flash storage or NFS file system that contains these files and use symlinks to map them to the image that is loaded into the ramdisk. The mount point for this filesystem is /rfs. You can specify if this is an NFS mount or a local drive in board/fermi/<board_variant>/rootsfs_overlay/etc/fstab.
The files replaced with symlinks are:

/etc/hostname -> /rfs/etc/hostname
/etc/hosts -> /rfs/etc/hosts
/etc/krb5.keytab -> /rfs/etc/krb5.keytab
/etc/network/interfaces -> /rfs/etc/network/interfaces
/etc/ssh_host_key -> /rfs/etc/ssh_host_key
/etc/ssh_host_key.pub -> /rfs/etc/ssh_host_key.pub
/etc/ssh_host_dsa_key -> /rfs/etc/ssh_host_dsa_key
/etc/ssh_host_dsa_key.pub -> /rfs/etc/ssh_host_dsa_key.pub
/etc/ssh_host_ecdsa_key -> /rfs/etc/ssh_host_ecdsa_key
/etc/ssh_host_ecdsa_key.pub -> /rfs/etc/ssh_host_ecdsa_key.pub
/etc/ssh_host_rsa_key -> /rfs/etc/ssh_host_rsa_key
/etc/ssh_host_rsa_key.pub -> /rfs/etc/ssh_host_rsa_key.pub
/usr/local/products/data -> /rfs/usr/local/products/data

Booting¶

The boot process is somewhat board-specific and you should refer to your board's documentation for more details. That said, we have developed some documentation that could help you out:

A Makefile target is provided for deploying boot images to the proper location on fecode-bd:

make install_target_images BOARD=<board> VARIANT=<variant>

Board Support¶

Board support has been developed for several popular SBCs and SoMs used in Accelerator Division. If your board is not listed, don't panic! Adding a board support isn't that hard and you can probably get away with copying the board support package for a similar board and tweaking some configuration settings.

Supported SBCs and SoMs¶

Product	defconfigs	Architecture	Kernels Supported	Deployments
MVME 5500	fermi_mvme5500_generic_defconfig	VME / PowerPC	2.6.20
MVME 8100	fermi_mvme8100_generic_defconfig	VME / QorIQ (PowerPC)	3.0.6	Booster BPMs
MVME 8105	fermi_mvme8105_generic_defconfig	VME / QorIQ (PowerPC)	3.0.6
Concurrent 405x	fermi_conc405x_generic_defconfig	VME / Intel Core Duo	3.16.7	IOTA BPMs
VMIC VMIVME7805	fermi_vmivme7805_generic_defconfig	VME / Intel ???	3.16.7
MitySOM 5CSX	fermi_mitysom5csx_lnmxmd_defconfig	SOM / ARM	4.1.33-ltsi-rt38	Linac Marx Modulators

Board Support Explained¶

There are two components to a board support package in ees-buildroot:

A buildroot defconfig file - this file contains the Buildroot configuration values that deviate from their default settings. It mostly consists of high-level configuration settings such as the architecture type, enabled packages, tool-chain location and location of the Linux kernel config, patches and other files. This file is located in the configs/ and has the form fermi_<board>_<variant>_defconfig. To load the configuration into Buildroot run
```
make fermi_<board>_<variant>_defconfig
```

Board support directory - this directory contains all of the ancillary configuration files (Linux kernel config, Busybox config, gclibc/uclibc config), patch files and post-build scripts. Board directories are located in board/fermi/ and have the following structure:

board/                         Main board support directory
  fermi/                       Fermi board support directory
    common/                    Configurations/patches/rootfs files/etc. common to all Fermilab board support packages
      patches/                 Patches that are fermi-specific but not board-specific
        rootfs_overlay/        Fermilab-specific root filesystem overlay
          etc/
            passwd             Users that can login to Fermilab boards
          root/
            .k5login           Users that can ksu to root
    <board_variant>/
      linux.config             Linux kernel configuration for <board_variant>
      busybox.config           Busybox configuration for <board_variant>
      uclibc.config            ucLibc / glibc configuration for <board_variant>
      patches/                 Patch files specific for <board_variant>
      rootfs_overlay/          Root filesystem overlay specific to <board_variant>

Fermilab Common Board Support¶

Notice the common/ directory structure under board/fermi/. This directory contains board support files that are common to all of the Fermilab board-support packages. For example, the /etc/passwd rootfs-overlay file contains the usernames of Fermilab personnel that are allowed to login to all embedded nodes.

Creating a New Board Support Package¶

If you're building Buildroot for a totally new board (not a variant of an existing board) then you probably want to build a new tool-chain as well. If a tool-chain that you wish to use already exists (i.e. if you are creating a variant support package for an existing board), when you go to configure Buildroot make sure that you select the external tool-chain for the board from:

/usr/local/products/elsd/

Otherwise, have buildroot build a toolchain for you and extract it for external use later.

Steps for creating a new board support package:

Grab ees-buildroot from Git
Copy board/fermi/someboard_variant to board/fermi/newboard_variant
Copy configs/fermi_someboard_variant_defconfig to fermi_newboard_variant_defconfig
Open configs/fermi_newboard_variant_defconfig, replace refernces to board/fermi/someboard_variant with board/fermi/newboard_variant
Run make fermi_newboard_variant_defconfig to load new configuration
Run make menuconfig
Run make linux-menuconfig to configure kernel (optional) and make linux-update-defconfig to save
Run make busybox-menuconfig to configure busybox (optional) and make busybox-update-defconfig to save
Run make uclibc-menuconfig to configure uclinc (optional) and make uclibc-update-defconfig to save
Any board_variant-specific root filesystem changes should go in board/fermi/newboard_variant/rootfs_overlay (any fermi-specific / board-agnostic changes should go in board/fermi/common/rootfs_overlay)
Place any board-specific kernel or package patches in board/fermi/newboard_variant/patches (if you have fermi-specific / board-agnostic patches they should go in board/fermi/common/patches)
Run make, test new kernel
(optional) If you built a new toolchain now is a good time to extract the toolchain, reconfigure Buildroot to build with the external toolchain you just extracted, re-build and test again
Run make savedefconfig to save configuration to defconfig.
```
make savedefconfig
```
Copy defconfig to configs/fermi_newboard_variant_defconfig
```
cp defconfig configs/fermi_<board>_<variant>_defconfig
```

Commit new board configuration to Git:

       git add board/fermi/newboard_variant
       git add configs/fermi_newboard_variant

Extracting a Toolchain¶

Once you have created a board support package, you should consider extracting a tool-chain to speed-up re-builds and to allow other developers to build for your target.

make install_toolchain BOARD=<board> VARIANT=<variant>

The BOARD argument is required and should identify your board properly. The VARIANT argument is optional - generic will be used if you omit it.

Using a pre-built External Toolchain¶

Some toolchains can be downloaded as prebuilt, ready to use, toolchains. An example of this is the Linaro ARM toolchain used by the MitySOM 5CSX System On Module (Intel Cyclone V w/ ARM HPS SOM). The Linaro ARM toolchain was downloaded from:
https://releases.linaro.org/archive/14.09/components/toolchain/binaries/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux.tar.xz and installed on adlinux.fnal.gov at /usr/local/products/elsd/mitysom5csx/gcc-linaro-arm-Linux-gnueabihf-4.9.

The Buildroot toolchain configuration is then setup as follows:

Toolchain Type -> External toolchain
Toolchain -> Linaro ARM 2014.09
Toolchain origin -> Pre-installed toolchain
Toolchain path -> /usr/local/products/elsd/mitysom5csx/gcc-linaro-arm-Linux-gnueabihf-4.9

Development Tools¶

Once a tool-chain is extracted you can use it to develop your applications from your sandbox directory. We provide a Makefile include that you can use for building C/C++ applications located at

/usr/local/products/elsd/include/elsd-latest.mk

To use it:

Define global variables for your project in your Makefile -
1. OS (EES_TARGET_OS)
2. OS version (EES_TARGET_OS_VERSION)
3. OS variant, optional - generic assumed (EES_TARGET_OS_VARIANT)
4. Target board name (EES_TARGET_BOARD)
5. Target board architecture (EES_TARGET_ARCH)
6. Project name (EES_PROJECT)
Include /usr/local/products/elsd/include/elsd-latest.mk in your makefile

Example usage from adinstbpm:

EES_TARGET_OS   = linux
EES_PROJECT     = bpmd

ifeq ($(BOARD),MVME_8100)
        EES_TARGET_BOARD      = mvme8100
        EES_TARGET_ARCH       = powerpc
        EES_TARGET_OS_VERSION = 3.0.6
        EES_TARGET_OS_VARIANT = generic
        BPMD_BOARD_ID         = 1
else
        EES_TARGET_BOARD      = conc405x
        EES_TARGET_ARCH       = i686
        EES_TARGET_OS_VERSION = 3.16.7
        EES_TARGET_OS_VARIANT = generic
        BPMD_BOARD_ID = 0
endif

include /usr/local/products/elsd/include/elsd-latest.mk

EES_CFLAGS      = -I$(EES_ERL_LIBS)/cdev-1.2/include -I$(EES_ERL_LIBS)/acnet-2.1/include $(EES_PREF_CFLAGS)     $(EES_PREF_LXRT_CFLAGS) -fno-strict-aliasing -D'BOARD_SUPPORT=$(BPMD_BOARD_ID)'
EES_CPPFLAGS    = -I$(EES_ERL_LIBS)/cdev-1.2/include -I$(EES_ERL_LIBS)/acnet-2.1/include $(EES_PREF_CPPFLAGS)   $(EES_PREF_LXRT_CPPFLAGS) -fno-strict-aliasing -std=c++0x -I$(EES_INC) -I$(EES_ERL_LIBS)/cdev-1.2/include -I$(EES_ERL_LIBS)/acnet-2.1/include -I$(E
ES_PREFIX)/usr/include/libnl3 -D'BOARD_SUPPORT=$(BPMD_BOARD_ID)'
EES_LDFLAGS     = $(EES_PREF_LDFLAGS)   $(EES_PREF_LXRT_LDFLAGS) -lbbpm250x12 -lbtime -lnl-genl-3 -lnl-3 -lrt -lconfig++
ERLANG_INT      = erl_interface-3.10.4
MYLIBS          = $(EES_ERLANG_LIBDIR)/lib/$(ERLANG_INT)/lib

MODULES = \
        BBPM250x12DAQDeviceAdapter.o \
        BBPM250x12DAQReader.o \
        BoosterStateMachine.o \
        BPM.o \
        BPMError.o \
        BPMSample.o \
        BTimeTSGDeviceAdapter.o \
        CLIAbortCommand.o \
        CLIArmCommand.o \
        CLIClearReadingsCommand.o \
        CLICommand.o \
        CLIConfigReloadCommand.o \
        CLIDumpCommand.o \
        CLIDumpFlashCommand.o \
        CLIReadoutCommand.o \
        CLIHelpCommand.o \
        CLIListCommand.o \
        CLIShowCommand.o \
        CLIShutdownCommand.o \
        CLISMDisableCommand.o \
        CLISMEnableCommand.o \
        ClosedOrbitReading.o \
        ConfigurationManager.o \
        Controller.o \
        ControlMQClient.o \
        DAQClosedOrbitReadoutRequest.o \
        DAQFlashOrbitReadoutRequest.o \
        DAQFlashTurnReadoutRequest.o \
        DAQRawReadoutRequest.o \
        DAQReader.o \
        DAQReadoutRequest.o \
        DAQReadoutRequestQueue.o \
        DAQSmoothOrbitReadoutRequest.o \
        DAQSystem.o \
        DAQSystemArmed.o \
        DAQSystemFailure.o \
        DAQSystemState.o \
        DAQSystemStatus.o \
        DAQSystemInitialization.o \
        DAQSystemReadout.o \
        DAQSystemReady.o \
        DAQSystemTriggered.o \
        DAQTBTReadoutRequest.o \
        FlashReading.o \
        IReading.o \
        MapCLICommands.o \
        MachineContext.o \
        Measurement.o \
        MeasurementSpecification.o \
        NoMachineStateMachine.o \
        RawReading.o \
        RawSample.o \
        ReadoutSpecification.o \
        SharedMemoryClient.o \
        SharedMemoryHeader.o \
        SharedMemoryManager.o \
        SmoothOrbitReading.o \
        StateMachine.o \
        TBTReading.o \
        TCLKTimestamp.o \
        TriggerSpecification.o

all: all_buildroot

all_buildroot: bpmd bpmcli boosterbpm_acsys boostercrate_acsys

clean:
        -rm -f *.o bpmd bpmcli boosterbpm_acsys boostercrate_acsys

boosterbpm_acsys: $(MODULES) BoosterBPMACNET.o boosterbpm_fef_init.o
        $(CXX) $(MODULES) BoosterBPMACNET.o boosterbpm_fef_init.o -o boosterbpm_acsys $(EES_LDFLAGS) \
        $(EES_ERL_LIBS)/cdev-1.2/priv/fef_driver_lib.o -L${MYLIBS} -lpthread -lerl_interface -lei

boostercrate_acsys: $(MODULES) BoosterCrateACNET.o boostercrate_fef_init.o
        $(CXX) $(MODULES) BoosterCrateACNET.o boostercrate_fef_init.o -o boostercrate_acsys $(EES_LDFLAGS) \
        $(EES_ERL_LIBS)/cdev-1.2/priv/fef_driver_lib.o -L${MYLIBS} -lpthread -lerl_interface -lei

bpmcli: $(MODULES) cli.o
        $(CXX) $(MODULES) cli.o -o bpmcli $(EES_LDFLAGS)

bpmd: $(MODULES) main.o
        $(CXX) $(MODULES) main.o -o bpmd $(EES_LDFLAGS)

install_buildroot_target:
        install -d $(EES_BIN)
        install -D -m 0755 $(@D)/bpmd $(EES_BIN)
        install -D -m 0755 $(@D)/bpmcli $(EES_BIN)
        install -D -m 0755 $(@D)/boosterbpm_acsys $(EES_PREFIX)/usr/local/products/acsysfe/lib/cdev-1.2/priv
        install -D -m 0755 $(@D)/boostercrate_acsys $(EES_PREFIX)/usr/local/products/acsysfe/lib/cdev-1.2/priv
        install -D -m 0755 $(@D)/example-bpmd.conf $(EES_PREFIX)/etc/bpmd.conf
        install -D -m 0755 $(@D)/startacsys-bpmd $(EES_PREFIX)/usr/local/products/acsysfe/bin/startacsys
        install -D -m 0755 $(@D)/S70bpmd $(EES_PREFIX)/etc/init.d/

install_test: all
        scp -q bpmd nova:/fecode-bd/vxworks_write/fe/ees/bbpmts
        scp -q bpmcli nova:/fecode-bd/vxworks_write/fe/ees/bbpmts
        scp -q boosterbpm_acsys nova:/fecode-bd/vxworks_write/fe/ees/bbpmts/
        scp -q boostercrate_acsys nova:/fecode-bd/vxworks_write/fe/ees/bbpmts/
        scp -q example-bpmd.conf nova:/fecode-bd/vxworks_write/fe/ees/bbpmts/bpmd.conf
        scp -q S70bpmd nova:/fecode-bd/vxworks_write/fe/ees/bbpmts

install_bbpmt2: all
        scp -q bpmd nova:/fecode-bd/vxworks_write/fe/ees/bbpmt2
        scp -q bpmcli nova:/fecode-bd/vxworks_write/fe/ees/bbpmt2
        scp -q boosterbpm_acsys nova:/fecode-bd/vxworks_write/fe/ees/bbpmt2
        scp -q boostercrate_acsys nova:/fecode-bd/vxworks_write/fe/ees/bbpmt2
        scp -q S70bpmd nova:/fecode-bd/vxworks_write/fe/ees/bbpmt2

Changes to elsd.mk¶

The elsd.mk Makefile is tracked in the ees-buildroot project Git repository under the elsd/ directory. To deploy changes so other developers on adlinux can use them, use the install_elsd target provided in the Buildroot Makefile:

make install_elsd VERSION=<version>

The VERSION argument is required. If you omit it, a directory listing of the current elsd include directory will be displayed to remind you of the current version number.

Remote Deployment¶

You can transfer new builds to your target using scp. Most targets live inside the controls firewall so in order to scp you must first configure ssh to proxy through outland.fnal.gov. Add the following to your ~/.ssh/config file:

Host target.fnal.gov
      ProxyCommand ssh user@outland.fnal.gov nc %h %p

Then to copy your_program to the target:

# scp -q your_program root@target.fnal.gov:/usr/local/products/bin/your_program

Keep in mind that the root filesystem is in a ramdisk, so the new file that you copy over will disappear after you reboot unless you rebuild and deploy your kernel+ramfs with the new changes. To deploy a new kernel, you can add an entry for nova.fnal.gov to your ~/.ssh/config file and copy the kernel to the linux_boot area:

# scp -q output/images/bzImage nova.fnal.gov:/fecode-bd/linux_boot/ees/project/bzImage.dev

Remote Debugging¶

You can debug executables on your target using GDB's remote features. Start the application with gdbserver:

$ gdbserver :2345 /usr/local/products/bin/your_program

Then, from the host that you built ees-buildroot on, establish a secure tunnel for GDB into the controls vlan:

$ ssh -f -N -L 2345:target.fnal.gov:2345 outland

Alternatively, you could log into the controls VPN and address your target directly.
Move into the build directory and connect to the target via the tunnel:

$ cd output/build/your_program
$ gdb your_program
(gdb) target remote localhost:2345

Project

General

Profile

AD EE Support Department » Linux-Buildroot » ees-buildroot

Wiki