How to monitor the status of CI builds

The Lar_CI Web Application

To monitor the status of your builds into the CI system, our team created an experiment based web application to help you with your monitoring job.

The application can be found at the following link:

This application offer to you all the necessary tools to monitor the status and the performance of your builds in a short and long term.
It offers a quick overview of the status of the last ten builds and offers also dynamic graphs to show the overtime performance of the different build phases.

The main page of the web application

CI System Homepage

The previous image shows the homepage of the LarCI dashboard.

This dashboard is divided into four main sections:
  1. The tabs on the top to select the experiment specific dashboard
  2. The search and filter section that can be accessed through the ☰ button on the navigation bar
  3. The legend section ( on the right of the image )
  4. The main section with CI build status and details

The search and filter section

The purpose of this section is to offer to the user a tool to filter the results shown in the web application.
This panel allows you to:
  • Show the builds up to a defined build number.
  • Show the builds in a defined time range.
  • Filter the builds for platform or for test suite used.

The legend section

This section show the association between the color of the different phases of the builds,and their status
  • [#] yellow -> The phase is running
  • [#] gray -> The phase is pending
  • [#] green -> The phase was successful
  • [#] orange -> The phase failed in warning status
  • [#] red -> The phase failed
  • [#] light blue -> The phase have been skipped
    NOTE: A warning status is when the tests are successful but the product size or names are different when the output is compared to a reference file

The main section

This dashboard section is where the most important information are shown.
This section shows in a table of the status of the last ten CI builds executed.

The table is divided into multiple columns.

The first three columns show some basic information about the CI build number, the time when the CI build started and the platform on which the CI build have been executed.

The next columns show the status of the different phases of the CI build.
Each phase is represented by a square with a bullet

In this moment a Lar CI build s divided into five phases:
  1. Check revision
    Shows the status of the CI phase that checks if there were new commit to LArSoft repositories since the last CI build, if so, the CI checkout and build LArSoft from scratch, otherwise it uses the code built from the previous CI build and it skips the other CI phases.
  2. Checkout
    Shows the status of the checkout of the code from the repository.
  3. Build
    shows the status of the building process of the code into the CI System.
  4. Unit Test
    Shows the progress of the unit tests configured.
  5. Install
    Show the progress of the install process of the code into the CI System.
  6. CI tests
    Show the progress of the CI tests defined for this build.

Dynamic graphs and statistics

Every bullet and the build number into the web application is clickable.
Performing a click on a bullet open a different web page that shows detailed information on the phase.

Build Details

Checkout Phase Details

Build Phase Details

Unit Tests Details

Install Phase Details

CI Tests Phase Details