Backward compatibility testing for art releases

The goal of backwards compatibility testing is to achieve reasonable certainty that new code is capable of reading old data files. The current version of art provides a complete i/o system only through the modules RootInput and RootOutput, so it is the backwards compatibility of files written and read using these modules that is addressed here. Backwards compatibility issues for any additional i/o format would need to be addressed when that i/o format is introduced.

What needs to be tested?

We need to test code we write that is used for schema evolution of data products. This does not include writing tests to verify ROOT's automated schema evolution (except in special cases where we identify a bug in ROOT we need to track). This does include any introduction of or changes to streamer code we write, or custom i/o rules in selection XML files.

We need to test any changes in the storage of metadata (all the things stored in an art/ROOT file that are not data products), and for any changes in the structure of the file storage of these things (e.g., the renaming of a TTree object).

Each change in art/ROOT file format version indicates that a change in the file structure has been introduced, and so backward compatibility tests should be introduced for that version.

When have art/ROOT file format versions changed?

File format version art version Release notes
12 > 2.10 not yet released
11 2.09.00 2.09 release notes
10 2.08.00 2.08 release notes
9 2.07.03 2.07.03 release notes
8 1.14.00 1.14.00 release notes

Since nobody is using a version of art older than 1.14.00 in production, we do not need to introduce backward compatibility testins for older versions of art.

The plan for testing.

We need to assure:

1. Translation from each old disk format to the current in memory format is correct.

2. Writing the recovered in-memory format to the current disk format is correct.

To do this, for each supported old file format, we need a test that will:

1. Read the old format, and compare the result to the expected in-memory format. This can be done with an analyzer module for each old format we still support, which verifies that the old format converts to the new in-memory format correctly. This test should also write the translated products to disk (in the new format, of course).

2. Read the newly-translated data in another program, and compare with the expected in-memory format, using the same test module.

These tests should exercise each of the facilities we provide for writing data products:

1. A data product that is a vector of simple structs.
2. A PtrVector into such two such data products; the PtrVector should contain pointers into each of the two products.
3. An Assns collection.
4. A map_vector.
5. A data product with embedded Ptrs.
6. A aggregable subrun and run data product.
7. An unaggregable subrun and run data product.

What about forward compatibility testing?

The art project has never provided a guarantee that files written by new releases of art could be read by old releases of art. Backward compatibility testing does not provide such assurance.