Outline for Paper

  1. Introduction
  2. Challenges for Modern Spectroscopic Surveys
    1. Why we and the field need something like this.
  3. What must pipeline do for a survey?
    1. what are the scientific setup and analysis steps that are needed to get this right; centered around describing the physics/astronomy of each module in the pipeline.
  4. A Solution: The SPOKES Pipeline
    1. Pipeline Architecture (Glue)

      This figure shows the underlying pipeline architecture; the concrete and struts holding the pipeline together. The architecture uses a specific i/o protocol to allow modules to quickly and safely access specific data needed only for that module's calculation.
    2. Data Format and Handling
    3. Modules
    4. Pipeline Flow

      This figure shows the pipeline flow from beginning to end, which data groups are used when.
  5. Results: Testing and Quality Assurance
    1. Data and Parameters
      1. Input: Simulation Data
      2. Input: Baseline Parameters

        This table shows the principle input parameters that might be varied to see the differing effects in the science output. Note that multiple modules use similar information, showing why we use the data format we do. This is actually even more important for hte galaxy data: no need to carry every galaxy tag around, only use what you need when you need it.
      3. Output: Survey and Cosmological Information
        Here we are lacking the plots that we need
    2. Quality Assurance Methods
    3. Computation Performance
      PLOT: time vs. Ngals; memory usage per module;
    4. Science Performance
      PLOT: dn(z)/dz; FoM; etc.
  6. Final Remarks and Conclusions
  7. References regarding large-scale pipeline simulations