Project

General

Profile

History

IFBeam Data

IFBeam Framework Performance Test

Performance Test
Data Server URL: http://dbweb0.fnal.gov/ifbeam/app/index

Data Access URL Syntax

CHEP 2012 Abstract

CHEP 2012 Implementation of IFBeam DB slides: https://cdcvs.fnal.gov/redmine/attachments/download/6560/IFBeamPresentation.pdf

Disk Upgrade plan for Jan 2013

IFBeam Offline Data Access
https://cdcvs.fnal.gov/redmine/projects/ifdhc/wiki/BeamFolders

IF Beam Data C API

Bundle Management Requirements

Troubleshooting for PPD

Legacy Beam Data C API (MiniBooNE)

This area contains the information for internal use by the group members

Data Flow

The switch-over does not require any manual script or program stopping/starting/restarting. Node role is controlled solely by presence of a file named 'PRIMARY_NODE' in ~dbiapp/ifbeam directory. Every possible combination is allowed in principle, though it is not recommended to keep the system running in any of the last two for a long time.

Node 1 Node 2 Mode
PRIMARY_NODE normal
PRIMARY_NODE normal
PRIMARY_NODE PRIMARY_NODE not optimal, creates additional load on DB server
data accumulation, data flow will resume when master role is assigned
The main differences in the master/standby node functionality are:
  • Master node watches the files in merge/long (merge/short) directory, sends them to the database and then moves into merge/long.2 (merge/short.2) directory, from where they will be compressed, archived and passed for remote copy to Minos node. The processed filename is sent to the standby node.
  • Standby node waits for the messages from the master with the processed filenames and moves them into merge/long.2 directory, from where they will be compressed and archived.

Data Flow Diagram


Local collector
       \                                                         /Primary? Data >-> DB; Filename >-> Standby node
        \                                                       /
storage/long >> push_out_mv.sh >> merge/long >> transfer_filter.py >> merge/long.2 >> gzip_move.sh  >> merge/long_archive
                                        /                      /                         \
                    remote push_out.sh-/   Master >-> Filename/                           \Primary? >> merge/long_export >> export_rsync.sh >->-> Minos node
                                      /
                  remote push_out.sh-/

Two possible scenarios of switch-over

  • Planned switch-over
    • Remove PRIMARY_NODE file on current master node
    • Create PRIMARY_NODE file on standby node
    • Reconfigure invoker on standby node using ifbeam app webpage to start data collectors
    • Done
  • Master node failure
    • Create PRIMARY_NODE file on standby node to restore data flow to the DB
    • Reconfigure invoker on standby node using ifbeam app webpage to start data collectors
    • Done

IFBeam Framework Performance Test

Performance Test