Project

General

Profile

Database Replication

There are two basic approaches to database replication that we make use of in the NOvA experiment: PostgreSQL built-in database replication, and table-by-table replication using a third party software package called RubyRep. The built-in database replication makes a copy of an entire database. This is extremely useful for backup purposes. Table-by-table replication offers greater flexibility, and is how we push data from our database tables in the master server at FNAL to slave servers at other institutions in ~real time.

Table-by-table replication

Set-up

Here are instructions for how to set-up table-by-table replication between the FNAL master and a local database.

  1. We assume at this point that you have installed a PostgreSQL dB server, with a database called "nova" on port 5432.
  2. Next, install RubyRep on the same machine as your dB server. This package is very easy to install, simply follow instructions on their website. We recommend use of the JRuby version, as installation is incredibly easy.
  3. Now generate a RubyRep configuration file:
    > rubyrep generate FNALdBRep.conf
    

    and edit the contents of the file so that they are:
    RR::Initializer::run do |config|
      config.left = {
        :adapter  => 'postgresql',
        :database => 'nova_prod',
        :username => 'novarep', 
        :password => '************', # contact Jon Paley for the password
        :host     => 'ifdbprod.fnal.gov'
      }
    
      config.right = {
        :adapter  => 'postgresql',
        :database => 'nova',
        :username => '[login account name]', # your choice
        :password => '************', # your choice
        :host     => 'localhost'
      }
    
      config.options[:sync_conflict_handling] = :left_wins
      config.options[:right_change_handling] = :ignore
      config.include_tables /./   # this replicates all tables
    end
    
  4. Check that RubyRep finds all differences between the two databases
  5. Initiate replication:
    > rubyrep replicate -c FNALdBRep.conf >>& FNALdBRep.log &
    

    If for some reason this rubyrep process dies or is killed at any time, simply execute the above command and RubyRep will correctly update the local copy of the database.

Tests

Here are some notes of tests done with rubyrep.