Project

General

Profile

Fermicloud Test Setup

Currently the web service is setup for testing on fermicloud090.fnal.gov, port 20003,
attached to the minerva development SAM setup.

So we'll be using the calls as described in Interface definitions with a base URL of:

http://fermicloud090.fnal.gov:20003/sam/minerva/api

The calls all work as GET or POST calls as advertised, the POST calls also work as
GET calls to simplify testing, but aren't recommended to be called that way in
production, in case someone puts a web cache in front, etc.

For a full script using these calls, see source:gaudi/wrap_gaudi.sh.

If you get connection refused errors, you log into fermicloud090, and
ups start sam_web development
to start the web service. It's currently running under "mengel", and
cherrypy restarts automatically if the files are updated, so doing a

  setup sam_web development, git
  cd $SAM_WEB_DIR
  git pullin  

should get it up to the latest revision and start the latest code running.

Testing it with bash/curl

  1. Pick a unique project name.
       project="${USER}_`date +%s`_$$" 
    
  2. Find a dataset name (i.e. run_798_raw)
       dataset="run_798_raw" 
    
  3. start a project with:
       url=`curl \
             --data name=$project \
             --data station=minerva \
             --data defname=$dataset \
             --data username=sam \
             --data group=minerva \
             --output - \
             http://fermicloud090.fnal.gov:20003/sam/minerva/api/startProject`
        echo project url is $url
    
or in python:
import urllib
params = urllib.urlencode({
        'name': projectname, 
        'station': 'minerva', 
        'defname': defname,
        'username': 'sam',
        'group': 'minerva',
})
f = urllib.urlopen("http://fermicloud090.fnal.gov:20003/sam/minerva/api/startProject", params)     
url = f.read()


This will give you back a url in shell variable $url
  1. In a worker-job, find the project's url with:
       url=`curl \
             --data name=$project \
             --data station=minerva \
             --output - \
             http://fermicloud090.fnal.gov:20003/sam/minerva/api/findProject`
        echo project url is $url
    

    This will give you back a url in shell variable $url
  2. Register a consumer process with:
       cpid=`curl   \
              --data appname="demo" \
              --data appversion="1" \
              --data deliverylocation=$hostname \
              --data username="sam"  \
              --output - \
          $url/establishProcess`
       echo CPID is $cpid
    

    This will give you your consumer process id in $cpid
  3. Now we'll have a base url for this process:
       consumerURL="$url/process/$cpid" 
    
  4. Get a file (note the use of --retry to have it poll for a file)
        file=`curl \
              --retry 150 \
              --data '' \
              --output - \
              $consumerURL/getNextFile`
    
  5. release the file when done
         curl \
              --data "status=$status" \
              --data "filename=$file" \
              $consumerURL/releaseFile
    
  6. end your process (Note the use of --data '' to force a POST)
         curl \
              --data '' \
              $consumerURL/endProcess    
    
  7. set your status
         curl \
              --data "status=completed" \
              $consumerURL/setStatus
    
  8. Repeat steps 4..9 for each consumer; then end your project (if you're done)
         curl \
              --data 'force=1' \
              $url/endProject