Project

General

Profile

Obstac for observers

DES observers should read from here through the "Surprises and gotchas" section

What obstac does

During observing, obstac has two states: autoobs=on and autoobs=off. When autoobs is off, obstac does nothing. When autoobs is on, it monitors the length of the queue and, whenever the duration of the queue falls below the configured targed queue duration, it adds observations to the queue until the duration exceeds target queue duration, and then stops until the next time the queue falls below it.

An indicator above the observing queue shows whether autoobs is enabled:

Currently, the configured queue duration is 900 seconds. This may be modified be setting the min_duration keyword in the queue section of the obstac configuration file, and restarting the obstac SISPI roles.

Turning obstac on and off

Autoobs can be turned of and off using the "Enable Auto" (or "Disable Auto") button in the exposure control tab of the observer console:

Redoing observations

If observers note that an exposure is unambiguously bad, and the observation needs to be repeated, observers can mark the exposure to be redone. Find the offending exposure in the exposure browser in the SISPI interface, and select it to bring up details on that exposure. One of the details is a "redo" checkbox; checking this checkbox will mark it as needing to be redone. This mechanism does not allow for old (previous night? previous instance of SISPI?) to be declared bad. See the instructions in "Obstac for SISPI experts" for more on how to declare older exposures bad.

In the future, there will be a mechanism for the output of data management processing to be used for declaring observations as needing to be repeated. Until this mechanism is ready, the instructions in "Obstac for SISPI experts" will need to be followed for this task.

Surprises and gotchas

Everything here is work-arounds for bugs. The bugs will eventually be fixed.

Filling to completion

A fix for this bug is awaiting testing.

Once obstac starts filling the queue, it will not stop until the queue duration has reached its target duration (900 seconds), even if autoobs is turned off during the fill. It will stop after it reaches the target duration; the only thing to do is wait.

The start-of-exposure race condition

This bug should be fixed now; any evidence of recurrence should be reported to the SISPI and obstac developers.

Obstac determines which exposures have been completed (and therefore do not need to be scheduled) by looking at the database of completed exposures, the exposure in progress, and the contents of the queue. Currently, there is a short period of time after SISPI removes an exposure from the queue but before it reports it as being "in progress," and the exposure is invisible to obstac during this time. If obstac selects an exposure in this time, it may select the observation just removed from the queue.

The work-around is to prevent obstac from selecting targets when SISPI is in this state. This can be done by turning off autoobs shortly before the end of each exposure (and therefore before SISPI removes the next exposure from the queue) and turn it back on again once the next exposure has begun (SISPI's count-down timer begins).

Obstac for SISPI experts

Configuring obstac

Finding the obstac configuration file

The obstac configuration file is set by the obstac_config_file option in the Architect "ini" file used to start the SISPI instance.

Switching to a new configuration file

When the configuration file is changes, the obstac-related roles in SISPI must be restarted in order. These roles are OBSTACSRVR, AUTOOBS, and OBSTACWEB.

Changing the tactician

Obstac selects exposure using bits of code (python callable objects) called "tacticians." The Global Tactician section of the configuration file sets the name of the callable object to be used, and the python module in which this object is defined:

[Global Tactician]
module = obstac.tacticians.FirstYearDeepFirstTactician
object = top_tactician

In most tacticians, the object name is top_tactician. The module can be set to any python module in obstac's PYTHONPATH that defines an object that implements the tactician API.

Changing airmass limits

The airmass_limit options set the maximum airmass at which obstac is willing to schedule an observation. This option appears twice in the configuration file: once in the Supernova Tactician section, which sets the limit for supernova observations, and once in the Survey Tactician section, which sets the limit for wide survey observations.

Note that obstac selects exposures based on effective exposure time (which maps to S/N on a point source), and the airmass is one parameter in this calculation: raising the hard airmass cutoff directly may not result in obstac being willing to observe a higher airmass.

Changing the sky brightness limits

The sky_limit options set the maximum modeled sky brightness at which obstac is willing to schedule an observation. This option appears twice in the configuration file: once in the Supernova Tactician section, which sets the limit for supernova observations, and once in the Survey Tactician section, which sets the limit for wide survey observations. The value of this option should be set to a sequence of five numbers, one for each filter, in the order g, r, i, z, and Y.

Note that obstac selects exposures based on effective exposure time (which maps to S/N on a point source), and the sky brightness is one parameter in this calculation: loostening the sky brightness cutoff directly may not result in obstac being willing to observe a higher sky brightness.

Changing the time for start and end of observing

The sun_min_zd option of the Day section defines the closest the sun can be to zenith for times which obstac will schedule an exposure, effectively setting the times of the beginning and end of observing.

Ignoring observations

One of the easiest ways to change obstac's behavior is to add exposures to the "obs_to_ignore" table.

Here is an example showing how to replace the current contents of this table with all hexes in the minisurvey:

[sispi@observer2 ~]$ psql -U neilsen -h server2 decam_prd
Password for user neilsen: 
psql (9.0.3)
Type "help" for help.

decam_prd=> SET ROLE decam_prd;
SET
decam_prd=> SET SEARCH_PATH TO obstac; 
SET
decam_prd=> DELETE FROM obs_to_ignore;
DELETE 400
decam_prd=>   INSERT INTO obs_to_ignore 
decam_prd->   SELECT hex_id, tiling_id, filter_id 
decam_prd->   FROM hexes
decam_prd->     NATURAL JOIN program_hexes
decam_prd->     NATURAL JOIN program_tilings
decam_prd->     NATURAL JOIN program_filters
decam_prd->     NATURAL JOIN programs
decam_prd->     NATURAL JOIN filters
decam_prd->     WHERE des_program_name IN ('mini1a', 'mini1b', 'mini2');
INSERT 0 1750
decam_prd=> \q

Redoing exposures

There are two mechanisms for telling obstac not to consider a given exposure as completing a needed observation. The first is the "discard" flag in the exposure table, which is set when the observers check the "redo" checkbox in SISPI. The second mechanism uses the "dm_done" and "dm_accept" flags of the exposure table, intended for recording feedback from data management. Tools for setting the later are not yet complete.

To set, for example, exposure 1234567 to be redone using the first method, one can use a query like this:

UPDATE exposure.exposure
SET discard=TRUE
WHERE id=1234567;

You can set multiple exposures, for example 1234567, 1234601, and 1234732, bad, one can use a query like this:

UPDATE exposure.exposure
SET discard=TRUE
WHERE id IN (1234567, 1234567, 1234601);

If you want to use the DM flags rather than the redo flag, you can replace setting the "discard" column with setting the "dm_accept" and "dm_done" columns. For example, the first of the above queries becomes:

UPDATE exposure.exposure
SET dm_done=TRUE, dm_accept=FALSE
WHERE id=1234567;

Developers documentation

Additional documentation aimed and obstac developers can be found in the doc directory of the obstac subversion product.