Support #6253

Minos+ Fermilab TSW

Added by Arthur Kreymer over 6 years ago. Updated 5 months ago.

Start date:
Due date:
% Done:


Estimated time:
40.00 h
Spent time:
Duration: 290


Review and update the 2014 draft Minos+ Fermilab TSW


#1 Updated by Arthur Kreymer over 6 years ago

Date: Tue, 13 May 2014 10:51:56 -0500
From: Keith Chadwick <>
To: Arthur Kreymer <>
Cc: Keith Chadwick <>, Ruth Pordes <>,
Margaret Votava <>
Subject: Minos TSW


As you may be aware, we are currently working through the various experiment
specific MOUs and
re-writing them under the aegis of Fermilab's TSW (technical statement of
work) program and the
CS ITIL initiatives.

A highly draft of the CS-Minos TSW has been written, and it is now ready to
receive your edits:

Please download this document and start the editing process.

I am available on site this week if you have any questions, next week I will
be in Europe for 9 days at
HEPiX and CERN, but will be available via email.


#3 Updated by Arthur Kreymer over 6 years ago

  • Description updated (diff)

#4 Updated by Arthur Kreymer over 6 years ago

Jerry noted the need to change Minos to Minos+

I concur, and we should add very early language
noting that Minos+ refers to both Minos and Minos+

#5 Updated by Arthur Kreymer over 6 years ago

Keith Chadwith asked the status of our draft.

I had done a paper quick scan,
mainly removing direct and indirect NOvA content.

I stored CD-Minos-TSW in our wiki for editing today.
This is tagged 0.4 and dated 5/6/2014 in Sharepoint.

I also noticed a second document in Sharepoint,
Minos Tech Statement of Work, dated 5/13/2014,
which seems to have mostly specific Minos+ content,
including the Minos+ official logo

#6 Updated by Arthur Kreymer over 6 years ago

I have posted a much better draft to

Notes :

Minos means Minos and Minos+ throughout
Section Note
1       Mentioned SCPPM, added Batch/Sim/Core group, removed Database Support non-existent group
1.1     Changed CS-DocdB to CD-DocDB ( that's what it is )
1.2     Mentioned disk and tape
2.1     Removed Windows Domain, we do not use this for ops
2.3.1   Added http://nusoft/minos web site
2.5     FD database is not supported by CS
2.5.1   Removed Oracle (not in use by the time this will be approved)
        SAM references are now all to Postgresql in this document
2.8     Calls for network switch list and diagram. 
        I think this belongs in Minor Application.
        Corrected Bluearc exports, removed /grid/app no longer used
        AFS home areas can move elsewhere
        AFS is present home of www-numi, will move to new central servers
2.10    Readytalk is used for  meetings
        Point to point video used to certify remote shift stations
3.1/3.2 Need review by DAQ group.
3.3.1   corrected local/remote computing description
3.3.3   Removed the detailed info regarding GridFTP and group IDs
3.3.6   Removed a stray paragraph duplicated from the batch section
        Removed the 'more and more' phrase Changed CPU to Computing
3.4.2   Do we really need a list of PREP equipment in the TSW ?
3.6.1   Added reference to the Minor Application
3.7     Removed references to ART
3.7.3   Corrected FTS to reflect Minos future usage
3.8.1   Corrected from NOvA to Minos raw data numbers small wordsmith, changed small to volatile for project disk
  5       Allow AFS or other for home areas
  6       No exception for monitoring of tape and disk
3.9.1   Added Conditions database to the list and described appropriately
        Changed support from 24x7 to 12x7 Removed mention of Minor Application which is DAQ-only

#7 Updated by Arthur Kreymer almost 6 years ago

The Minos+ TSW draft went missing from Redmine,
but was restored Dec 12, as noted in

Main items needed :
o DAQ update, including network list and diagram
o PREP equipment list ?
I don't think this belongs in TSW, but could be referred to.
o Update DocDB support level ( 24x7 usage, 12x7 support )

#8 Updated by Arthur Kreymer almost 6 years ago

  • Due date set to 02/26/2015
  • % Done changed from 0 to 30

I think much of the TSW DAQ content can be taken from the 2013 MOU draft.
It seems the PREP bit can just refer to the overall Fermilab MOU.

I think that email notification from thie Wiki are working again,
we'll see when I post this.

#9 Updated by Arthur Kreymer almost 6 years ago

  • % Done changed from 30 to 80

I have posted a semi-final version of the Minos TSW.

See CS-Minos-TSW-20150310.pdf

See the detailed notes there.
Main things needed before signing are
o network map and list of routers
o cleanup of footnotes and references

#10 Updated by Arthur Kreymer almost 6 years ago

This draft is converted to strict .docx form for CS review.

It contains the latest Networking content from the Networking group per RITM0191697

I am putting drafts only on Fermipoint now,
as these are starting to be looked by CS people.

April 18 2015
Version 0.951

1 - Typo: both the the  - to the

1 - Removed reference to DAQ computers in the control roomt
    these no longer exist

1.2.2  Typo 6 10 years -> 10 years

2.3.1 added web site ( any day  now )

2.5 Soudan Database servers -> databases

2.7 Integrate new draft tables and graphics from 

2.8 Removed one stray <experiment>

2.8.1 corrected case for Blue1 -> blue1 etc

2.8.2 corrected bullets indentation,

2.8.2 AFS
      Added /afs/ path
      Added migration to /nashome and /web/sites/

3.2.1 Corrected font to Cambria 11 from 10

      Moved Respsibilities from FIFEMON to Grid section

3.3.2 Removed the null list of Prep Enhanced Services

3.4.2 Capitalized Redmine

3.4.6 Removed 'if necessary'  from use of Build Service

3.6   Corrected desktops typo

3.7.1 corrected samweb typo

3.8.1 ordered beams NuMI and BNB
      Terabytes and files  dCache typos

3.8.1 SCPPM -> SPPM for consistency

3.9.1 Tickets -> Service Desk Incidents maintain typo to -> for long term  - typo

3.10    maintenance typo

#11 Updated by Arthur Kreymer almost 6 years ago

  • Status changed from New to Assigned
  • Priority changed from Normal to High

This draft has improved network diagrams, imported from original .png images.
No more dark grey backgrounds.

A remaining document format issue is the inability to have
four-digit numbered elements.
Pushed a few items down to n.n.n.n.n to get around this.

Will need expert help to get things all properly numbered.

#12 Updated by Arthur Kreymer over 5 years ago

The Minos TSW in Fermipoint has been updated, per notes from Margaret Votava.
Mainly we removed a lot of text that belonged in the SLA.

Date: Wed, 22 Apr 2015 17:02:47 -0500
From: Margaret Votava <>
To: Arthur E Kreymer <>
Subject: minos TSW

Hi Art, 

Do you believe it to be done now? If so, please send it out to quadrants leads and Bakken for

I have a few: 

 *  The database hosting is meant for items that the experiment negotiates directly with the DBA
    team. If it is used as an underlying technology for other services, it should not be called out.
    That will be in the SLA for that service. 
 *  Section 1 under “contacts”, Brian in mentioned by name and role, you are mentioned by role only.
    I think you should include your name. 
 *  Section 1.1. I think think we can guarentee that change and maintenance activities will not
    adversely affect operations. Maybe we want to say minimize impact? 
 *  Section 1.2. To be clear, all of Minos production and reconstructed data is meant to live on
    disk. You never plan to stage from tape. 
 *  Section 2.5. I think it can all be deleted with the exception on the information regarding the
    far detector database. 
 *  2.5.3, 2.5.4, 2.5.5 need to be promoted up a level. 
 *  3.4.3. You don’t want standard ECL services, you want enhanced SLA services. How it is handled
    with PPD is described in that SLA.
 *  3.7.1. We won’t be supporting things 8x7, but 12x7 in this extended response time response
 *  3.7.1. I don’t think SCD will debug file delivery problems to remote sites. We would triage. 
 *  3.7.1. I don’t think should have the on shift person notify the offline manager in critical
    incidents. We should talk that through. 
 *  3.7.1 I don’t know how we can plan for the unplanned. 
 *  Section 3.9 is really "scientific databases applications" 
 *  The ECL should only be mentioned under “scientific collaboration tools”, nowhere else – not under
    database hosting or database applications. 
 *  The only offerings Minos uses in 3.9 is the conditions database application and the IFBEAM. The
    others should be elsewhere.  See Steve White wrt to the applications SLA. 
 *  3.9.1. Take out reference to Minerva. 
 *  3.9.1. IF beam will only be 12x7
 *  3.9.2 needs to be reworked. 
 *  There is no mention of the production operations service? 


#13 Updated by Arthur Kreymer over 5 years ago

Sorry, I overlooked adding OPOS to the TSW in the latest draft.
Rebuilt the TOC for legibility .
Changed SAM and FTS to Enhanced, for 7 day support.

#14 Updated by Arthur Kreymer over 5 years ago

I corrected the OPOS group name to
Offline Production Operations Service
and corrected to service offering, pser the SLA draft, to
Experiment Offline Production Processing

And added the EOPP SLA 5418 to the SLA table.

#15 Updated by Arthur Kreymer over 5 years ago

To: wingmc,bakken,fuess,lyon,votava
CC: jthomas,lang,minos_batch
Subj: Minos TSW draft review

Please review the Minos TSW draft dated April 28, at

Margeret suggests a 2 week review, targeting signatures around May 15.

#16 Updated by Arthur Kreymer over 5 years ago

I will make the requested changes to the Web Services and DocDB sections.

Date: Fri, 1 May 2015 12:04:13 -0500
From: Ramon C. Pasetes <>


NCS has no changes to this TSW.


Date: Fri, 1 May 2015 12:27:42 -0500
From: Michael K. Rosier <>
To: Arthur E Kreymer <>
Cc: Peter J. Rzeminski II <>


I reviewed the Minos TSW and I think you might want to modify section 2.3.1 to reflect the SLA a bit

Current: ======
2.3.1 Apache Central Web Server - Shared Virtual Host
Note this SLA provides 24x7 support for this offering. The Minos collaboration has the following
websites covered by this SLA

Proposed: ========
2.3.1 Apache Central Web Server - Shared Virtual Host
Note this SLA provides:
- 24x7 availability (99.2% uptime)
- Offering Support Availability: 8 x 5 Monday through Friday, 8am - 5pm U.S. Central Time, not including
Fermilab work holidays.

In the event that there are critical security incidents involving the sites (,, listed in this TSW, such as a defacement, the Web
Systems Administration team would address those 24x7. I just want to make sure there's no expectation
for general support requests to be handled during non-business hours.

Hopefully this helps.


Date: Fri, 1 May 2015 13:07:15 -0500
From: Joe Klemencic <>
To: Jon A Bakken <>,
Subject: Re: Fwd: Minos TSW draft review

Are there any security concerns for blocking/exemption from blocking?

On 5/1/2015 9:49 AM, Jon A Bakken wrote:

Please review. Given short time period, send your changes to Art
directly and cc me.


*Jon A. Bakken *
/Head, Core Computing Division /

Date: Fri, 1 May 2015 18:39:08 +0000 (GMT)
From: Arthur Kreymer <>

I think that security concerns are covered by the standard SLA's.

Minos formarly had a Minor Application Plan documenting special needs.
The MAP would have been menioned in older MOU's.

But MAP's are not longer being used.

Current policy and practice is to abide by all security policies,
applying for individualy exemptions as needed.
This is already mentioned in the DAQ section of the TSW.

Date: Fri, 1 May 2015 13:42:03 -0500
From: Joe Klemencic <>

Great, thanks. As they come in, we'll process them.
We don't want to block an active DAQ system, so it is important to get them in.

Date: Fri, 1 May 2015 14:13:24 -0500
From: Gerald M Guglielmo <>
To: Arthur E Kreymer <>
Cc: ccd-smgt <>, Heath B O'Connell <>, Keenan Newton <>

    Section 2.8 say 24x7 support for DocDB. I asked Keenan Newton about this and he said, “DocDB is 8x5.
We don’t have the resources for constant on call staff.” Please update that section to reflect 8x5

Date: Fri, 1 May 2015 19:54:36 +0000 (GMT)
From: Arthur Kreymer <>

Sorry, that should have been corrected to match the Microboone TSW.
I will update this, to specify
24x7 service availability with 8x5 service support

I would prefer just to refer to Enhanced DocDB support per the SLA,
but cannot find an SLA for DocDB.

Just an apparent 2014 draft lacking target response times.

This is perhaps an issue, as the official SLA's are in DocDB.

Date: Fri, 1 May 2015 15:13:28 -0500
From: Heath B O'Connell <>

Dear Art,

We're currently finishing up the Content Management SLA which will include DocDB (along with SharePoint
and Indico). It will have 24x7 service availability (as a goal) with 8x5 service support.



#17 Updated by Arthur Kreymer over 5 years ago

I added the requested CWS and DocDB updates in draft 0.97 of the TSW
There have been no other changes requested.

2.3.1 Apache Central Web Server - Shared Virtual Host

This SLA provides
24x7 availability (99.2% uptime)
Offering Support Availability: 8x5 Monday through Friday,
8 AM - 5 PM U.S. Central Time not including Fermilab work holidays

2.8 DocDB

24x7service availability with 8x5 service support is requested.

#18 Updated by Arthur Kreymer over 5 years ago

Minos TSW updated to version 0.98
Via fermi-ts -> RemoteApp

Cosmetic changes for consistency with the Template TSW

o Restored a missing fragment of the 3.7.1 SAM line including...
the samweb client, SAM projects, and monitoring.

o Added 3.3 GPCF from the Template,
This had been incorrectly listed as a Grid/Cloud reponsibility.

#19 Updated by Arthur Kreymer over 5 years ago

  • % Done changed from 80 to 90

Per discussion with Gabrielle and Igor, IFBEAM support is Standard,
Added a note regarding the special IFBEAM collector file copies
for legacy Minos database applications.

#20 Updated by Arthur Kreymer over 5 years ago

All suggested revisions have been incorporated.
There have been no further suggestions since May 14.

I have posted V1.0 to CS-doc-4256, version 7 for signature.

#21 Updated by Arthur Kreymer over 5 years ago

Margaret has started the signature process under REQ000000232187

#22 Updated by Arthur Kreymer over 5 years ago

  • Status changed from Assigned to Resolved
  • % Done changed from 90 to 100

2015/07/28 email from votava indicates the TSW is signed.

Thanks to all !

#23 Updated by Arthur Kreymer 5 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF