SAM Meeting Minutes:
Nate, Gavin, Jan, Michael G, Dominick, Chris, Eric, Robert I, Andrew, Peter, Ryan, Craig, Adam
8 to 10 days for small percentage (up to 10%) of priority production streams
Request to purge issued
Robert I update:
33k files deleted (raw2root)
Uncounted from calibration
12k are left
8k are errors
4k are waiting for confirmation that they made it to tape
Queue is empty
FTS improvements implemented
Slower fs easier to handle
MetadataExtractor is hacked, don't use as is
Need to update to latest ART version to pick up new extraction tools
XRootD is still untested, by us, need to get ART on data management areas
350 to 400 files and hour, will work for 10% sample size we're planning
BUT need to coordinate, SAM Gatekeeper needed for the near future
Biggest fire is calibration constants, Chris needs to be able to declare his files to SAM, so he can run over them.
FTS completion time ~30 hours
Run over them by hand just like MC to get the constants would probably be faster.
Using the BlueArc drop box system would just land us where we were a week ago, out of disk space, and the throughput would be the same.
Start with 100 files of FD cosmics then scale up by factors of 2
In the future, should think about running multiple stages in the same job (raw2root, reco, pid), but aren't ready yet with reco stages. This cuts down on intermediary files and sending files over the network.
Chris will run data calibration constants manually, not through SAM.
Michael is the data managing contact. He is to be contacted of any ramp up. He will be in WH12XO in Andrew's office.
dCache dropbox is ready for slow submission. FTS restart needed when dCache drop box is configured. Gavin will configure the drop box.
FTS status page, best resource
Can look at enstore dcache pages, not that helpful
Michael is a good contact here as well
Need more people to help monitoring.
Chris runs data calibration constants manually
Gavin configures dCache dropbox
Dominick to help with MC reco processing
Luke will run absolute calibration for data after Chris is done
Batch data production to begin
Database issues still a mystery
Low on space on Bluearc, needs addressing soon
Jan and Pavan to start coordinating today
Gavin is SAM Gatekeeper
Educating users is important. Discussion about tutorial and best practices.
Need to follow up offline.