art 2.1 is dangerously creative with output file names
From a discussion on the art-users list I learned that in art v2.1
output files may or may not be written with the names specified by the
user. One of the comments was
This is another manifestation of the frequent conflict between
what's the right thing when you're dealing with valuable data, or in
a batch-processing environment, and what's convenient in a rapid
I think the current behavior is worse than that. In a "development
cycle" the old behavior of overwriting existing files was convenient.
If you don't want to overwrite "valuable data" on the other hand, the
correct behavior is to stop the program with an error message. If my
fcl configuration directs art to write out a file named "a.root", my
script will be reading "a.root" in the next step. If art ignores my
configuration and "creatively" comes with a different output file
name, I (or my script) will likely be looking at a stale file and
perhaps getting wrong physics conclusion.
I am filing this as a bug, because the newly implemented behavior
breaks the basic agreement that art software does what it is told to
do. It should either write user-specified files, or give an error if
it can't. It would be fine to have an option to switch between
"do-not-overwrite" and "ok-to-overwrite" modes, but making up new
output file names is just wrong.
#1 Updated by Thomas Junk about 4 years ago
There is a noclobber option to mv, and the noclobber environment variable can be queried (mv clobbers by default, even with set noclobber). The unmoved new file wouls have an uninformative name, RootOutputNNNNNN.root, instead of one based on what the user wanted.
But even then, there is a hazard that a user may ignore the error message and read a stale file.
#4 Updated by Kyle Knoepfel about 4 years ago
- Status changed from New to Assigned
- Assignee set to Kyle Knoepfel
- Estimated time set to 4.00 h
- SSI Package art added
- SSI Package deleted (
This issue was discussed at last week's stakeholders meeting. The resolution proposed at that meeting was:
Restore the old behavior (i.e. previously-written files are allowed to be overwritten); however, for configurations that invoke the new output file-handling facilities, require the presence of the sequence-number placeholder in the filename. If the placeholder is not present, throw an exception notifying the user of the expected filename requirements.
This proposal will be sent to the stakeholders' list.