Broken art_sam_wrap.sh in 1_7_0
There are some apparent problems art_sam_wrap.sh in ifdhc 1_7_0. It seems that the input and outputs of at least two ifdh operations.
I'm not sure if establishProcess ever actually caused a problem problem, but it seems safe to update the line in art_sam_wrap to pass an empty string:
consumer_id=`IFDH_DEBUG= ifdh establishProcess "$projurl" "$appname" "$ART_VERSION" "$hostname" "$GRID_USER" "art" "$description" "$limit" ""`
The change is just the "" at the end.
A real problem was induced by the was in the fetchInput line. art_sam_wrap seems to expect that the last line in stdout is the file name. It gets the file name like this:
fname=`IFDH_DEBUG= ifdh fetchInput "$uri" | tail -1 `
For some reason, however, ifdh fetchInput no longer has newlines, all of the output comes out in one line.
I've put in a hack to get around this:
fname=`IFDH_DEBUG= ifdh fetchInput "$uri" | sed "s/\ \//\n\//g" | tail -1 `
But really, the whole thing could probably be made to be more robust.
#1 Updated by Marc Mengel over 5 years ago
While I haven't been able to reproduce the exact problem you describe, there is one issue
that needs fixing in the art_sam_wrap.sh in v1_7_0 which we can easily work around -- it doesn't know how to find itself. It still has v1_2_10 as the version of itself to setup --
So if you get a chance, could you try it again, but pass:
in the command line options to art_sam_wrap.sh, so it can find ifdh_art, etc.?
#4 Updated by Dominick Rocco over 5 years ago
Ok, now I understand why I typed 1_7_0, I took it from "which art_sam_wrap.sh"
I can reproduce it with this script:
It sets $SAM_PROJECT_NAME, $USER and $CLUSTER to mimic the grid environment, then runs art_sam_wrap.sh.
#5 Updated by Marc Mengel over 5 years ago
- Status changed from New to Resolved
- Target version set to v1_7_2
- % Done changed from 0 to 90
- Estimated time set to 3.00 h
Okay, found it. Turns out I sent all the debug messages to stderr except for one, which
on top of it doesn't end in a newline. Thus it gets smushed on the same line with the
input file, depending on lock file output...
So it looks like this is a fix for v1_7_2, and we're going to have to drop v1_7_1.
#6 Updated by Marc Mengel over 5 years ago
Hm... Maybe I should explain that more clearly:
earlier releases, with IFDH_DEBUG on, printed lots of messages to stdout, hence
we needed to tail -1 the "ifdh fetchInput" output. The plan was in this release,
to send all of that to stderr, and thus the tail -1 would be redundant and not needed.
However, I missed one message going to stdout, which on top of that didn't end in a newline;
so while we used to print it, more debug output that did have a newline got printed after it,
and the tail -1 thing still worked. Now however, all those other debug messages are going to
stderr, so the output only has the newline-free debug message and then the filename on one line.
Second, the art_sam_wrap script is running ifdh fetchInput with IFDH_DEBUG= (that is blank) in an
attempt to turn of debugging, but that didn't work because the code was just looking to
see if IFDH_DEBUG was set at all, and a blank string still counted, so it was actually always
turning the debug ON.
However the convolving factor was that CPN lock output was still going to stdout, so if you
ran the art_sam_wrap script on files coming from bluearc, the LOCK output put a new line in
the output stream and the whole thing worked again, so my tests of art_sam_wrap, which
happened to get files from bluarc and thus the cpn LOCK output, worked. But when you pull
from /pnfs, boom. you get no lock output and the debug message on the same line with the actual
local file path.