jobsub_fetchlog doesn't allow to download logs for Production jobs using a different Production proxy
This issue has been reported by NOvA in INC000000937342.
The job with ClusterID firstname.lastname@example.org has been submitted using the NOvA managed proxy.
When the user get a Production proxy using his credentials, jobsub_fetchlog doesn't allow to download the logs and terminate with the following error:
jobsub_fetchlog -G nova --jobid email@example.com --dest-dir 5057545 --role=Production ERROR: Information for job firstname.lastname@example.org not found. Make sure that you specified the appropriate Role for this job. The role must be the same as what it was for submission. If you used jobsub_q or jobsub_history to find this job ID, double check that you specified --group incorrectly. If the job is more than a few weeks old, it was probably removed to save space. Jobsub_fetchlog --list will show the sandboxes that are still on the server. JOBSUB SERVER CONTACTED : https://jobsub01.fnal.gov:8443 JOBSUB SERVER RESPONDED : https://jobsub01.fnal.gov:8443 JOBSUB SERVER RESPONSE CODE : 404 (Failed) JOBSUB SERVER SERVICED IN : 0.639716863632 sec JOBSUB CLIENT FQDN : novagpvm14.fnal.gov JOBSUB CLIENT SERVICED TIME : 30/Mar/2018 16:40:01
While, if the user logs in as novapro, and set up the managed Production proxy, the command successfully download the logs.
#6 Updated by Dennis Box 5 months ago
- Assignee changed from Dennis Box to Vito Di Benedetto
- Status changed from Assigned to Feedback
Hi Vito, I am pretty sure this long standing problem has been cleared up by #22039 .
Could you please test this by submitting/fetching from fermicloud042.fnal.gov?
#7 Updated by Vito Di Benedetto 5 months ago
I submitted couple of jobs to test this using
fermicloud042.fnal.gov as jobsub server, and it looks like the issue is fixed.
I submitted jobs using the NOvA managed proxy with role Production.
- Then I was able to fetch logs as user
novaproand using the managed proxy.
- I was able to fetch logs as user
novaproand using a proxy with role Production created using my credential.
- I was able to fetch logs with my user, using a proxy with role Production created using my credential.
I also verified that on the production jobsub server the issue is still reproducible.