Project

General

Profile

Bug #22981

detect and work around bad cvmfs server during uploads

Added by Dennis Box 4 months ago. Updated 8 days ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Target version:
Start date:
07/23/2019
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:
Duration:

Description

rcds02.fnal.gov went belly up a couple of days ago. When the jobsub client tries to check for a tarballs existence, it expect 'PRESENT' or 'MISSING" as a reply. Rcds02 was returning 'Unrecognized DN'.
exists_url = [u'https://rcds01.fnal.gov/pubapi/exists?cid=nova/f3e4886a9c098ca5d14a8b0f6f35f6538268fc10', u'https://rcds02.fnal.gov/pubapi/exists?cid=nova/f3e4886a9c098ca5d14a8b0f6f35f6538268fc10']
ACTION URL : https://rcds01.fnal.gov/pubapi/exists?cid=nova/f3e4886a9c098ca5d14a8b0f6f35f6538268fc10

CREDENTIALS : {'cert': '/tmp/x509up_u8351', 'proxy': '/tmp/x509up_u8351', 'key': '/tmp/x509up_u8351'}

exists = MISSING
ACTION URL : https://rcds02.fnal.gov/pubapi/exists?cid=nova/f3e4886a9c098ca5d14a8b0f6f35f6538268fc10

CREDENTIALS : {'cert': '/tmp/x509up_u8351', 'proxy': '/tmp/x509up_u8351', 'key': '/tmp/x509up_u8351'}

exists = Unrecognized DN

The client goes on to choose 1 of the 2 servers rcds01,02 randomly to upload if 'PRESENT' was not a reponse from one of them.

If it chooses rcds02, this is the result:
output=<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
root@localhost to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
</body></html>

Make the client detect these types of errors and remove the bad server from the list for uploading.

History

#1 Updated by Dennis Box 4 months ago

  • Status changed from New to Resolved

merged into release candidate 8 of 1.3

#2 Updated by Dennis Box 8 days ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF