Project

General

Profile

Feature #24616

Improve the handling of curl and wget

Added by Marco Mambelli 28 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
07/09/2020
Due date:
% Done:

0%

Estimated time:
Stakeholders:
Duration:

Description

File download is essential and it is tricky.

curl and wget have different versions behaving differently and many things can go wrong.
The functions in glidein_startup.sh need to be done more robust and modular, and maybe exported in glidein_lib.sh to be available to other scripts

Issues w/ current implementation:
  • New problems come up periodically
  • no debug messages (only warnings)
  • some confusing messages, e.g.`wget_version=$(wget --version | head -1 )` may print "wget: command not found"
  • wget/curl bits in fetch_file_base
Suggesting:
  • perform_download: generic function invoking both, to be added, common parameters, error checking at the end, maybe consistent error messages
  • perform_curl: all curl specific stuff should be here
  • perform_wget: all wget specific stuff should be there
Example problem at RAL:


Also available in: Atom PDF