Bug #6370

Submitting glideins using EC2 is hitting 8K limit for URI

Added by Parag Mhashilkar over 6 years ago. Updated about 4 years ago.

Parag Mhashilkar
Target version:
Start date:
Due date:
% Done:


Estimated time:
First Occurred:
Occurs In:


Its not clear if we can do something in glideinwms or should we check with condor. Condor does EC2 submit using GET. HTTP has a limit or 8K. Until now using a grid proxy

This came up while testing Glideinwms submitting to Fermicloud. Neha created a proxy that by default was a 1024 bit proxy instead of 512 proxy. URI generated by condor to launch an ec2 instance was 8K+. Since we add proxy to the ec2 metadata, this is partially in our court too.


#1 Updated by Anthony Tiradani over 6 years ago

Use HTCondor 8.1.5+ for the Factory. This is a known problem and was addressed in 8.1.5.

#2 Updated by Dave Dykstra over 6 years ago

To be precise, it isn't http that has an 8K byte URL limit. Different servers, http proxies, and clients have different limits. 8K is certainly pushing things, however.

#3 Updated by Anthony Tiradani over 6 years ago

The Condor developers changed the HTTP method from GET to POST in HTCondor 8.1.5. POST has no explicit size limit. HTCondor 8.1.5 has been tested against OpenStack and is currently being used by CMS for the AI and HLT clouds for production work.

In the case of OpenStack, part of the scaling strategy is to front the EC2 protocol with Apache. Apache has a hard coded limit of 8K for GET. They do provide a knob to adjust that value, but for any value > 8K, you have to recompile Apache. Apache imposes no limit on POST.

What is being used to handle EC2 requests in OpenNebula?

Additional notes:
Jetty has a 4K limit
Python's "built in" server doesn't limit GET (OK, there is no built in server, but creating a very simple HTTP server in python is trivial)

#4 Updated by Anthony Tiradani over 6 years ago

Further explanation can be found with some references here:

#5 Updated by Parag Mhashilkar over 6 years ago

Thanks to Dave and Tony for additional info. I will ask Neha to try it out with different condor

#6 Updated by Parag Mhashilkar about 4 years ago

  • Status changed from New to Closed

This is not an issue anymore

Also available in: Atom PDF