Idea #2181
Feature #2180: Use Condor Job Log instead of Schedd log or classads for job data on the pilot
Get Pilot logs back to the factory from an EC2 VM
0%
Description
This would depend a lot on Issue #2180 to reduce the size of the pilot logs that are currently returned from the pilots.
EC2 API defines ec2-get-console-output which allows you to get the last 64kb of the console output of a terminated VM. This data is saved for up to an hour after termination. I have put in a request for the Condor team to implement this call in the ec2_gahp so that we can retrieve logs using condor on the factory.
The log retrieval process would look like:
1) launch vm - save contextualization logs
2) launch pilot - save logs
3) shutdown
3a) aggressively compress logs
3b) base64 encode the compressed logs
3c) write base64 encoded logs to the console
4) the ec2_gahp will retrieve the console output and write it out as a job log.
Now the factory has all the logs from the VM. We can debug as necessary.
History
#1 Updated by Anthony Tiradani over 9 years ago
- Parent task set to #2180
#2 Updated by Parag Mhashilkar over 9 years ago
- Target version changed from v3_0 to v3_1
#3 Updated by Burt Holzman almost 9 years ago
- Assignee set to Anthony Tiradani
- Priority changed from Normal to Low
#4 Updated by Burt Holzman about 8 years ago
- Priority changed from Low to Normal
Upgrading this from "Low" to "Normal" -- this means that right now we don't really
get stdout or stderr back from a pilot job and then don't have proper pilot accounting data.
#5 Updated by Burt Holzman almost 8 years ago
- Target version changed from v3_1 to v3_x
#6 Updated by Burt Holzman over 7 years ago
- Target version changed from v3_x to v3_2_x
#7 Updated by Marco Mambelli almost 3 years ago
- Target version changed from v3_2_x to v3_4_x
#8 Updated by Marco Mambelli over 2 years ago
- Target version changed from v3_4_x to v3_5_x
#9 Updated by Marco Mambelli over 1 year ago
- Target version changed from v3_5_x to v3_7_x
#10 Updated by Marco Mambelli 2 months ago
- Assignee deleted (
Anthony Tiradani)