Project

General

Profile

Bug #24127

Reduce number of queries to collector

Added by Marco Mascheroni 3 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Urgent
Category:
Frontend
Target version:
Start date:
03/04/2020
Due date:
% Done:

0%

Estimated time:
First Occurred:
Occurs In:
Stakeholders:

CMS

Duration:

Description

CMS did a study highlighting how the frontend is responsible of about 100 queries per minute on the collector: https://docs.google.com/presentation/d/13C8IVjcjXacJZPPPNWtag6AQI8K34OoD1mx-Cg7ElwI/edit?usp=sharing

This has an impact on the collector performances. The estimation is a decrease in 10% of the collector duty cycle if the queries are stopped/redirected to the backup collector.

Most of the queries seems they are querying information about schedds (to follow up with a condor_q). We should cache these information since schedd info rarely change.

History

#1 Updated by Marco Mascheroni 3 months ago

I developed a patch to cache the schedd queries. I tested it in the CMS ITB frontend and immediately after I applied (at 16:57) the number of queries dropped from ~2000 to ~300. See the screenshot attached.

#2 Updated by Marco Mascheroni 3 months ago

  • Assignee changed from Marco Mascheroni to Marco Mambelli
  • Status changed from New to Feedback

#3 Updated by Marco Mambelli 3 months ago

  • Assignee changed from Marco Mambelli to Marco Mascheroni

Feedback sent via email

#4 Updated by Marco Mambelli 3 months ago

  • Status changed from Feedback to Resolved

#5 Updated by Marco Mambelli about 2 months ago

  • Status changed from Resolved to Closed


Also available in: Atom PDF