Margaret Votava - FNAL Project Sponsor/Scientific Computing Services Associate Head
Parag Mhashilkar - Project Lead
Marco Mambelli - Technical Lead
Dennis Box - Project Member
Lorena Lobato - Project Member
Marco Mascheroni - Project Member/OSG Factory Operations
Stu Fuess - HEPCloud Sponsor Proxy
Burt Holzman - HEPCloud Sponsor Proxy
Tony Tiradani - HEPCloud Technical Lead
Antonio Perez-Calero Yzquierdo - CMS
Eric Vaandering - CMS
James Letts - CMS
Dave Mason - USCMS T1 facility manager
Steve Timm - HEPCloud Technical Advisor
Tanya Levshina - FIFE
Ken Herner - FIFE
Joe Boyd - FIFE
Mike Kirby - FIFE
Jeff Dost - OSG Factory Operations
Brian Lin - OSG



Project Management



  • Steve Timm: What is the ETA on 3.4.2 bugs reported? Current frontend ignores the entries put in downtime.
    • Marco: Bugs will be addressed in v3.5 to be released in couple of weeks.
  • Steve Timm: How is transition to v3.5 going to happen
    • Factory is compatible with older frontends and frontends are not required to upgrade right away.
  • Margaret: What are we reviewing in planned code review?
    • Marco Mambelli: We did a code scrubbing recently to prepare for migration to python 3. We remove unused code and need to do another round of review.
  • Steve: What is the timeframe for work done in collaboration with the HTCondor
    • Marco Mambelli: Currently working on following tasks -
      • We are working on optimizing negotiations done by frontend
      • Common singularity solution provided by HTCondor
  • Tony: Almost all items on the roadmap depend on interactions with HTCondor team. Want to understand on how that is moving forward as we are not seeing much progress. Also want to understand direction taken.
    • Marco: We are working with the HTCondor team. Sometimes requirements out pace the solution provided. example, CMS, OSG need support for GPU, bind points, etc, not in condor yet but planned to upcoming versions. We will move to them as the solution is provided by HTCondor team.
  • Jeff: In past, moving from 3.2 to 3.3, 3.3 was a development series, and 3.2 bug fix. Is 3.5 going to replace 3.4 as production series?
    • v3.5 is dependent on HTCondor 8.7 and will be 8.8 compatible, people can use v3.4 with old condor
    • Brian Lin: Need to work with the rest of HTCondor team to get HTCondor 8.8 in OSG. It is expected to be released next month
      • Jeff: Want to make sure we are ready to move.
  • James: CMS is currently testing v3.4.2 in ITB. Testing is expected to last about for week.
  • Brian suggested we put GlideinWMS v3.5 in osg upcoming to give them more time for the frontend admins to upgrade to v3.4.2 to avoid draining their pools before moving to v3.5.
  • Antonio: Are Acquisition Engine and Decision Engine Fermilab products?
    • AE is factory like and was used and was supposed to be developed as part of the proposal by the HTCondor team
    • DE is frontend like and is a Fermilab product
    • Monitoring: We are moving monitoring to CERN using elastic search and tools and system. we are querying condor daemons and injecting in grafana. How do we do the injection of monitoring info after glideinwms makes changes to monitoring?
      • For 3.4 3.5 there wont be any changes. After that monitoring will be a plugin that can keep publishing those files in parallel and publish json to influx db
    • Tanya: It would be good to do it in coordination. We have been working with graphite and grafana dashboard and elastic search for quite sometime at Fermilab
  • Brian: Dennis talked about Glideinwms unit testing in containers. Does it run all the tests?
    • Dennis: Yes. Reporting is lacking and needs manual work. Work is being done to automate it.
  • Antonio: Scalability tests of frontend done by CMS discovered few issues? Do you need more input on this point?
    • Marco Mascheroni is working on this and will try again soon and get back to CMS as he needs more info.


  • Marco: Close old pull requests in the Github.
  • Marco: Setup a meeting with CMS (Antonio, James, Marco Mascheroni) and landscaping (Tanya, Kevin, Joe) Krista, after Thanks Giving