Project

General

Profile

Bug #17028

Proxy Error

Added by Anna Mazzacane over 3 years ago. Updated over 3 years ago.

Status:
Closed
Priority:
Normal
Target version:
Start date:
06/22/2017
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
First Occurred:
Scope:
Internal
Experiment:
-
Stakeholders:
Duration:

Description

Proxy Error while loading POMS pages is appearing again.
NOvA is experiencing this problem and opened a ticket (RITM0575905).
Robert thinks it could be a configuration problem. Vladimir will look at.
Marc is working on loading the campaign page of a specific experiment first, and then populate the info stats as they become available (see bug#16989).


Subtasks

Bug #16989: Rework show_campaigns Closed

History

#1 Updated by Anna Mazzacane over 3 years ago

  • Target version set to v2_1_3

#2 Updated by Marc Mengel over 3 years ago

Some digging around between Robert Illinworth and myself; Robert strace-ed some of our poms processes, and they were spending an inordinate amount of time looking at session lock files; so one idea is to use the 'expclicit' locking mode for the session as per:
http://docs.cherrypy.org/en/latest/_modules/cherrypy/lib/sessions.html

So I'm going to look at our session code and see how hard that would be; it should be sufficient to lock it for the two calls that fill in the session in service.py.

#3 Updated by Marc Mengel over 3 years ago

From: Robert Illingworth <>
Subject Re: POMS "performance"

I think we’ve discovered another issue - a lot of the worker threads end up waiting for the http session lock. If you want to do any ajax stuff in parallel, especially with long running operations, you can’t use the default implicit session lock mode, otherwise it all gets serialized.

Robert

#4 Updated by Marc Mengel over 3 years ago

I put some session changes in 3419fb28 -- (there's also another change rolled in there for fetchlog using the web service...) Basically we configure it to 'explicit' session locks, and add couple of explicit acquire/release calls for said locking, and we don't wait for whole requests on session locks anymore.
The only wart is that for the FileSession sessions, the code assumes the session should be locked when you finish the call, or it throws ugly errors; so there's a bit:

    # Here is how to add aditional hooks. Left as example    
    def _setup(self):    
        cherrypy.Tool._setup(self)    
        cherrypy.request.hooks.attach('before_finalize',    
                                      self.finalize_session,
                                      priority=10)
    def finalize_session(self):    
        # file session save complains if session isn't locked?!?    
        # so re-lock it on the way out(?)    
        cherrypy.session.acquire_lock()        

which exists just to re-lock it just before the session end code happens. I'm not sure why the file session code makes it so difficult to do explicit locking, but there you go...

#5 Updated by Marc Mengel over 3 years ago

Also adding some bits to https://github.com/cherrypy/cherrypy/issues/1345 ... which talks about this.

#6 Updated by Anna Mazzacane over 3 years ago

  • Status changed from Assigned to Resolved

#7 Updated by Anna Mazzacane over 3 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF