Project

General

Profile

Bug #13608

Repository view not updating

Added by Richard Neswold about 3 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
08/22/2016
Due date:
% Done:

0%

Estimated time:
Duration:

Description

I pushed some changes to my repository (https://cdcvs.fnal.gov/redmine/projects/protocol-compiler/repository) and, 3 days later, I still don't see them in Redmine. The file that was changed was genpython.cpp. Whatever job that runs periodically to update the repo page isn't updating my page.

History

#1 Updated by Marc Mengel about 3 years ago

Richard,

Um... when I look at the repository view, the latest change is:

6eb57473            08/19/2016 09:57 AM    Richard Neswold    
The generated C++ code now uses the system's preferred character type. This means we can go back to using std::string, std::istringstream, etc.

and genpython.cpp is listed as being revised in the last 3 days. Perhaps your browser is caching the page? Try shift-reload?

#2 Updated by Richard Neswold about 3 years ago

That's not the latest change. When I do a git log on my system, the latest entry is:

commit cdd21af61cc6d1b13bc5e3f12cd91bc385f987de
Author: Rich Neswold <neswold@fnal.gov>
Date:   Fri Aug 19 15:59:54 2016 -0500

    Two bugs in the Python generator:

    * The .next() method of iterators doesn't take an argument.
    * struct.unpack() always returns a tuple, so we need to extract the first element.

git pull and git push both state everything is up to date. genpython.cpp is, indeed, the file that changed. As you can see in the repo view, its revision and comment columns are blank.

#3 Updated by Richard Neswold about 3 years ago

  • Status changed from New to Resolved

Marc Mengel wrote:

Perhaps your browser is caching the page? Try shift-reload?

I tried reloading dozens of times on Friday. I tried again this morning without success (which is what prompted me to report the issue.) However, after reporting it, I'm now seeing the entry. If it happens again, I'll re-open it.

#4 Updated by Richard Neswold over 1 year ago

It's happening again. Interestingly, it's happening in the same project!

I pushed commit e4460f2e last friday and it still doesn't show up in the revision history three days later. If I click on the file I changed in the last commit, I see my changes, so the repository is correct on cdcvs.fnal.gov (and git says everything is up-to-date when I try to push.)

#5 Updated by Richard Neswold over 1 year ago

I pushed a new commit I made this morning and now I see Friday's commit in the history. We'll have to wait to see if today's commit appears.

#6 Updated by Marc Mengel over 1 year ago

Hm... Just a few notes before any logs roll off...

So looking at the ~p-protocol-compiler/cvshlog (timestamp changes of which are supposed to trigger the calls to fetch_changesets vi /usr/local/admin/do_repositories) we see updates:

[root@ccdcvsvm p-protocol-compiler]# tail cvshlog
...
Fri Jun  8 12:57:18 2018 (neswold@(null)) : cvsh -c git-upload-pack '/cvs/projects/protocol-compiler' [sk]
Fri Jun  8 12:59:38 2018 (neswold@(null)) : cvsh -c git-upload-pack '/cvs/projects/protocol-compiler' [sk]
Fri Jun  8 13:00:39 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk]
Mon Jun 11 09:26:38 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk]
Mon Jun 11 09:54:58 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk]

But looking at the ssl_access_logs, that go back a full week, we only have one call generated by the script to update that repository in Redmine:

[root@ccdcvsvm logs]# zgrep fetch_changesets ssl_access_log* | grep protocol
ssl_access_log:ccdcvsvm.fnal.gov - - [11/Jun/2018:09:40:39 -0500] "GET /redmine/sys/fetch_changesets?id=protocol-compiler&key=[redacted] HTTP/1.0" 200 -

Now I note that the updates that got missed were just before the top of the hour; and our cron.d entry runs every 20 minutes:

0    1 * * * root     /usr/local/admin/fix_repo_group_permissions > /tmp/fix_repo_group_permissions.out 2>&1

and our script looks for files changed in the last 20 minutes.

for d in p-*
do
   if find $d/cvshlog -mmin -20 -print 2>/dev/null | grep -q cvshlog 
   then
       proj=`echo $d | sed -e 's/^p-//'`
       echo "Fecthing changesets for $proj" 
       if [ "$d" != "p-novadaq" ]
       then
       wget -O /dev/null "https://cdcvs.fnal.gov/redmine/sys/fetch_changesets?id
=$proj&key=[redacted]" 
       fi
   fi
done

So if we're right at the 20 minute boundary, we could miss a change that happened 21 minutes ago. I really need to make the check in the script (the 20 minutes in the find) wider than the repeat time on running the script so we don't miss stuff. This could have the effect of double-calling the update script sometimes, but that shouldn't hurt much...

Ooh.. also, we should make the list of repositories that need doing, and then do them, 'cause if calling the wget is slow on one of them that staggers the start...

So I'll look into fixing that a bit later this morning hopefully...

In the meantime, if udpates aren't showing up, a good workaround is to just ssh into the p-project account (i.e ssh echo hi) a couple of times a few minutes apart to get the script's attention

#7 Updated by Richard Neswold over 1 year ago

Thanks for diagnosing this. It does appear to be a race condition between pushing to the repo and the update scripts running. The push I made today shows up in the history so it occurred during an idle script time.

#8 Updated by Marc Mengel over 1 year ago

Okay, so I've changed the script from:

#
# only update projects with ssh/cvshlog activity
#
for d in p-*
do
   if find $d/cvshlog -mmin -20 -print 2>/dev/null | grep -q cvshlog 
   then
       proj=`echo $d | sed -e 's/^p-//'`
       echo "Fecthing changesets for $proj
       ...

to

#
# only update projects with ssh/cvshlog activity
#
find p-*/cvshlog -mmin -23 -print |
    while read cl
    do
       d=`echo $cl | sed -e 's;/cvshlog;;'`
       proj=`echo $d | sed -e 's/^p-//'`
       echo "Fecthing changesets for $proj" 
       ...

which is to say, we now run a single find up front looking for changed cvshlog files, and it pipes that list into a while loop that does the changeset fetches.
And I widened it out to 23 minutes to make sure we cover stuff if we're off by a minute or two in startup.

I'm going to try doing some strategically timed pushes around 13:00 and see if I get them all the next few days, and if so, I'll close this out.



Also available in: Atom PDF