Repository view not updating
I pushed some changes to my repository (https://cdcvs.fnal.gov/redmine/projects/protocol-compiler/repository) and, 3 days later, I still don't see them in Redmine. The file that was changed was
genpython.cpp. Whatever job that runs periodically to update the repo page isn't updating my page.
#1 Updated by Marc Mengel about 4 years ago
Um... when I look at the repository view, the latest change is:
6eb57473 ￼ 08/19/2016 09:57 AM Richard Neswold The generated C++ code now uses the system's preferred character type. This means we can go back to using std::string, std::istringstream, etc.
and genpython.cpp is listed as being revised in the last 3 days. Perhaps your browser is caching the page? Try shift-reload?
#2 Updated by Richard Neswold about 4 years ago
That's not the latest change. When I do a
git log on my system, the latest entry is:
commit cdd21af61cc6d1b13bc5e3f12cd91bc385f987de Author: Rich Neswold <email@example.com> Date: Fri Aug 19 15:59:54 2016 -0500 Two bugs in the Python generator: * The .next() method of iterators doesn't take an argument. * struct.unpack() always returns a tuple, so we need to extract the first element.
git pull and
git push both state everything is up to date.
genpython.cpp is, indeed, the file that changed. As you can see in the repo view, its revision and comment columns are blank.
#3 Updated by Richard Neswold about 4 years ago
- Status changed from New to Resolved
Marc Mengel wrote:
Perhaps your browser is caching the page? Try shift-reload?
I tried reloading dozens of times on Friday. I tried again this morning without success (which is what prompted me to report the issue.) However, after reporting it, I'm now seeing the entry. If it happens again, I'll re-open it.
#4 Updated by Richard Neswold over 2 years ago
It's happening again. Interestingly, it's happening in the same project!
I pushed commit
e4460f2e last friday and it still doesn't show up in the revision history three days later. If I click on the file I changed in the last commit, I see my changes, so the repository is correct on
git says everything is up-to-date when I try to push.)
#6 Updated by Marc Mengel over 2 years ago
Hm... Just a few notes before any logs roll off...
So looking at the ~p-protocol-compiler/cvshlog (timestamp changes of which are supposed to trigger the calls to fetch_changesets vi /usr/local/admin/do_repositories) we see updates:
[root@ccdcvsvm p-protocol-compiler]# tail cvshlog ... Fri Jun 8 12:57:18 2018 (neswold@(null)) : cvsh -c git-upload-pack '/cvs/projects/protocol-compiler' [sk] Fri Jun 8 12:59:38 2018 (neswold@(null)) : cvsh -c git-upload-pack '/cvs/projects/protocol-compiler' [sk] Fri Jun 8 13:00:39 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk] Mon Jun 11 09:26:38 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk] Mon Jun 11 09:54:58 2018 (neswold@(null)) : cvsh -c git-receive-pack '/cvs/projects/protocol-compiler' [sk]
But looking at the ssl_access_logs, that go back a full week, we only have one call generated by the script to update that repository in Redmine:
[root@ccdcvsvm logs]# zgrep fetch_changesets ssl_access_log* | grep protocol ssl_access_log:ccdcvsvm.fnal.gov - - [11/Jun/2018:09:40:39 -0500] "GET /redmine/sys/fetch_changesets?id=protocol-compiler&key=[redacted] HTTP/1.0" 200 -
Now I note that the updates that got missed were just before the top of the hour; and our cron.d entry runs every 20 minutes:
0 1 * * * root /usr/local/admin/fix_repo_group_permissions > /tmp/fix_repo_group_permissions.out 2>&1
and our script looks for files changed in the last 20 minutes.
for d in p-* do if find $d/cvshlog -mmin -20 -print 2>/dev/null | grep -q cvshlog then proj=`echo $d | sed -e 's/^p-//'` echo "Fecthing changesets for $proj" if [ "$d" != "p-novadaq" ] then wget -O /dev/null "https://cdcvs.fnal.gov/redmine/sys/fetch_changesets?id =$proj&key=[redacted]" fi fi done
So if we're right at the 20 minute boundary, we could miss a change that happened 21 minutes ago. I really need to make the check in the script (the 20 minutes in the find) wider than the repeat time on running the script so we don't miss stuff. This could have the effect of double-calling the update script sometimes, but that shouldn't hurt much...
Ooh.. also, we should make the list of repositories that need doing, and then do them, 'cause if calling the wget is slow on one of them that staggers the start...
So I'll look into fixing that a bit later this morning hopefully...
In the meantime, if udpates aren't showing up, a good workaround is to just ssh into the p-project account (i.e ssh firstname.lastname@example.org echo hi) a couple of times a few minutes apart to get the script's attention
#8 Updated by Marc Mengel over 2 years ago
Okay, so I've changed the script from:
# # only update projects with ssh/cvshlog activity # for d in p-* do if find $d/cvshlog -mmin -20 -print 2>/dev/null | grep -q cvshlog then proj=`echo $d | sed -e 's/^p-//'` echo "Fecthing changesets for $proj ...
# # only update projects with ssh/cvshlog activity # find p-*/cvshlog -mmin -23 -print | while read cl do d=`echo $cl | sed -e 's;/cvshlog;;'` proj=`echo $d | sed -e 's/^p-//'` echo "Fecthing changesets for $proj" ...
which is to say, we now run a single find up front looking for changed cvshlog files, and it pipes that list into a while loop that does the changeset fetches.
And I widened it out to 23 minutes to make sure we cover stuff if we're off by a minute or two in startup.
I'm going to try doing some strategically timed pushes around 13:00 and see if I get them all the next few days, and if so, I'll close this out.