Project

General

Profile

Feature #6585

Get google searches working...

Added by Marc Mengel over 5 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
Target version:
-
Start date:
07/01/2014
Due date:
% Done:

0%

Estimated time:
Duration:

Description

So it turns out Redmine makes a dynamically generated robots.txt (see https://cdcvs.fnal.gov/redmine/robots.txt)
to prevent bots from scanning parts of Redmine that would bog things down, etc. but this doesn't work
in our currenc configuration, because we map it to a subtree (/redmine) and

  1. robots.txt is only looked for at the top level
  2. it doesn't have the /redmine in the paths

So we need to put in a script to convert the Redmine one to a correct one and put it
at http{s,}://cdcvs.fnal.gov/robots.txt so it will take effect.

make_robots_txt (260 Bytes) make_robots_txt First draft of script to make robots.txt Marc Mengel, 07/01/2014 11:56 AM

History

#1 Updated by Marc Mengel over 5 years ago

Put a draft together of a script to do this, need to put it in crontab and
give it teeth (it has to rename robots.txt.new robots.txt at the end).

We'll need to do a change request for this.



Also available in: Atom PDF