Message331963
> (...) refers users, for file structure, to http://www.robotstxt.org/orig.html. This says nothing about the effect of an empty file, so I don't see this as a bug.
That is incorrect. From that url you can find:
> The presence of an empty "/robots.txt" file has no explicit associated semantics, it will be treated as if it was not present, i.e. all robots will consider themselves welcome.
So this is definitely a bug. |
|
Date |
User |
Action |
Args |
2018-12-17 10:02:06 | larsfuse | set | recipients:
+ larsfuse, terry.reedy |
2018-12-17 10:02:06 | larsfuse | set | messageid: <1545040926.74.0.788709270274.issue35457@psf.upfronthosting.co.za> |
2018-12-17 10:02:06 | larsfuse | link | issue35457 messages |
2018-12-17 10:02:06 | larsfuse | create | |
|