Perhaps tempdir_name could get the slash prefixed in setUp() rather than in every test. There is one case that creates index.html which could probably be changed to use self.tempdir, and then the rest could be renamed to say self.base_url.

Rejecting requests that do not start with a slash could be a compatibility problem. Do you think potential security benefits (i.e. having unexpected variations on the format of the URL) outweigh that?

Either way, it would be good to retain a test without a slash.