The attached patch removes test_bytes.FixedStringTest and its subclasses, as they report success when they don't actually test anything at all.

ByteArrayAsStringTest and BytesAsStringTest inherit tests from test.string_tests.BaseTest.

So they do, I failed to notice that.  New patch just removes the empty test methods.

This LGTM, but the purpose of suppressing these tests is not absolutely clear to me. I suppose they were suppressed in times when bytes and batearray had no the lower() and upper() methods and the __contains__() methods accepted only integers. It will be good if developers which touched this code in past will made a review.

I said this elsewhere but I'll repeat it here.
I think the whole string_tests could/should be reorganized (and possibly documented).  There are lot of classes and mixins on several files, and more than once I found tests that were accidentally not run or had other problems.  This got worse after Python 3, where bytes and strings share even less code.  Reorganizing this should make things easier to follow and will probably solve issues like this.  This of course is a bigger project than simply fixing these methods, so it should probably be addressed in a separate issue.

test_contains() does not appear to be overriding anything, so there is no problem removing that.

Unmasking the other three methods will allow six new test methods to run (via ByteArrayAsStringTest and BytesAsStringTest). There are already three equivalent methods defined in buffer_tests.py run via BytearrayPEP3137Test. I suggest to remove those three from buffer_tests. The net result will be the same tests run for bytes(), and three new tests for bytearray(). The buffer_tests file is not used anywhere else.

Currently buffer_tests.py is used only for bytearray, and it looks that it duplicates string_tests.py. May be remove all duplicated tests and move the rest of tests (if any) just into test_bytes.py?

New changeset 22c2dd04a3d3 by Martin Panter in branch '3.5':
Issue #19587: Remove masked and redundant tests in test_bytes
https://hg.python.org/cpython/rev/22c2dd04a3d3

New changeset 8545a082fcaa by Martin Panter in branch 'default':
Issue #19587: Merge test_bytes cleanup from 3.5
https://hg.python.org/cpython/rev/8545a082fcaa

Thanks Serhiy for reviewing. You are right that buffer_tests is only run on bytearray; I must have gotten mixed up. I fixed the comment in the version I committed.

Yes I think I agree with eliminating buffer_tests.py, which is only run for bytearray. I opened Issue 26257 for that.