Ok, it's simple really. When seeding from something else than an integer, seed() takes the hash of the object (instead of considering all its bytes, which might be considered a weakness since you lose entropy -- also, Python hash() is not supposed to be cryptographically strong). The hash is different in 32-bit and 64-bit mode (although the lower 32 bits are the same, at least for a bytes object), and since all the bits are taken into account the initial state is different.

So the easy workaround for the OP is to seed with an integer rather a bytes object.