Ok, I had three such keys. I run RegDelNull.exe and managed to remove two of them. The third one is still there.
I'm reasonably sure that this machine is clean, anyway:

1. the Python installer should not fail silently when an exception is thrown during pip installation. There is also a stackoverflow question about "where is pip in python 3.4, I can't find it!" from someone likely having the same problem.

2. ensurepip should not crash on unrelated registry entries it doesn't like, it should just ignore them.

Thanks for the pointer to null keys and the script to find them.