A library implementing this should definitely go on PyPI first to shake out design issues. Then we'd need a PEP.

As someone who has a simplistic version of this code around, and who's done a bit of string formatting, I can assure you that there are a lot of issues to be thought through.

- Should it be 0.5M, or 500K? Is there a cutoff to switch over? Is it configurable?
- What if I prefer 1000K to 1M? Or even 1,000.0K or 1.000,0K (localized)?
- How many decimals to display? Can you suppress trailing zeros? Should it be 1.0M, or 1M?
- Space between the number and the units?
- As you mention, MiB vs. MB. And there are schemes where MB means 1000^2 and others where MB means 2^20. And as Serhiy says, schemes where 1 MB = 1024000 bytes. How to handle all of these.
- Localization. Should this be specified as a locale, use the current locale, just ignored (see PEP 378 for an example), or have the delimiters passed in?

I'm sure there are other decisions to be made.

If you're serious about this (and I hope you are!), then I think finding current best practices both within and outside of the Python universe should be researched.

I do like the idea of the "h" format specifier, which would be an argument to move it in to the stdlib. That said, we never did come to agreement on something much simpler: engineering notation for floats (see issue 8060), because no one put the time into writing up a concrete proposal.