> The release process needs to have a step that validates that the CHM
> file works from IDLE.

I personally can't perform such testing steps, perhaps unless somebody
maintains a list of such steps for me.

Instead, I think such problems must be detected during beta testing.
The fact that it hasn't been detected so far, even though it existed
since 2.6.0, indicates that the problem can't be that serious.

> FWIW, I think the version naming convention is unnecessary and makes the
> IDLE linkage too fragile.  It would be better to just always call the
> file Python.chm.  By way of comparison, the regular html docs do not get
> custom names

That's not true: they are called e.g.

python-2.6.2-docs-html.tar.bz2
(see http://docs.python.org/download.html)

I believe that the rationale for calling the CHM file that way is the
same as for putting a version number into any other output.

FWIW, the renaming had caused some troubles in the past, but most of
it is over. Renaming it back will cause more troubles for sure.