How to reproduce:
1. Start IDLE -> "Python Shell" window opens.
2. Open Options|"Configure Idle...".
3. In "idle" dialog select "Keys" tab.
4. Click "Set as New Custom Key Set" button.
5  In "New Custom Key Set" dialog, type a name, click "Ok".
4. Click "Apply". Then notice that in the folder

   C:\Documents and Settings\<user>\.idlerc,

   two new files appear, config-keys.cfg and config-main.cfg. The
contents of the latter is

[Keys]
default = 0
name = <Key Set Name>

SO FAR ALL IS OK.

5. Now go back to "idle" dialog, click "Delete Custom Key Set", then
"Apply". Notice that config-keys.cfg file disappeared, but
config-main.cfg neither disappeared nor changed. My guess, THIS IS THE BUG.

6. Now click "Ok" - no reaction. Click "Cancel" - crash: the Python
windows disappear. Try to restart IDLE - it WOULD NOT START until
config-main.cfg is manually deleted.

Interesting, this is reproducible in any version under any OS but it is
more noticeable under Windows. In the step 6 you mentioned, IDLE is busy
printing warnings that you will never see if you don't run IDLE from a
terminal. Also, when you tried restarting IDLE I believe it were
actually starting up but wasting a lot of time with this non-existing
key binding.

The problem is that when you ask to delete a custom key set, given the
steps pointed out, IDLE will first remove the custom key bindings and
then will try to remove the bindings associated with the just removed
key bindings which will generate a lot of warnings since they no longer
exist.
A simple solution is to invert some steps when "Apply" is pressed so the
bindings are removed while they still exist in the config-keys.cfg file.
But if you have a config-keys.cfg like you described below and start
IDLE, it will still take a long time to load (at least it does in a
virtual machine here).

Note the config-main.cfg contains all custom configurations and appears if you make any one of them. Mine currently says

[EditorWindow]
font = lucida sans unicode

[General]
autosave = 1

So deleting it is a hack workaround until the bug is fixed.

#13836 is another report of key-binding deletion not working correctly, although IDLE did eventually open and allow re-deletion.

I am unable to reproduce this bug with the latest development versions in the 2.7 and 3.3 branches. 

Can someone verify this as well?