No, it shouldn't.

handle_close() is more like a callback method called by asyncore itself when the connection is about to be closed "cleanly", and it's not supposed to be called directly by the user (see for example how recv() and send() methods are implemented and issue 1740572).
Also it is usually overridden and filled with application code you hardly want to be executed when you just want to close all connections at once.

When you use close_all() you just want to abruptly close all existing connections, without even caring about pending data waiting to be read or written, hence close() must be called.

Closing out as won't fix as long as some actual reason to change the current behavior is provided.