This is working in pretty much the only way it can work.  Python is correctly retrieving the string from the environment (it includes the quotation marks).  It is correctly passing that string to os.path: os.path takes the exact string that represents the filename.  In windows
it would be equally valid to do this:

    set dir3="just "" a"" test"
    %dir3%\hello.bat

Your "simple fix" would not address that case (and would break existing programs that rely on the current behavior).  There is no way to handle all the possible variations here.  The only thing that can be done is what python does do: return exactly the string stored in the environment variable, and expect exactly the filename as input to functions that expect filenames.

In many contexts in Windows it works perfectly fine to have the filename without quotes around it stored in the environment variable and to use that:

  dir %dir1%

works fine, for example.  The Windows rules for quoting are non-obvious and not completely consistent, I'm afraid, and there's nothing python can do to paper over that fact.