r72313 (PEP 383) created the PyUnicode_FSConverter() function: encode an object to a byte string using the default file system encoding. PyBytes and PyByteArray are leaved unchanged (just increment the reference counter), PyUnicode is encoded to PyBytes (if the encoder produces something else, an error is raised).

In my opinion, a file name is a character string (Windows) or a byte string (POSIX) and a bytearray object is unexpected. Only few function support this type: no function of os.path accept them. In the Python test suite, no function use bytearray for filenames.

It's already complex to support 2 types (bytes and str) for filenames in os.path, I think that a third type is too much and has no real world use case (the module manipuling filenames is os.path and it doesn't support bytearray and nobody noticed that).

Suppport bytearray is complex because we need to acquire a lock (using PyObject_GetBuffer()) and release the lock (o->ob_type->tp_as_buffer->bf_releasebuffer(o, 0)... that's not really intuitive...). posixmodule.c uses functions bytes2str() and realease_bytes() to handle bytearray easily. But these functions are static and other modules have to reimplement them.

I propose the reject bytearray in PyUnicode_FSConverter(), or to simplify the API and fix Python to accept bytearray filename everywhere especially in os.path.

***

Reject bytearray in PyUnicode_FSConverter() is trivial only there is only one test that have to be changed in the whole Python3 test suite: test_empty_bytearray in test_bytes.py. This function shows that bytearray is complex and introduce subtle bugs (the lock/GIL issue). All code using PyUnicode_FSConverter() would become simpler.

Example:

----
/* Release the lock, decref the object. */
static void
release_bytes(PyObject* o)
{
        if (PyByteArray_Check(o))
                o->ob_type->tp_as_buffer->bf_releasebuffer(o, 0);
        Py_DECREF(o);
}

...
realease_byte(path);
----

becomes "Py_DECREF(path);" and release_bytes() can be removed.

And
------------
static char*
bytes2str(PyObject* o, int lock)
{
        if(PyBytes_Check(o))
                return PyBytes_AsString(o);
        else if(PyByteArray_Check(o)) {
                if (lock && PyObject_GetBuffer(o, NULL, 0) < 0)
                        /* On a bytearray, this should not fail. */
                        PyErr_BadInternalCall();
                return PyByteArray_AsString(o);
        } else {
                /* The FS converter should have verified that this
                   is either bytes or bytearray. */
                Py_FatalError("bad object passed to bytes2str");
                /* not reached. */
                return "";
        }
}

...
path = bytes2str(opath);
------------

becomes "path = PyBytes_AS_STRING(opath);" (or maybe "path = PyBytes_AsString(opath);" if you don't trust PyUnicode_FSConverter)  and bytes2str() can be removed.

***

Simplify the API means that bytes2str() and release_bytes() should become protected functions (not static, but prefixed by "_Py_") or maybe public ("Py_" prefix).

But the most complex part is to modify os.path to support bytearray, and this task would not be funny :-)

Patch removing bytearray filename support: it mostly removes code.

Or perhaps the bytearray can be converted to a bytes object. This is not optimal performance-wise but is unlikely to make a difference in real-world code (if you are passing a filename to an external API, chances are some IO will occur which will dwarf the cost of creating a separate bytes object).

But I agree that supporting bytearrays in filename-taking functions, while "nice" from a consistency point of view, isn't really useful in practice. So I would be ok to remove that support if it simplifies (or avoids complexifying) the logic for those functions.

I'm in favor of removing that support, too, but please check with python-dev whether anybody thinks a proper deprecation cycle (deprecated in 3.2, removed in 3.3) is needed.

Antoine Pitrou wrote:
> 
> Antoine Pitrou <pitrou@free.fr> added the comment:
> 
> Or perhaps the bytearray can be converted to a bytes object. This is not optimal performance-wise but is unlikely to make a difference in real-world code (if you are passing a filename to an external API, chances are some IO will occur which will dwarf the cost of creating a separate bytes object).
> 
> But I agree that supporting bytearrays in filename-taking functions, while "nice" from a consistency point of view, isn't really useful in practice. So I would be ok to remove that support if it simplifies (or avoids complexifying) the logic for those functions.

+1

bytearrays are basically the remains of the attempt to use mutable
byte string objects in Python 3.x. They may gain some usefulness
in the future, but I doubt that this will be in the area of filenames.

MvL, MaL and Antoine Pitrou agreed to drop support of bytearray filenames in Python. I choosed to remove it directly in Python 3.2 because it wasn't really used, open() and os.path never supported bytearray filenames, and I will simplify the other issues related to surrogates in filenames.

Fixed: r80361 (py3k), don't backport to 3.1.