classification
Title: Expose linux extended filesystem attributes
Type: enhancement Stage: resolved
Components: Extension Modules Versions: Python 3.3
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: Arfrever, benjamin.peterson, neologix, pitrou, python-dev, skrah
Priority: normal Keywords: patch

Created on 2011-08-09 20:24 by benjamin.peterson, last changed 2011-09-14 15:46 by python-dev. This issue is now closed.

Files
File name Uploaded Description Edit
xattrs.patch benjamin.peterson, 2011-08-09 20:24 review
xattrs.patch benjamin.peterson, 2011-08-18 17:08 review
xattrs.patch benjamin.peterson, 2011-08-28 12:59 review
xattrs.patch benjamin.peterson, 2011-08-28 16:49 review
Messages (12)
msg141842 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-08-09 20:24
These allow extra metadata to be attached to files.
msg142293 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-17 22:04
I think the functions should:
- accept both str and bytes for keys and values; when it's an str, encoding it using the FS encoding
- return str for attribute names, using the FS encoding
- return bytes for attribute values, although returning str might be reasonable as well

I also get the following test failures:

======================================================================
FAIL: test_fds (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1579, in test_fds
    self._check_xattrs(getxattr, setxattr, removexattr, listxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised

======================================================================
FAIL: test_lpath (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1564, in test_lpath
    os.llistxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised

======================================================================
FAIL: test_simple (test.test_os.ExtendedAttributeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1560, in test_simple
    os.listxattr)
  File "/home/antoine/cpython/default/Lib/test/test_os.py", line 1542, in _check_xattrs
    setxattr(fn, b"user.test", b"bye", os.XATTR_CREATE)
AssertionError: OSError not raised
msg142372 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-08-18 17:08
Here is a new patch, implementing Antoine's suggestions.
msg143095 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-08-28 12:59
And here is the next version, taking into account neologix's review.
msg143104 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-08-28 16:34
Is it normal that listxattr() succeeds but getxattr() fails with ENOTSUPP?

>>> os.listxattr("/")
[]
>>> os.getxattr("/", "foo")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 95] Operation not supported

This is on 2.6.38.8.
msg143105 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-08-28 16:40
2011/8/28 Antoine Pitrou <report@bugs.python.org>:
>
> Antoine Pitrou <pitrou@free.fr> added the comment:
>
> Is it normal that listxattr() succeeds but getxattr() fails with ENOTSUPP?
>
>>>> os.listxattr("/")
> []
>>>> os.getxattr("/", "foo")
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> OSError: [Errno 95] Operation not supported

The reason you're getting ENOSUP is you have to use the proper prefix.
"user.*" for example.
msg143106 - (view) Author: Benjamin Peterson (benjamin.peterson) * (Python committer) Date: 2011-08-28 16:49
After Antoine's review...
msg143294 - (view) Author: Roundup Robot (python-dev) Date: 2011-09-01 02:16
New changeset 4eb0b1819bda by Benjamin Peterson in branch 'default':
expose linux extended file system attributes (closes #12720)
http://hg.python.org/cpython/rev/4eb0b1819bda
msg143987 - (view) Author: Arfrever Frehtes Taifersar Arahesis (Arfrever) * Date: 2011-09-13 20:25
There is an inconsistency in used header and library.
attr/xattr.h and libattr.so belong to attr package (http://savannah.nongnu.org/projects/attr).
glibc provides sys/xattr.h and libc.so.
Both libattr.so and libc.so define getxattr(), setxattr() and other functions.
Modules/posixmodule.c includes attr/xattr.h from attr package, but libpython3.3.so isn't linked against libattr.so and is linked against libc.so, so functions from glibc are used at run time.

I suggest to use sys/xattr.h:
- sys/xattr.h instead of attr/xattr.h in Modules/posixmodule.c, configure, configure.in and pyconfig.h.in
- HAVE_SYS_XATTR_H instead of HAVE_ATTR_XATTR_H in Modules/posixmodule.c and pyconfig.h.in
msg143989 - (view) Author: Roundup Robot (python-dev) Date: 2011-09-13 21:20
New changeset 33f7044b5682 by Benjamin Peterson in branch 'default':
Use xattr functions from sys/xattr.h instead of attr/xattr.h (closes #12720)
http://hg.python.org/cpython/rev/33f7044b5682
msg144027 - (view) Author: Stefan Krah (skrah) * (Python committer) Date: 2011-09-14 15:10
The OS X buildbots fail to compile posixmodule.c:

gcc -fno-strict-aliasing -g -O0 -Wall -Wstrict-prototypes    -I. -I./Include    -DPy_BUILD_CORE  -c ./Modules/posixmodule.c -o Modules/posixmodule.o
./Modules/posixmodule.c: In function ‘try_getxattr’:
./Modules/posixmodule.c:10045: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c:10045: error: (Each undeclared identifier is reported only once
./Modules/posixmodule.c:10045: error: for each function it appears in.)
./Modules/posixmodule.c: In function ‘getxattr_common’:
./Modules/posixmodule.c:10083: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘posix_getxattr’:
./Modules/posixmodule.c:10101: warning: passing argument 3 of ‘getxattr_common’ from incompatible pointer type
./Modules/posixmodule.c: In function ‘posix_lgetxattr’:
./Modules/posixmodule.c:10119: error: ‘lgetxattr’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘wrap_fgetxattr’:
./Modules/posixmodule.c:10129: error: too few arguments to function ‘fgetxattr’
./Modules/posixmodule.c:10130: warning: control reaches end of non-void function
./Modules/posixmodule.c: In function ‘posix_setxattr’:
./Modules/posixmodule.c:10165: error: too few arguments to function ‘setxattr’
./Modules/posixmodule.c: In function ‘posix_lsetxattr’:
./Modules/posixmodule.c:10190: warning: implicit declaration of function ‘lsetxattr’
./Modules/posixmodule.c: In function ‘posix_fsetxattr’:
./Modules/posixmodule.c:10216: error: too few arguments to function ‘fsetxattr’
./Modules/posixmodule.c: In function ‘posix_removexattr’:
./Modules/posixmodule.c:10239: error: too few arguments to function ‘removexattr’
./Modules/posixmodule.c: In function ‘posix_lremovexattr’:
./Modules/posixmodule.c:10262: warning: implicit declaration of function ‘lremovexattr’
./Modules/posixmodule.c: In function ‘posix_fremovexattr’:
./Modules/posixmodule.c:10285: error: too few arguments to function ‘fremovexattr’
./Modules/posixmodule.c: In function ‘listxattr_common’:
./Modules/posixmodule.c:10327: error: ‘XATTR_LIST_MAX’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘posix_listxattr’:
./Modules/posixmodule.c:10369: warning: passing argument 2 of ‘listxattr_common’ from incompatible pointer type
./Modules/posixmodule.c: In function ‘posix_llistxattr’:
./Modules/posixmodule.c:10385: error: ‘llistxattr’ undeclared (first use in this function)
./Modules/posixmodule.c: In function ‘wrap_flistxattr’:
./Modules/posixmodule.c:10394: error: too few arguments to function ‘flistxattr’
./Modules/posixmodule.c:10395: warning: control reaches end of non-void function
./Modules/posixmodule.c: In function ‘all_ins’:
./Modules/posixmodule.c:11342: error: ‘XATTR_SIZE_MAX’ undeclared (first use in this function)
make: *** [Modules/posixmodule.o] Error 1
program finished with exit code 2
elapsedTime=20.601350
msg144029 - (view) Author: Roundup Robot (python-dev) Date: 2011-09-14 15:46
New changeset f325439d7f84 by Benjamin Peterson in branch 'default':
only compile xattrs on glibc (closes #12720)
http://hg.python.org/cpython/rev/f325439d7f84
History
Date User Action Args
2011-09-14 15:46:22python-devsetstatus: open -> closed

messages: + msg144029
2011-09-14 15:10:27skrahsetstatus: closed -> open
nosy: + skrah
messages: + msg144027

2011-09-13 21:20:55python-devsetstatus: open -> closed
resolution: fixed
messages: + msg143989
2011-09-13 20:25:41Arfreversetstatus: closed -> open
resolution: fixed -> (no value)
messages: + msg143987
2011-09-01 02:16:43python-devsetstatus: open -> closed

nosy: + python-dev
messages: + msg143294

resolution: fixed
stage: patch review -> resolved
2011-08-28 16:49:57benjamin.petersonsetfiles: + xattrs.patch

messages: + msg143106
2011-08-28 16:40:42benjamin.petersonsetmessages: + msg143105
2011-08-28 16:34:58pitrousetmessages: + msg143104
2011-08-28 12:59:15benjamin.petersonsetfiles: + xattrs.patch

messages: + msg143095
2011-08-18 18:43:11pitrousetnosy: + neologix
2011-08-18 17:08:26benjamin.petersonsetfiles: + xattrs.patch

messages: + msg142372
2011-08-17 22:29:17Arfreversetnosy: + Arfrever
2011-08-17 22:04:21pitrousetnosy: + pitrou
messages: + msg142293
2011-08-09 20:24:55benjamin.petersoncreate