I'm doing some USB user-space driver programming 
in Python 2.1.3 under Linux 2.4.

When using a particular ioctl (HIDIOCGNAME), the 
return value as well as the (copy_to_user) binary data 
is significant.

Here is the code from line 547 of hiddev.c (kernel 
2.4.19-pre7):
  if (_IOC_NR(cmd) == _IOC_NR(HIDIOCGNAME(0))) {
    int len;
    if (!hid->name) return 0;
    len = strlen(hid->name) + 1;
    if (len > _IOC_SIZE(cmd)) len = _IOC_SIZE(cmd);
    return copy_to_user((char *) arg, hid->name, len) ?
      -EFAULT : len;
  }

So here is the problem:  fcntl.ioctl() will only give me 
one or the other value, but not both.

I can work around this by initialising the input buffer to 
all chr(0), and then strip them off after.  But there will 
be a time when an ioctl call *requires* both values.

Until now I have appreciated the simple ioctl interface, 
but now it is shown to be an oversimplification.

It may be that POSIX, or some standard somewhere 
says that ioctl should not be used to do this, but 
maybe Python should support it anyway.

I suggest either:
1.  A new function ioctl2(fd, op, arg) that returns a 
2-tuple of (return_value, binary_buffer) and does not 
try to interpret the return value.
2.  An optional 4th parameter whose presence (but 
not value) requests the 2-tuple mentioned in (1).

Gotta love Python!

Logged In: YES 
user_id=29957

Ouch. I think you're right - and I think that the
ioctl2() implementation is probably the only solution.
(I'm not so keen on the name, but I guess it kinda 
follows the 'standard' of other python functions, e.g.
popen)
I know changing the return of ioctl would be mega-ugly,
unless there was a new, optional "return a tuple"
argument. I doubt that's the appropriate fix. 

If the former approach is the appropriate one, it should
be simple enough to add an ioctl2() to the C source. I 
can probably whip it up now...

Logged In: YES 
user_id=29957

possible patch at
http://sourceforge.net/tracker/index.php?func=detail&aid=555883&group_id=5470&atid=305470
I've only lightly tested it, not with any calls that return
useful values in the return code (couldn't find any easily, 
didn't feel like figuring out userspace drivers right now :)
Give it a whirl...

Logged In: YES 
user_id=21627

Thinking about the problem again, I think the right solution
would be to support array objects as arguments to ioctl -
those would then be passed into the system call.

Am I correctly understanding the problem, that this would
not require a new function?

Logged In: YES 
user_id=543663

Re Loewis:
Not sure what you mean, perhaps an example of the 
problem from me would help:

In C:
    char buf[256], strbuf[256];
    int ret, fd;
    fd = open(...)
    ret = ioctl(fd, IOCTL_NUMBER, buf);
    strncpy(strbuf, buf, ret);

This is impossible with the current fcntl module:
    buf = "X" * struct.calcsize("256s")
    buf = fcntl.ioctl(fd, IOCTL_NUMBER, buf)
    # The positive ioctl() return value is lost, and
    # hence I dont know how much of buf is significant

Extract from asm/unistd.h:

#define __syscall_return(type, res) \
do { \
        if ((unsigned long)(res) >= (unsigned long)(-125)) { \
                errno = -(res); \
                res = -1; \
        } \
        return (type) (res); \
} while (0)

So positive values are valid, but are being lost.

Anthony's patch looks good (thanks, I have not had a 
chance to test it myself yet), and the above example would 
be:
    buf = "X" * struct.calcsize("256s")
    ret, buf = fcntl.ioctl2(fd, IOCTL_NUMBER, buf)
    strbuf = buf[:ret] # Result!

Now, to test my understanding of your last comment about 
array objects Loewis, do you mean:
- pass in a mutable list that gets changed in place to [ret, 
buf]
- return (ret, buf) if called like this: ioctl(fd, IOC, [buf])
- or something else - please give example.

Thanks people.

Logged In: YES 
user_id=6656

grahamh: I think Martin understood you fine.

Remember that arrays are mutable, whereas strings are not.

So the idea would be that you'd write

buf = array.array('c', '\000'*256)
ret = fcntl.ioctl(fd, IOCTL_NUMBER, buf)

then the first "ret" bytes of "buf" would be valid.

However this possibly breaks backwards compatibility, as you
can already pass arrays into the ioctl function and they are
not modified.

Logged In: YES 
user_id=543663

Thanks Michael, its all incredibly clear now (sorry Martin).

I think using arrays sounds a great idea.
IMVHO as to backward compatibility, AFAICT its not 
documented that arrays can be passed to ioctl() anyway (only 
says int and string no?), so it's only breaking undocumented 
behaviour.

Logged In: YES 
user_id=21627

I would not be concerned about the backwards compatibility,
either: if anybody is passing arrays, there is a good chance
that the ioctl does not modify the buffer, or that the
caller would not worry about the (missing) modification - or
else we had seen reports before.

So documenting in Misc/NEWS and the documentation that
mutable  objects supporting the (write) buffer interface may
now mutate seems sufficient.

Logged In: YES 
user_id=543663

I just realised - using array objects and changing in-place 
oversomes the arbitrary limit of 1024 bytes.  We could then 
achieve the full 16384 byte nirvana - hurrah!

Logged In: YES 
user_id=6656

I'm not *trying* to be a party-pooper, but there's another
problem: the point of passing a mutable array is so that
fcntl.ioctl can return ioctl(2)'s return code rather than
the string it returns now.  So if someone does use arrays
(or I guess Numeric.arrays?) for ioctls this would very
likely cause breakage.  I don't know how likely that is. 
Probably not very.  But I know that if this change broke my
code, I'd be a bit miffed.

All this is a bit annoying, as mutating arrays is clearly
the right solution -- where's that time machine?

Logged In: YES 
user_id=29957

Hm. The solaris /proc lib I did in python, I used 'struct'
to dismantle C structures, but I did consider using byte
arrays. So that's one data point. 
(ok, it's not ioctls, but it's similar sort of data).
I'm loathe to go down the 'argument is mutable' optional
flag, but I really would prefer to make this work rather
than the ioctl2() approach, which really ain't that nice.

Logged In: YES 
user_id=21627

I'm in favour of a you-may-mutate-the-buffer flag. A
transition strategy would be:
- in 2.3, make the flag optional, with default 0
- in 2.4, warn if it is not specified
- in 2.5, change the default
All the time, if the buffer isn't mutable, specifying the
flag should not be required.

Logged In: YES 
user_id=6656

OK, that would seem to be easy enough to implement (well,
the handling of writeable buffers, not all the optional
hair).  I like this:

>>> buf = array.array('h', [0]*4)
[19139 refs]
>>> fcntl.ioctl(0, termios.TIOCGWINSZ, buf)
0
[19139 refs]
>>> buf
array('h', [47, 137, 841, 615])
[19139 refs]

I like this less:

>>> import array, fcntl
[19089 refs]
>>> buf = array.array('h', [0]*3)
[19093 refs]
>>> fcntl.ioctl(0, termios.TIOCGWINSZ, buf)
0
[19095 refs]
>>> del buf
Debug memory block at address p=0x401c3cb8:
    6 bytes originally requested
    The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
    The 4 pad bytes at tail=0x401c3cbe are not all
FORBIDDENBYTE (0xfb):
        at tail+0: 0x67 *** OUCH
        at tail+1: 0x02 *** OUCH
        at tail+2: 0xfb
        at tail+3: 0xfb
    The block was made by call #16015 to debug malloc/realloc.
    Data at p: 2f 00 89 00 49 03
Fatal Python error: bad trailing pad byte
Aborted

I'm not sure what we can possibly do about this?  Copy into
a fixed buffer like we do today, then copy out again after
the ioctl?  Probably the best option.

Logged In: YES 
user_id=6656

Try the attached.

(a) should have docs.  This is getting hairy (if this then
that, but if this and not that then the other...).
(b) I've made no attempt to get this to hand out sane error
messages (see above).
(c) would be nice to have tests.  how on earth do you test
ioctl?

Logged In: YES 
user_id=29957

There's a very simple test_ioctl.py in 
patch #555883 (the ioctl2() sample implementation).
It does a getpgrp(), then a TIOCGPGRP and checks 
it gets the same stuff. No, it won't work on Windows.

I'm going to close off #555883 as Invalid.

Logged In: YES 
user_id=543663

Still here...
Thanks for the help people.
Things have been a bit crazy ATM, but should be back to 
working with this again soon, so I'll let you know how it goes.

Logged In: YES 
user_id=6656

I'd like to either move this forward or close it.

Is/was there agreement that my patch would be OK if brushed
up as described below, i.e. add tests, docs, provide better
error messages?

(actually, I note that my patch's use of
Py_BEGIN_ALLOW_THREADS is amusingly broken... gotta love the
C preprocessor).

Logged In: YES 
user_id=21627

The patch looks good, in principle, so please correct it, and 
provide documentation (do mention that you need to pass 
1024 bytes to avoid copying).

Logged In: YES 
user_id=6656

Assign to me in the hope that I'll stop forgetting about it
(sigh).

I'm about 70% of the way through what's needed I'd guess. 
Try to finish for 2.3a2...

Logged In: YES 
user_id=6656

Argh, how does SF CVS know to slow down just as I start to
work on my bugs?

Anyway, I think this patch will do.  Adds a test, docs, and
beats up on ioctl's docstring.

The only issue left that I know about is the error message
that you get when none of the PyArg_ParseTuples pass -- but
telling the full story would make the error message
ridiculously long, so I don't much care.

Tempted to change it to

ioctl: ha ha, you stuffed up!  read the docstring

<wink>.

Martin, can you review?

Logged In: YES 
user_id=6656

Hmm, Martin's away and this should probably go in 2.3a2. 
Anthony, can you give it a once-over?

Logged In: YES 
user_id=21627

The patch looks fine, please apply it (along with a NEWS entry).

Also file a bug report indicating that the warning should be
added in 2.4, or come up with another place to collect
things that need to be changed in a future version (there
are quite a number of such transitions going on).

Logged In: YES 
user_id=6656

We got there in the end!  Checked in as

Misc/NEWS revision 1.686
Lib/test/test_ioctl.py revision 1.1
Doc/lib/libfcntl.tex revision 1.31
Modules/fcntlmodule.c revision 2.38

Not closing because of the "change needed in 2.4" bit --
should there be a PEP for this?  Or would a bug in the
tracker do?

Logged In: YES 
user_id=21627

Creating a new bug report and closing this one would be
best, IMO, otherwise, everybody has to read through all
these messages to find out that it is still open just
because of a tiny detail. I'd title it

Python 2.4: Warn about omitted mutable_flag.

Logged In: YES 
user_id=6656

OK, done: 696535.

I added a Python 2.4 category :-)