Title: fix test_grp failing when NIS entries present
Type: behavior Stage: resolved
Components: Extension Modules Versions: Python 3.1, Python 3.2, Python 2.7
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: anthonybaxter Nosy List: anthonybaxter, barry, bobbyi, brian.curtin, dmalcolm, doerwalter, jafo, jbrouwers, loewis, nnorwitz, r.david.murray, tseaver
Priority: high Keywords: patch

Created on 2003-07-22 23:07 by nnorwitz, last changed 2010-12-14 16:26 by r.david.murray. This issue is now closed.

File name Uploaded Description Edit
grp.diff nnorwitz, 2003-07-22 23:07 patch to Modules/grpmodule.c
test_grp.patch bobbyi, 2010-10-25 23:45 Patch for test_grp in py3k review
Messages (17)
msg17264 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2003-07-22 23:07
I didn't really think this should go into 2.3, but I'll
let you make the decision.  This patch fixes the
test_grp failure on RedHat 6.2/Alpha (asmodean) in the
snake-farm.  I thought it was specific to RH 6.2,
apparently it's not.  If you add a + as the last line
in /etc/group the test will fail on RH 9 too.

Walter Doerwald may know more about how best to fix
this.  I'm not certain if it's really a problem in the
extension module or the test.
If you want to fix the test, the patch is included here:

     def check_value(self, value):
         # check that a grp tuple has the entries and
         # attributes promised by the docs
+        if value == ('+', None, 0, []):
+            # some libc's return the last line of +
+            return
msg17265 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-27 16:22
Logged In: YES 

Remind me what + on a line means in /etc/group.  "man group"
on RH9 gives no clue.  I'm still nervous about it, can we
defer until 2.3.1?
msg17266 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2003-07-28 18:36
Logged In: YES 

pwdmodule.c hasn't changed in 7 months, so I'd say this is a
problem with the test. But ('+', None, 0, []) doesn't seem
to be a valid entry, so (although it's not a bug in Python)
maybe Python should hide this entry.
msg17267 - (view) Author: Neal Norwitz (nnorwitz) * (Python committer) Date: 2003-07-28 21:56
Logged In: YES 

I don't recall what the + is for either.  Does it have
something to do with YP/NIS?  I don't think this is really
critical so I'm droping the priority.  If  pwdmodule.c is
changed, it will be an interface change so it's questionable
for 2.3.1.

This was found on the snake farm, never reported by a user.
msg17268 - (view) Author: Jean M. Brouwers (jbrouwers) Date: 2003-07-28 23:24
Logged In: YES 

The '+ ...' line at the end of the /etc/passwd file tells
the system to also pickup information from the appropriate
map in LDAP, or NIS (yellow pages).  See also section 7.2 on
this page:

msg17269 - (view) Author: Barry A. Warsaw (barry) * (Python committer) Date: 2003-07-29 03:01
Logged In: YES 

Thanks Jean for the independent verification.  We'll fix
this for Python 2.3.1.
msg17270 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2003-08-30 17:28
Logged In: YES 

Now that 2.3 is out the door, should this be checked in?
msg17271 - (view) Author: Anthony Baxter (anthonybaxter) (Python triager) Date: 2003-11-06 15:35
Logged In: YES 

I'll fix this for 2.3.3, because it causes test failures on
the HP testdrive boxes, and that annoys me <wink>.
msg17272 - (view) Author: Walter Dörwald (doerwalter) * (Python committer) Date: 2005-10-09 19:56
Logged In: YES 

Anthony, can this go into 2.5?
msg55360 - (view) Author: Sean Reifschneider (jafo) * (Python committer) Date: 2007-08-28 09:14
A few comments on this one:

Perhaps the tests should be patched instead of the grp module?  Because
it may be useful for someone to know that old-style NIS maps are being

I presume if the change is made in grp that it should also be made in pwd?

Shouldn't the patch do a "continue" instead of a break?  I believe the
"+" entry means "include the contents of the NIS map here (I don't
believe it can be used for LDAP), and as far as I know does not HAVE to
be included at the end of the file.

We probably don't care, but just to be canonical it seems that there are
also "+name" and "-name" entries in /etc/group and /etc/passwd.  "+name"
in an entry is the same as a "name" entry, but the additional fields are
read from NIS.  "-name" means that "name" is not a valid user (in other
words, don't let "name" login, even if they exist in NIS).  I can't
imagine we want to implement those though, because...

Note that the "+" syntax is the old syntax from before nsswitch was
introduced.  As far as I know, + syntax was deprecated back in 1995 when
nsswitch.conf was added.

In conclusion, I think the tests that the tests that annoy Anthony
should be fixed, but I'm not convinced the behavior of grp should be
changed.  Particularly without a corresponding change in pwd.


See the following URL for more information about the +/- syntax:
msg97970 - (view) Author: Brian Curtin (brian.curtin) * (Python committer) Date: 2010-01-17 19:35
Ping. Anyone know if this is still an issue? I don't have access to RH.
msg104607 - (view) Author: Tres Seaver (tseaver) * Date: 2010-04-30 01:16
I don't have a host where I actually use netgroups in /etc/groups, and so can't provoke the bug::

  $ hg branch
  $ make
  $ ./python -E -tt Lib/test/ test_grp
  1 test OK.

I can verify that the patch still applies to the 'release26-maint' branch::

  $ patch -p0 < /tmp/grp.diff 
  patching file Modules/grpmodule.c
  Hunk #1 succeeded at 134 (offset 12 lines).

and that the tests still pass aftewards::

  $ make
  $ ./python -E -tt Lib/test/ test_grp
  1 test OK.
msg119585 - (view) Author: Bobby Impollonia (bobbyi) Date: 2010-10-25 23:45
I am encountering this issue with py3k on a Debian machine. test_grp consistently fails due to a line with a plus sign in /etc/group

I believe that the previous patch attached to this bug is not correct due to the issues others have raised: It only handles lines with a plus sign as the name (not any name starting with '+' or '-') and it breaks out of the loop when it finds one, silently discarding any remaining lines.

I've attached a new patch which preserves the functionality of getgrall but updates its docstring to document that any group names starting with '+' or '-' represent NIS-related entries. The patch fixes the test itself so that it skips over NIS-related entries rather than attempting to look them up with grp.getgrnam.

With this patch applied (to the py3k branch HEAD), all regression tests pass on my machine.
msg120092 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-10-31 21:55
I think this approach (of file19364) is reasonable.
msg120695 - (view) Author: Bobby Impollonia (bobbyi) Date: 2010-11-08 00:00
Martin, are you able to check in this fix? If not, what should I do and whom should I talk to so this can be checked in?

msg120696 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-11-08 00:02
I'm able to, but I may not find time. Ask on python-dev; if you feel you need to push this, offer to review issues in return.
msg123959 - (view) Author: R. David Murray (r.david.murray) * (Python committer) Date: 2010-12-14 16:26
Committed to py3k in r87238, 3.1 in r87239, and 2.7 in r87240.  Thanks, Bobby.
Date User Action Args
2010-12-14 16:26:48r.david.murraysetstatus: open -> closed

nosy: + r.david.murray
messages: + msg123959

resolution: fixed
stage: patch review -> resolved
2010-11-08 00:02:11loewissetmessages: + msg120696
2010-11-08 00:00:22bobbyisetmessages: + msg120695
2010-11-01 00:01:35eric.araujosetversions: + Python 3.1, Python 2.7, - Python 2.6, Python 3.3
2010-10-31 21:55:02loewissetnosy: + loewis
messages: + msg120092
2010-10-25 23:45:57bobbyisetfiles: + test_grp.patch
title: fix test_grp failing on RedHat 6.2 -> fix test_grp failing when NIS entries present
nosy: + bobbyi

messages: + msg119585

versions: + Python 3.2, Python 3.3
2010-04-30 01:16:01tseaversetnosy: + tseaver
messages: + msg104607
2010-01-17 20:28:45dmalcolmsetnosy: + dmalcolm
2010-01-17 19:35:35brian.curtinsetversions: + Python 2.6, - Python 2.5
nosy: + brian.curtin

messages: + msg97970

type: behavior
stage: patch review
2008-01-04 00:57:20christian.heimeslinkissue779218 superseder
2008-01-04 00:10:00christian.heimessetkeywords: + patch
versions: + Python 2.5, - Python 2.3
2007-08-28 09:14:55jafosetnosy: + jafo
messages: + msg55360
2003-07-22 23:07:48nnorwitzcreate