classification
Title: netrc parsing is overly strict
Type: Stage:
Components: Versions: Python 3.7, Python 2.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: ianwremmel, xiang.zhang, xtreak
Priority: normal Keywords:

Created on 2018-10-05 17:21 by ianwremmel, last changed 2018-10-10 03:20 by xiang.zhang.

Messages (4)
msg327155 - (view) Author: Ian Remmel (ianwremmel) Date: 2018-10-05 17:21
This started as a bug report for httpie https://github.com/jakubroztocil/httpie/issues/717#issuecomment-426125261

And became a bug report for requests https://github.com/requests/requests/issues/4813

> But turned out to be an issue with Python's netrc parser:
> 
>
> it appears that auth via netrc is broken if ~/.netrc includes entries that are not exactly login/password tuples. For example, I have the following entries for circle ci and heroku:
>
> ```
>    machine api.heroku.com
>      login <redacted>
>      password <redacted>
>      method interactive
>    machine circleci.com
>      login <redacted>
> ```
>
> both of these entries prevent my entry for github.com from working with httpie (but curl works just fine).


I've used the following script to test python 2.7 and 3.7:

```
import netrc
import os.path

netrc.netrc(os.path.expanduser('~/.netrc')).authenticators('api.github.com')
```

Python 2:
```
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    netrc.netrc(os.path.expanduser('~/.netrc')).authenticators('api.github.com')
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/netrc.py", line 35, in __init__
    self._parse(file, fp, default_netrc)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/netrc.py", line 117, in _parse
    file, lexer.lineno)
netrc.NetrcParseError: bad follower token 'method' (/Users/ian/.netrc, line 7)
````

Python 3:
```
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    netrc.netrc(os.path.expanduser('~/.netrc')).authenticators('api.github.com')
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/netrc.py", line 30, in __init__
    self._parse(file, fp, default_netrc)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/netrc.py", line 111, in _parse
    file, lexer.lineno)
netrc.NetrcParseError: bad follower token 'method' (/Users/ian/.netrc, line 7)
```
msg327201 - (view) Author: Karthikeyan Singaravelan (xtreak) * (Python triager) Date: 2018-10-05 23:50
Thanks for the report. There is no spec for .netrc files and the closest I can find is [0]. The error is present in master also. Could this be considered as an enhancement?

[0] https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html

Thanks
msg327204 - (view) Author: Ian Remmel (ianwremmel) Date: 2018-10-06 00:02
Yea, somehow, I suspected it was because there's no formal spec :)

I guess technically it's an enhancement, but given that configuration dictated by third-parties can break the environment, it does feel like a bug. 

For example, I can't use a python app to authenticate to github via netrc because of how heroku says I have to configure my netrc.

Also, there are probably two subtly different issues:
1. a machine with a password but no login breaks parsing
2. a machine with an unrecognized key breaks parsing
msg327453 - (view) Author: Xiang Zhang (xiang.zhang) * (Python committer) Date: 2018-10-10 03:20
My PR https://github.com/python/cpython/pull/127 has tried to solve some restrictions of the current netrc library. It's somewhat outdated since no reviewer for a long time. As you can see there are also other libraries suffering from the restrictions.

Revising it I think it still doesn't solve the case here, although it should be easy. There is no formal spec we can refer to. :-( But since there is real world problem and curl allows it, we should take it into mind.
History
Date User Action Args
2018-10-10 03:20:15xiang.zhangsetnosy: + xiang.zhang
messages: + msg327453
2018-10-06 00:02:54ianwremmelsettitle: netrc parding is overly strict -> netrc parsing is overly strict
2018-10-06 00:02:30ianwremmelsetmessages: + msg327204
2018-10-05 23:50:33xtreaksetmessages: + msg327201
2018-10-05 17:46:37xtreaksetnosy: + xtreak
2018-10-05 17:21:24ianwremmelcreate