classification
Title: urllib2 bug on CentOS
Type: behavior Stage:
Components: Extension Modules Versions: Python 2.6
process
Status: closed Resolution: not a bug
Dependencies: Superseder:
Assigned To: Nosy List: loewis, orsenthil, pr0gg3d, rk3dov, rpetrov
Priority: normal Keywords:

Created on 2009-07-29 10:35 by rk3dov, last changed 2009-08-03 20:02 by rpetrov. This issue is now closed.

Messages (10)
msg91034 - (view) Author: Anton (rk3dov) Date: 2009-07-29 10:35
This code gives HTTP Error 500 on CentOS:
---------------------------------------------
import urllib2
url = 'http://wm.exchanger.ru/asp/XMLWMList.asp?exchtype=1'
t = urllib2.urlopen(url).read()
---------------------------------------------

tcpdump:
---------------------------------------------
...
GET /asp/XMLWMList.asp?exchtype=1?3d2ebf80 HTTP/1.1
Accept-Encoding: identity
Host: wm.exchanger.ru
Connection: close
User-Agent: Python-urllib/2.6
...
---------------------------------------------
"?3d2ebf80" appended to request

% uname -a
Linux xxx.xxx 2.6.18-028stab062.3-ent #1 SMP Thu Mar 26 15:12:05 MSK 
2009 i686 i686 i386 GNU/Linux

The same problem with python 2.4.3. On other systems this code works 
nice.
msg91063 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-07-29 19:01
What C compiler have you been using?
msg91069 - (view) Author: Anton (rk3dov) Date: 2009-07-29 20:21
% gcc -v
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --
infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-
checking=release --with-system-zlib --enable-__cxa_atexit --disable-
libunwind-exceptions --enable-libgcj-multifile --enable-
languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --
disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-
gcj-1.4.2.0/jre --with-cpu=generic --host=i386-redhat-linux
Thread model: posix
gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)
msg91089 - (view) Author: Anton (rk3dov) Date: 2009-07-30 09:43
The same problem with gcc 4.3.2
msg91111 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-07-30 20:40
Can you rule out hardware problems?
msg91127 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2009-07-31 06:18
I would also suggest run the following command:

$python /usr/local/lib/python2.6/test/regrtest.py -v test_urllib2
test_urllib test_socket > output-of-tests.txt

And attach output-of-tests.txt. It might help in getting to
the root cause of the issue.
msg91226 - (view) Author: Francesco Del Degan (pr0gg3d) Date: 2009-08-03 16:35
I don't think that this is a python issue, because of this:

[root@localhost ~]# curl -O http://wm.exchanger.ru/asp/XMLWMList.asp?exchtype=1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
[root@localhost ~]# curl -O http://wm.exchanger.ru/asp/XMLWMList.asp?exchtype=1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9299  100  9299    0     0  15471      0 --:--:-- --:--:-- --:--:-- 22134


I done two request in rapid succession and into first i got a redirect to

	0x0000:  4500 008f 3abb 0000 3206 1d98 d49e ad94  E...:...2.......
	0x0010:  ac10 01d3 0050 b220 47ee 6cdb 8b3d 6233  .....P..G.l..=b3
	0x0020:  5011 0001 edc2 0000 4854 5450 2f31 2e31  P.......HTTP/1.1
	0x0030:  2033 3032 204d 6f76 6564 2054 656d 706f  .302.Moved.Tempo
	0x0040:  7261 7269 6c79 0d0a 436f 6e74 656e 742d  rarily..Content-
	0x0050:  4c65 6e67 7468 3a20 300d 0a4c 6f63 6174  Length:.0..Locat
	0x0060:  696f 6e3a 202f 6173 702f 584d 4c57 4d4c  ion:./asp/XMLWML
	0x0070:  6973 742e 6173 703f 6578 6368 7479 7065  ist.asp?exchtype
	0x0080:  3d31 3f34 6430 3266 3136 380d 0a0d 0a    =1?4d02f168....

as you can see, the ?4d02f168 part comes from the site, hence the 500 error from second request.

In the second try, i got correct response.

The weird thing is that into other systems, no curl request triggers a redirect from the site,
and in centOS only we have this weird behaviour.
msg91227 - (view) Author: Francesco Del Degan (pr0gg3d) Date: 2009-08-03 16:38
Update: Now into the same system (CentOS) without any mod:

>>> import urllib2
>>> url = 'http://wm.exchanger.ru/asp/XMLWMList.asp?exchtype=1'
>>> t = urllib2.urlopen(url).read()
>>>
>>> t
'<?xml version="1.0"?>.........

i thinks that you should try to look for some bugs into CentOS 
distribution.
msg91228 - (view) Author: Senthil Kumaran (orsenthil) * (Python committer) Date: 2009-08-03 16:41
Thanks for the update. I am closing this as Invalid.
msg91239 - (view) Author: Roumen Petrov (rpetrov) * Date: 2009-08-03 20:02
10x for closing. It seems to me python is not mature as http client.
History
Date User Action Args
2009-08-03 20:02:25rpetrovsetnosy: + rpetrov
messages: + msg91239
2009-08-03 16:41:03orsenthilsetstatus: open -> closed
resolution: not a bug
messages: + msg91228
2009-08-03 16:38:53pr0gg3dsetmessages: + msg91227
2009-08-03 16:35:21pr0gg3dsetnosy: + pr0gg3d
messages: + msg91226
2009-07-31 06:18:57orsenthilsetnosy: + orsenthil
messages: + msg91127
2009-07-30 20:40:26loewissetmessages: + msg91111
2009-07-30 09:43:20rk3dovsetmessages: + msg91089
2009-07-29 20:21:22rk3dovsetmessages: + msg91069
2009-07-29 19:01:49loewissetnosy: + loewis
messages: + msg91063
2009-07-29 10:37:15rk3dovsettype: behavior
2009-07-29 10:35:57rk3dovcreate