When using urllib2 to fetch page data from an https server, I found that only the first 32768 characters of the download were retrieved. Other browsers returned the full documents, so it does not appear to be a server issue. If http, rather than https is used on the same server, the full document is retrieved. No problems with shorter documents (<32768 characters). They were not truncated.

Do you have the sample server URL or test script?

Here is the code (security info removed) and the output. I noticed that the problem is a bit different between 2.6.5 and 2.7.3 (on one the use of authentication is different) so I've included the output for both:

import urllib2

userData="Basic XXXX KEY GOES HERE"

emlUrl="https://pasta.lternet.edu/package/metadata/eml/knb-lter-vcr/25/27"
emlReq=urllib2.Request(emlUrl)
emlReq.add_header('Authorization', userData)
emlSock=urllib2.urlopen(emlReq,timeout=60)
emlString=emlSock.read()
print "Https,authenticated: "+str(len(emlString))

emlReq=urllib2.Request(emlUrl)
emlSock=urllib2.urlopen(emlReq,timeout=60)
emlString=emlSock.read()
print "Https,Not authenticated: "+str(len(emlString))

emlUrl="http://pasta.lternet.edu/package/metadata/eml/knb-lter-vcr/25/27"
emlReq=urllib2.Request(emlUrl)
emlReq.add_header('Authorization', userData)
emlSock=urllib2.urlopen(emlReq,timeout=60)
emlString=emlSock.read()
print "Http,authenticated: "+str(len(emlString))


emlReq=urllib2.Request(emlUrl)
emlSock=urllib2.urlopen(emlReq,timeout=60)
emlString=emlSock.read()
lengthHttpsNotAuthenticated=len(emlString)
print "Http,authenticated: "+str(len(emlString))

OUTPUT when run on PC using Python 2.6.5
Https,authenticated: 32768
Https,Not authenticated: 32768
Http,authenticated: 40898
Http,authenticated: 40898

OUTPUT when run on Ubuntu Linux (12.4LTS):
Https,authenticated: 32768
Https,Not authenticated: 40898
Http,authenticated: 40898
Http,authenticated: 40898

Python 2.7 is no longer maintained. Please create a new issue if you are seeing this problem on 3.9+.