classification
Title: Please include pdb with windows distribution
Type: enhancement Stage: patch review
Components: Build Versions: Python 3.2
process
Status: closed Resolution: accepted
Dependencies: Superseder:
Assigned To: Nosy List: cgrebeld, jbit, lemburg, loewis, lylefile, stutzbach
Priority: normal Keywords: patch

Created on 2005-09-24 19:30 by lylefile, last changed 2012-05-01 17:41 by loewis. This issue is now closed.

Files
File name Uploaded Description Edit
pdbzip-20100804-1.patch jbit, 2010-08-03 22:42 review
pdb.patch stutzbach, 2010-09-03 21:02
Messages (17)
msg61207 - (view) Author: Lyle Thompson (lylefile) Date: 2005-09-24 19:30
Checking out the source files and rebuilding is not 
guaranteed to build a program database (pdb) file that 
represents the distribution python DLL. This may be due 
to a differences in the windows version or visual studio 
service pack on my system vs the one used to build the 
distribution, but tracking it down can be very time 
consuming. Including the pdb in the distribution should 
be trivial and avoid this problem entirely. Although I can 
fix the problem for future releases of our product, I am 
also supporting previous releases that included the 
standard DLL included in the distribution.
msg61208 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2005-09-27 20:03
Logged In: YES 
user_id=21627

Including the PDB files in the MSI is probably not
acceptable to the majority of the users. python24.pdb is
3.5MiB in size, so the MSI file to download would grow
considerably.

Why do you need the PDB file in the first place?
msg61209 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2005-12-28 00:12
Logged In: YES 
user_id=21627

Update: distribution in a separate zipfile would be
possible, provided that a patch to the build process (in
Tools/msi/msi.py) would be contributed.
msg61210 - (view) Author: Lyle Thompson (lylefile) Date: 2006-01-03 19:06
Logged In: YES 
user_id=1351115

A PDB file is required for debugging crash dumps from 
complex pythonic programs. This ability is absolutely 
critical for commercial programs using python. Technically, 
the crashes only happen within pyds or dlls, but it is 
often very vitally important to be able to find out what 
python code was being interpreted at the time. 
Unfortunately, I've even seen "pure" python crash in one of 
the supplied pyd's. Of course, the pdb file should be in a 
separate download, as few users require the ability to 
reverse engineer crash dumps.
msg61211 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2006-01-03 22:01
Logged In: YES 
user_id=21627

Just in case I wasn't clear in my last message: patches in
that direction are appreciate; without a contributed patch,
nothing will happen.
msg81671 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2009-02-11 21:03
Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
be enough.

The distributed DLL already points to the source location on your system
:-) E.g. for Python 2.5, this was c:\loewis\25\python\PCbuild\python25.pdb

Or are you thinking of having a separate MSI installer with all the .pdb
files included ?
msg81673 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2009-02-11 21:20
> Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
> be enough.

It's not clear to me what is being requested: just that single PDB file,
or also the PDB files for all of the extension modules? Also, shouldn't
the file be compressed somehow?

I also dislike the clutter this would add to the release page, which
already has a confusing number of files available for download.
msg81686 - (view) Author: Marc-Andre Lemburg (lemburg) * (Python committer) Date: 2009-02-11 23:02
On 2009-02-11 22:20, Martin v. Löwis wrote:
> Martin v. Löwis <martin@v.loewis.de> added the comment:
> 
>> Hi Martin, wouldn't just uploading the pythonXX.pdb to the release page
>> be enough.
> 
> It's not clear to me what is being requested: just that single PDB file,
> or also the PDB files for all of the extension modules? Also, shouldn't
> the file be compressed somehow?
> 
> I also dislike the clutter this would add to the release page, which
> already has a confusing number of files available for download.

I see these possibilities:

 * add just the Python DLL PDF file to the page

 * add a ZIP file with just the PDB files for all components

 * add a second MSI file which includes and installs the PDB
   files alongside the .dll/.pyd/.exe files

 * add the PDB files to the standard MSI files

We could put such files into a separate section on the release page.

Note that the PDB files are just around 1.6MB in compressed form
(probably less if integrated into the MSI file), so the last option
looks like the easiest solution.

FWIW: I don't think that the current release page layout looks
cluttered:

    http://www.python.org/download/releases/2.6.1/
msg102494 - (view) Author: Chris Grebeldinger (cgrebeld) Date: 2010-04-06 21:10
Hi Martin,
  As an aside to the request, is the pdb for the python dll currently available anywhere?
msg102503 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-04-06 22:56
>   As an aside to the request, is the pdb for the python dll currently available anywhere?

Only on my hard disk, and then only for the very last release. The
status of the issue is the same as in 2006: without patches, nothing
will happen.
msg112390 - (view) Author: James Lee (jbit) Date: 2010-08-01 22:09
Right now if you have any moderately complex Python based application (or extension set) the only way to easily debug it on windows is by building Python yourself, which is a horrible solution since it means you may end up with a subtly different version to what your crash dump is against.
While including the PDB in the MSI distribution would require a patch (and might not be the best solution since 99% of people won't need it), copying the PDB to some place does not, so I'm not sure why this hasn't been done years ago.
Even if it's not in a super easy-to-find place (ie: not on the download page), placing the PDB somewhere will greatly help developers who need to debug Python internals in visual studio.
msg112393 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-08-01 22:25
> While including the PDB in the MSI distribution
> would require a patch (and might not be the best solution since 99%
> of people won't need it), copying the PDB to some place does not

This is not the case. I'm not at all clear what specific files are
being requested. So I'm unable to comply with this request as I
still don't understand it. FWIW, I have currently 108 PDB files
in my Python build area - I don't assume that people want me to upload
all of them (and it would be much too tedious to do so).

If a patch would be provided that zipped up (say) all stuff that
needs to be zipped up, and put that next to the MSI file, that
would be very much appreciated.
msg112402 - (view) Author: James Lee (jbit) Date: 2010-08-01 23:15
Ah, sorry, I see what you mean now... I thought the request for patch was to modify the installer itself, but it meant just make the msi.py script generate a separate zip file alongside the .msi installer.
I'll take a look at providing a patch with this over the next couple of days since I'd really really like a build in the relatively near future to provide this :)
msg112714 - (view) Author: James Lee (jbit) Date: 2010-08-03 22:42
Attached is a simple patch against msi.py from the py3k branch. It generates a .zip file containing all the PDB files (minus a select few) in the PCbuild directory. I imagine most people only want python31.pdb, but it is very frustrating when you have a crash without debug information so all distributed executables PDBs are included.
One thing to watch out for is if you have built a debug build and not cleaned the directory it will also include debug PDBs (this could be easily fixed if needed).
I have tested the output by making an MSI package and corresponding zip, installing the MSI on another machine, causing a crash within a python module and pointing the visual studio debugger at the PDB files from the zip. Symbols are shown and source level debugging works if you point visual studio to a corresponding python source tree.
This should make debugging of crashdumps of python modules or programs which embed python much easier for windows developers.
msg115507 - (view) Author: Daniel Stutzbach (stutzbach) (Python committer) Date: 2010-09-03 21:02
Attached is a slightly improved patch.  It cleans up some whitespace issues with James's patch and filters out *_d.pdb files if they exist.  I tested it on my system and it correctly produces a .zip file containing the .pdb files, next to the .msi file

Martin, do you need this patch to have any other functionality in order to make the python-*-pdb.zip file available after a release?
msg115574 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2010-09-04 14:39
Thanks for the patch, committed as r84487. I'll try it out with the next 3.2 alpha release.

Leaving this open for possible backports.
msg159760 - (view) Author: Martin v. Löwis (loewis) * (Python committer) Date: 2012-05-01 17:41
All active branches use this now.
History
Date User Action Args
2012-05-01 17:41:09loewissetstatus: open -> closed

messages: + msg159760
2010-09-04 14:39:20loewissetresolution: accepted
2010-09-04 14:39:13loewissetmessages: + msg115574
2010-09-03 21:03:00stutzbachsetfiles: + pdb.patch

messages: + msg115507
2010-08-22 01:00:33BreamoreBoysetstage: patch review
versions: + Python 3.2, - Python 3.0, Python 2.7
2010-08-03 22:42:34jbitsetfiles: + pdbzip-20100804-1.patch
keywords: + patch
messages: + msg112714
2010-08-01 23:15:33jbitsetmessages: + msg112402
2010-08-01 22:25:02loewissetmessages: + msg112393
2010-08-01 22:09:45jbitsetnosy: + jbit
messages: + msg112390
2010-04-06 22:56:46loewissetmessages: + msg102503
2010-04-06 21:10:31cgrebeldsetnosy: + cgrebeld
messages: + msg102494
2009-02-11 23:02:11lemburgsetmessages: + msg81686
2009-02-11 21:20:03loewissetmessages: + msg81673
2009-02-11 21:03:32lemburgsetversions: + Python 3.0, Python 2.7
2009-02-11 21:03:14lemburgsetnosy: + lemburg
messages: + msg81671
2009-01-07 20:03:46stutzbachsetnosy: + stutzbach
2005-09-24 19:30:29lylefilecreate