➜
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2011-06-22 11:22 by vinay.sajip, last changed 2022-04-11 14:57 by admin. This issue is now closed.
| Messages (20) | |||
|---|---|---|---|
| msg138821 - (view) | Author: Vinay Sajip (vinay.sajip) *  |
Date: 2011-06-22 11:22 | |
This part of install_distinf.run():
if install_data.get_resources_out() != []:
resources_path = os.path.join(self.distinfo_dir,
'RESOURCES')
logger.info('creating %s', resources_path)
with open(resources_path, 'wb') as f:
writer = csv.writer(f, delimiter=',',
lineterminator='\n',
quotechar='"')
for tuple in install_data.get_resources_out():
writer.writerow(tuple)
fails at the writerow line:
creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/METADATA
creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/INSTALLER
creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/REQUESTED
creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/RESOURCES
Traceback (most recent call last):
File "/tmp/venv/bin/pysetup3", line 5, in <module>
sys.exit(main())
File "/usr/local/lib/python3.3/packaging/run.py", line 678, in main
return dispatcher()
File "/usr/local/lib/python3.3/packaging/run.py", line 667, in __call__
return func(self, self.args)
File "/usr/local/lib/python3.3/packaging/run.py", line 204, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.3/packaging/run.py", line 247, in _install
if install_local_project(target):
File "/usr/local/lib/python3.3/packaging/install.py", line 125, in install_local_project
return _run_install_from_dir(path)
File "/usr/local/lib/python3.3/packaging/install.py", line 160, in _run_install_from_dir
func(source_dir)
File "/usr/local/lib/python3.3/packaging/install.py", line 90, in _run_packaging_install
dist.run_command('install_dist')
File "/usr/local/lib/python3.3/packaging/dist.py", line 761, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.3/packaging/command/install_dist.py", line 526, in run
self.run_command(cmd_name)
File "/usr/local/lib/python3.3/packaging/command/cmd.py", line 329, in run_command
self.distribution.run_command(command)
File "/usr/local/lib/python3.3/packaging/dist.py", line 761, in run_command
cmd_obj.run()
File "/usr/local/lib/python3.3/packaging/command/install_distinfo.py", line 116, in run
writer.writerow(tuple)
TypeError: 'str' does not support the buffer interface
I think the open(resources_path) should use 'w' rather than 'wb' as the open mode.
Relevant part of setup.cfg:
resources =
virtualenvwrapper.sh = {scripts}
I know I can put it in the scripts = section, but I'm testing having something in resources, which ought to work ...
|
|||
| msg138913 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-06-24 11:24 | |
It is strange that neither the tests for install_distinfo and resources did not catch this. I agree about the nonsense of opening in binary mode. |
|||
| msg138921 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-06-24 11:52 | |
I suppose it's because I'm writing a script as if it were data? (i.e. to the {scripts} location.
|
|||
| msg142977 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-08-25 16:20 | |
Vinay: I don’t understand your message. I wasn’t part of the effort that added the resources subsystem, so I will work on a patch for this bug to learn more about it. |
|||
| msg142983 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-08-25 17:30 | |
Éric: I only meant that when writing data resources, one might reasonably use 'wb', but when writing scripts, which are text, 'w' is more appropriate. |
|||
| msg142984 - (view) | Author: trevor (trevor) | Date: 2011-08-25 18:17 | |
i see the same behavior - the error occurs leaving an empty RESOURCES file |
|||
| msg142986 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-08-25 18:42 | |
BTW, IIRC I have fixed it in the pythonv branch. https://bitbucket.org/vinay.sajip/pythonv |
|||
| msg143264 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-08-31 15:52 | |
> I only meant that when writing data resources, one might reasonably > use 'wb', but when writing scripts, which are text, 'w' is more > appropriate. I don’t see why. All text is bytes, so we can do all I/O in bytes when writing resources and avoid special-casing. > BTW, IIRC I have fixed it in the pythonv branch. > https://bitbucket.org/vinay.sajip/pythonv A link to a specific changeset or file would be great. [trevor] > i see the same behavior - the error occurs leaving an empty RESOURCES file Do you see that when running a test, a command or some other code? |
|||
| msg143265 - (view) | Author: trevor (trevor) | Date: 2011-08-31 16:00 | |
[eric.araujo] > Do you see that when running a test, a command or some other code? when attempting to install a self-created package. tools from 3.3a ~72060:1696e2789d91 |
|||
| msg143282 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-08-31 21:53 | |
> I don’t see why. All text is bytes, so we can do all I/O in bytes > when writing resources and avoid special-casing. I was only commenting on how the bug might have come about. > A link to a specific changeset or file would be great. https://bitbucket.org/vinay.sajip/pythonv/changeset/3e3a07a94f69#chg-Lib/packaging/command/install_distinfo.py |
|||
| msg143321 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-09-01 12:35 | |
@Éric: I also noticed that your latest comment "All text is bytes, so we can do all I/O in bytes when writing resources and avoid special-casing." contradicts your earlier position: "I agree about the nonsense of opening in binary mode." If using text mode, we might lose any specific encoding used for the source script unless we do encoding-detection as the interpreter does, and preserve that across the change-shebang-and-save. Using binary does avoid this, and should be doable with simpler code, so perhaps it is preferable - as long as we do it correctly so as to avoid the TypeError :-) |
|||
| msg143465 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-09-03 16:11 | |
> I also noticed that your latest comment > "All text is bytes, so we can do all I/O in bytes when writing > resources and avoid special-casing." > contradicts your earlier position: > "I agree about the nonsense of opening in binary mode." It does :) My earlier position was wrong. |
|||
| msg144015 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-09-14 08:31 | |
I looked at this bug again - I was getting a little confused about it ;-) The problem is happening not when writing out a resource, but the RESOURCES file listing the resources installed. This is a text file, of course, so my suggested fix of using open(resources_path, 'w', encoding='utf-8') would be reasonable. |
|||
| msg145237 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-10-09 08:07 | |
I’d like to fix this. How can I reproduce the bug? |
|||
| msg145247 - (view) | Author: Vinay Sajip (vinay.sajip) * |
Date: 2011-10-09 09:21 | |
I got the problem when installing a package with resources using pysetup3. Here's the relevant part of the console session: (venv) vinay@eta-natty:~/projects$ pysetup3 install nemo Installing from source directory: /home/vinay/projects/nemo running install_dist running build running build_py running build_scripts running install_lib creating /tmp/venv/lib/python3.3/site-packages creating /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/hook_loader.py to hook_loader.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/user_scripts.py to user_scripts.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/__init__.py to __init__.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/nemo.py to nemo.pyc running install_scripts changing mode of /tmp/venv/bin/nemo to 755 running pre_hook hooks.pre_install_data for command install_data running install_data running install_distinfo creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/METADATA creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/INSTALLER creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/REQUESTED creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/RESOURCES Traceback (most recent call last): File "/tmp/venv/bin/pysetup3", line 6, in <module> rc = packaging.run.main() # None interpreted as 0 File "/usr/local/lib/python3.3/packaging/run.py", line 653, in main return dispatcher() File "/usr/local/lib/python3.3/packaging/run.py", line 642, in __call__ return func(self, self.args) File "/usr/local/lib/python3.3/packaging/run.py", line 91, in wrapper return f(*args, **kwargs) File "/usr/local/lib/python3.3/packaging/run.py", line 164, in _install return not install_local_project(target) File "/usr/local/lib/python3.3/packaging/install.py", line 122, in install_local_project return _run_install_from_dir(path) File "/usr/local/lib/python3.3/packaging/install.py", line 160, in _run_install_from_dir func(source_dir) File "/usr/local/lib/python3.3/packaging/install.py", line 87, in _run_packaging_install dist.run_command('install_dist') File "/usr/local/lib/python3.3/packaging/dist.py", line 709, in run_command cmd_obj.run() File "/usr/local/lib/python3.3/packaging/command/install_dist.py", line 508, in run self.run_command(cmd_name) File "/usr/local/lib/python3.3/packaging/command/cmd.py", line 330, in run_command self.distribution.run_command(command) File "/usr/local/lib/python3.3/packaging/dist.py", line 709, in run_command cmd_obj.run() File "/usr/local/lib/python3.3/packaging/command/install_distinfo.py", line 113, in run writer.writerow(row) TypeError: 'str' does not support the buffer interface This is the relevant code in install_distinfo.run(): resources_path = os.path.join(self.distinfo_dir, 'RESOURCES') logger.info('creating %s', resources_path) if not self.dry_run: #with open(resources_path, 'w', encoding='utf-8') as f: with open(resources_path, 'wb') as f: writer = csv.writer(f, delimiter=',', lineterminator='\n', quotechar='"') for row in install_data.get_resources_out(): writer.writerow(row) If I substitute the commented out line above which replaces 'wb' with 'w' and encoding, I get this result: (venv) vinay@eta-natty:~/projects$ pysetup3 install nemo Installing from source directory: /home/vinay/projects/nemo running install_dist running build running build_py running build_scripts running install_lib byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/hook_loader.py to hook_loader.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/user_scripts.py to user_scripts.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/virtualenvwrapper/__init__.py to __init__.pyc byte-compiling /tmp/venv/lib/python3.3/site-packages/nemo.py to nemo.pyc running install_scripts changing mode of /tmp/venv/bin/nemo to 755 running pre_hook hooks.pre_install_data for command install_data running install_data running install_distinfo creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/METADATA creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/INSTALLER creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/REQUESTED creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/RESOURCES creating /tmp/venv/lib/python3.3/site-packages/nemo-0.1.dist-info/RECORD |
|||
| msg145460 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-10-13 15:43 | |
I started to look at this a few days ago and found out that there are no tests at all for writing RESOURCES. I need to look again at the documentation and code and add many tests. |
|||
| msg145515 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-10-14 13:41 | |
I have a test. |
|||
| msg145527 - (view) | Author: Roundup Robot (python-dev)  |
Date: 2011-10-14 15:06 | |
New changeset 1f3459b08298 by Éric Araujo in branch 'default': Fix writing of the RESOURCES file by packaging (#12386) http://hg.python.org/cpython/rev/1f3459b08298 |
|||
| msg145528 - (view) | Author: Éric Araujo (eric.araujo) * |
Date: 2011-10-14 15:10 | |
I’ve reproduced the bug and fixed it by opening in text mode, following all other calls to open in the same file. This should now work for ASCII paths and non-ASCII paths that use the same encoding as open’s default, but it’ll probably break with non-ASCII paths under a C locale, so I’ve opened #13178. |
|||
| msg147598 - (view) | Author: Roundup Robot (python-dev) |
Date: 2011-11-14 14:24 | |
New changeset cb49bc384957 by Éric Araujo in branch 'default': Fix writing of the RESOURCES file (#12386). http://hg.python.org/distutils2/rev/cb49bc384957 New changeset 2d469ccfe30e by Éric Araujo in branch 'python3': Merge fixes for #13170 and #12386 and other misc. changes from default http://hg.python.org/distutils2/rev/2d469ccfe30e |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022-04-11 14:57:18 | admin | set | github: 56595 |
| 2011-11-14 14:24:20 | python-dev | set | messages: + msg147598 |
| 2011-10-14 15:10:44 | eric.araujo | set | status: open -> closed nosy: + paul.moore messages: + msg145528 resolution: fixed stage: needs patch -> resolved |
| 2011-10-14 15:06:10 | python-dev | set | nosy:
+ python-dev messages: + msg145527 |
| 2011-10-14 13:41:33 | eric.araujo | set | messages: + msg145515 |
| 2011-10-13 15:43:29 | eric.araujo | set | messages: + msg145460 |
| 2011-10-13 15:42:06 | eric.araujo | link | issue13162 superseder |
| 2011-10-09 09:21:19 | vinay.sajip | set | messages: + msg145247 |
| 2011-10-09 08:07:57 | eric.araujo | set | messages: + msg145237 |
| 2011-09-14 08:31:18 | vinay.sajip | set | messages: + msg144015 |
| 2011-09-03 16:11:56 | eric.araujo | set | messages: + msg143465 |
| 2011-09-01 12:35:13 | vinay.sajip | set | messages: + msg143321 |
| 2011-08-31 21:53:12 | vinay.sajip | set | messages: + msg143282 |
| 2011-08-31 16:00:10 | trevor | set | messages: + msg143265 |
| 2011-08-31 15:52:41 | eric.araujo | set | messages: + msg143264 |
| 2011-08-25 18:42:45 | vinay.sajip | set | messages: + msg142986 |
| 2011-08-25 18:17:50 | trevor | set | messages: + msg142984 |
| 2011-08-25 17:30:11 | vinay.sajip | set | messages: + msg142983 |
| 2011-08-25 16:20:22 | eric.araujo | set | messages: + msg142977 |
| 2011-08-24 09:53:59 | trevor | set | nosy:
+ trevor |
| 2011-06-24 11:52:57 | vinay.sajip | set | messages: + msg138921 |
| 2011-06-24 11:24:29 | eric.araujo | set | assignee: tarek -> eric.araujo messages: + msg138913 stage: needs patch |
| 2011-06-23 07:51:11 | vinay.sajip | set | type: behavior |
| 2011-06-22 11:22:24 | vinay.sajip | create | |