To get theano compiled successfully

Issue resolved in pull request : https://github.com/python/cpython/pull/3336

Why not fix theano instead? Implicit stringification of an argument can hide other bugs.

The resolution field should be changed only when close an issue.

You did not follow the request I made on the PR to explain in this issue you opened what is failing and why.  Unless you convince us that this is actually a bug in python, we will close the issue and PR.

Hi David,

A led, I may try to reproduce this error, but it may take a while.

Thanks

On Wed, Sep 6, 2017 at 11:31 AM R. David Murray <report@bugs.python.org>
wrote:

>
> R. David Murray added the comment:
>
> You did not follow the request I made on the PR to explain in this issue
> you opened what is failing and why.  Unless you convince us that this is
> actually a bug in python, we will close the issue and PR.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue31361>
> _______________________________________
>
-- 
*____**____**____**____**____**____**____**____**____**____**____**____*
*____**____**____**____**____**____*
* Contact Info                                            *
 *US Mobile: 1-408-609-7628                   *

Em
ail: raliclo@gmail.com
Skype : ralic_lo
*____**____**____**____**____**____**____**____**____**____**____**____*
*____**____**____**____**____**____*

Bug encountered while installing multiple packages : 
1. keras
2. theano

Error Information : 
 ~/work/bottles @ dojos-MacBook-Pro (dojo)  
|=> pip3 install keras
Requirement already satisfied: keras in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.6/site-packages (from keras)
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 666, in _prepare_file
    check_dist_requires_python(dist)
  File "/usr/local/lib/python3.6/site-packages/pip/utils/packaging.py", line 48, in check_dist_requires_python
    feed_parser.feed(metadata)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/feedparser.py", line 175, in feed
    self._input.push(data)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/feedparser.py", line 103, in push
    self._partial.write(data)
TypeError: string argument expected, got 'NoneType'
________________________________________________________________________________
 ~/work/bottles @ dojos-MacBook-Pro (dojo)  
|=> pip3 install theano
Collecting theano
  Downloading Theano-0.9.0.tar.gz (3.1MB)
    100% |████████████████████████████████| 3.1MB 353kB/s 
Requirement already satisfied: numpy>=1.9.1 in /usr/local/lib/python3.6/site-packages (from theano)
Requirement already satisfied: scipy>=0.14 in /usr/local/lib/python3.6/site-packages (from theano)
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.6/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/usr/local/lib/python3.6/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 666, in _prepare_file
    check_dist_requires_python(dist)
  File "/usr/local/lib/python3.6/site-packages/pip/utils/packaging.py", line 48, in check_dist_requires_python
    feed_parser.feed(metadata)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/feedparser.py", line 175, in feed
    self._input.push(data)
  File "/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/email/feedparser.py", line 103, in push
    self._partial.write(data)
TypeError: string argument expected, got 'NoneType'

This looks to me as a bug in keras and theano. And may be in pip. Please file bug reports on corresponding bug trackers.

After patching, I've tested installed packages listed in the pip3.update. 
The default c compiler used is clang.  Few of them are failed, listed as following, after change the c compiler to be gcc-7. (Twisted av cymem cytoolz dulwich gnureadline pycrypto pydsm pycurl regex reportlab setproctitle). PyAudio and mercurial needs additional support to get them installed. 
I hope this demonstrate the reliance of this change.

  Running setup.py install for PyAudio ... error  
  Running setup.py install for Twisted ... error
  Running setup.py install for av ... error
  Running setup.py install for cymem ... error
  Running setup.py install for cytoolz ... error
  Running setup.py install for dulwich ... error
  Running setup.py install for gnureadline ... error
  Running setup.py install for mercurial ... error
  Running setup.py install for pycrypto ... error
  Running setup.py install for pycurl ... error
  Running setup.py install for pydsm ... error
  Running setup.py install for regex ... error
  Running setup.py install for reportlab ... error
  Running setup.py install for setproctitle ... error

Correct some type :

Few of them are failed, listed as following, after change the c compiler to
be gcc-7, they can be fixed.
(Twisted av cymem cytoolz dulwich gnureadline pycrypto pydsm pycurl regex
reportlab setproctitle).

*____**____**____**____**____**____**____**____**____**____**____**____*
*____**____**____**____**____**____*
* Contact Info                                            *
 *US Mobile: 1-408-609-7628                   *

Em
ail: raliclo@gmail.com
Skype : ralic_lo
*____**____**____**____**____**____**____**____**____**____**____**____*
*____**____**____**____**____**____*

On Fri, Sep 8, 2017 at 2:42 PM, Wei-Shun Lo <report@bugs.python.org> wrote:

>
> Wei-Shun Lo added the comment:
>
> After patching, I've tested installed packages listed in the pip3.update.
> The default c compiler used is clang.  Few of them are failed, listed as
> following, after change the c compiler to be gcc-7. (Twisted av cymem
> cytoolz dulwich gnureadline pycrypto pydsm pycurl regex reportlab
> setproctitle). PyAudio and mercurial needs additional support to get them
> installed.
> I hope this demonstrate the reliance of this change.
>
>   Running setup.py install for PyAudio ... error
>   Running setup.py install for Twisted ... error
>   Running setup.py install for av ... error
>   Running setup.py install for cymem ... error
>   Running setup.py install for cytoolz ... error
>   Running setup.py install for dulwich ... error
>   Running setup.py install for gnureadline ... error
>   Running setup.py install for mercurial ... error
>   Running setup.py install for pycrypto ... error
>   Running setup.py install for pycurl ... error
>   Running setup.py install for pydsm ... error
>   Running setup.py install for regex ... error
>   Running setup.py install for reportlab ... error
>   Running setup.py install for setproctitle ... error
>
> ----------
> Added file: https://bugs.python.org/file47128/pip3.update
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue31361>
> _______________________________________
>

Can you figure out what the input to feedparser is in those cases?  If pip or whatever is feeding in a None object, then that's not a bug in feedparser.  However, if the input is a string (or bytes via the bytes interface) and this is heppening, then there's probably something wrong, but we'll need to see the actual input to determine what.

Hi David,

From the error message, the source of the None object was from pip, and it
is grabbing data from the metadata of the package.

  File "/usr/local/lib/python3.6/site-packages/pip/utils/packaging.py",
line 48, in check_dist_requires_python
-- By adding 'print(metadata)' on line 46, I have the metadata of theano as
attached.

So another fix maybe adding str() in the packaging.py in pip, I tested it
too and worked fine. It appears to me as of now, getting pip to send the
metadata in string seems to be better.

As for the NoneType object error, somehow it no longer pops up, it might be
related to the original metadata, which may have not define 'METADATA' and
'PKG-INFO'.

You may close this pull request and issue, I believe that's not necessary
for now.

Thanks !

[image: Inline image 2]
​

*​Theano's metatdata*​
​
​ ​

Metadata-Version: 1.1

Name: Theano

Version: 0.9.0

Summary: Optimizing compiler for evaluating mathematical expressions on
CPUs and GPUs.

Home-page: http://deeplearning.net/software/theano/

Author: LISA laboratory, University of Montreal

Author-email: theano-dev@googlegroups.com

License: BSD

Description-Content-Type: UNKNOWN

Description: Theano is a Python library that allows you to define,
optimize, and efficiently evaluate mathematical expressions involving
multi-dimensional arrays. It is built on top of NumPy_. Theano features:

         * **tight integration with NumPy:** a similar interface to
NumPy's. numpy.ndarrays are also used internally in Theano-compiled
functions.

         * **transparent use of a GPU:** perform data-intensive
computations up to 140x faster than on a CPU (support for float32 only).

         * **efficient symbolic differentiation:** Theano can compute
derivatives for functions of one or many inputs.

         * **speed and stability optimizations:** avoid nasty bugs when
computing expressions such as log(1 + exp(x)) for large values of x.

         * **dynamic C code generation:** evaluate expressions faster.

         * **extensive unit-testing and self-verification:** includes tools
for detecting and diagnosing bugs and/or potential problems.

        Theano has been powering large-scale computationally intensive
scientific

        research since 2007, but it is also approachable enough to be used
in the

        classroom (IFT6266 at the University of Montreal).

        .. _NumPy: http://numpy.scipy.org/

        =============

        Release Notes

        =============

        Theano 0.9.0 (20th of March, 2017)

        ==================================

        This is a final release of Theano, version ``0.9.0``, with a lot of

        new features, interface changes, improvements and bug fixes.

        We recommend that everybody update to this version.

        Highlights (since 0.8.0):

         - Better Python 3.5 support

         - Better numpy 1.12 support

         - Conda packages for Mac, Linux and Windows

         - Support newer Mac and Windows versions

         - More Windows integration:

           - Theano scripts (``theano-cache`` and ``theano-nose``) now
works on Windows

           - Better support for Windows end-lines into C codes

           - Support for space in paths on Windows

         - Scan improvements:

           - More scan optimizations, with faster compilation and gradient
computation

           - Support for checkpoint in scan (trade off between speed and
memory usage, useful for long sequences)

           - Fixed broadcast checking in scan

         - Graphs improvements:

           - More numerical stability by default for some graphs

           - Better handling of corner cases for theano functions and graph
optimizations

           - More graph optimizations with faster compilation and execution

           - smaller and more readable graph

         - New GPU back-end:

           - Removed warp-synchronous programming to get good results with
newer CUDA drivers

           - More pooling support on GPU when cuDNN isn't available

           - Full support of ignore_border option for pooling

           - Inplace storage for shared variables

           - float16 storage

           - Using PCI bus ID of graphic cards for a better mapping between
theano device number and nvidia-smi number

           - Fixed offset error in ``GpuIncSubtensor``

         - Less C code compilation

         - Added support for bool dtype

         - Updated and more complete documentation

         - Bug fixes related to merge optimizer and shape inference

         - Lot of other bug fixes, crashes fixes and warning improvements

        A total of 123 people contributed to this release since 0.8.0, see
list below.

        Interface changes:

         - Merged ``CumsumOp/CumprodOp`` into ``CumOp``

         - In MRG module:

           - Replaced method ``multinomial_wo_replacement()`` with new
method ``choice()``

           - Random generator now tries to infer the broadcast pattern of
its output

         - New pooling interface

         - Pooling parameters can change at run time

         - Moved ``softsign`` out of sandbox to
``theano.tensor.nnet.softsign``

         - Using floatX dtype when converting empty list/tuple

         - ``Roll`` make the shift be modulo the size of the axis we roll on

         - ``round()`` default to the same as NumPy: half_to_even

        Convolution updates:

         - Support of full and half modes for 2D and 3D convolutions
including in ``conv3d2d``

         - Allowed pooling of empty batch

         - Implement ``conv2d_transpose`` convenience function

         - Multi-cores convolution and pooling on CPU

         - New abstract 3d convolution interface similar to the 2d
convolution interface

         - Dilated convolution

        GPU:

         - cuDNN: support versoin 5.1 and wrap batch normalization (2d and
3d) and RNN functions

         - Multiple-GPU, synchrone update (via platoon, use NCCL)

         - Gemv(matrix-vector product) speed up for special shape

         - cublas gemv workaround when we reduce on an axis with a
dimensions size of 0

         - Warn user that some cuDNN algorithms may produce unexpected
results in certain environments

           for convolution backward filter operations

         - ``GPUMultinomialFromUniform`` op now supports multiple dtypes

         - Support for ``MaxAndArgMax`` for some axis combination

         - Support for solve (using cusolver), erfinv and erfcinv

         - Implemented ``GpuAdvancedSubtensor``

        New features:

         - ``OpFromGraph`` now allows gradient overriding for every input

         - Added Abstract Ops for batch normalization that use cuDNN when
available and pure Theano CPU/GPU alternatives otherwise

         - Added gradient of solve, tensorinv (CPU), tensorsolve (CPU),
searchsorted (CPU), DownsampleFactorMaxGradGrad (CPU)

         - Added Multinomial Without Replacement

         - Allowed partial evaluation of compiled function

         - More Rop support

         - Indexing support ellipsis: ``a[..., 3]```, ``a[1,...,3]``

         - Added ``theano.tensor.{tensor5,dtensor5, ...}``

         - compiledir_format support device

         - Added New Theano flag ``conv.assert_shape`` to check
user-provided shapes at runtime (for debugging)

         - Added new Theano flag ``cmodule.age_thresh_use``

         - Added new Theano flag ``cuda.enabled``

         - Added new Theano flag ``nvcc.cudafe`` to enable faster
compilation and import with old CUDA back-end

         - Added new Theano flag ``print_global_stats`` to print some
global statistics (time spent) at the end

         - Added new Theano flag ``profiling.ignore_first_call``, useful to
profile the new gpu back-end

         - remove ProfileMode (use Theano flag ``profile=True`` instead)

        Others:

         - Split op now has C code for CPU and GPU

         - ``theano-cache list`` now includes compilation times

         - Speed up argmax only on GPU (without also needing the max)

         - More stack trace in error messages

         - Speed up cholesky grad

         - ``log(sum(exp(...)))`` now get stability optimized

        Other more detailed changes:

         - Added Jenkins (gpu tests run on pull requests in addition to
daily buildbot)

         - Removed old benchmark directory and other old files not used
anymore

         - Use of 64-bit indexing in sparse ops to allow matrix with more
then 2\ :sup:`31`\ -1 elements

         - Allowed more then one output to be an destructive inplace

         - More support of negative axis

         - Added the keepdims parameter to the norm function

         - Make scan gradient more deterministic

        Commiters since 0.8.0:

         - Frederic Bastien

         - Arnaud Bergeron

         - Pascal Lamblin

         - Steven Bocco

         - Ramana Subramanyam

         - Simon Lefrancois

         - Gijs van Tulder

         - Benjamin Scellier

         - khaotik

         - Chiheb Trabelsi

         - Chinnadhurai Sankar

         - Cesar Laurent

         - Reyhane Askari

         - Mohammad Pezeshki

         - Alexander Matyasko

         - Alexandre de Brebisson

         - Mathieu Germain

         - Nan Rosemary Ke

         - Pierre Luc Carrier

         - Olivier Mastropietro

         - Thomas George

         - Saizheng Zhang

         - Iulian Vlad Serban

         - Francesco Visin

         - Caglar

         - Faruk Ahmed

         - Harm de Vries

         - Samira Shabanian

         - Vincent Dumoulin

         - Nicolas Ballas

         - Jakub Sygnowski

         - Jan Schlüter

         - Samira Ebrahimi Kahou

         - Mikhail Korobov

         - Fei Wang

         - Kv Manohar

         - Jesse Livezey

         - Kelvin Xu

         - Matt Graham

         - Ruslana Makovetsky

         - Sina Honari

         - Bryn Keller

         - Ciyong Chen

         - Vitaliy Kurlin

         - Zhouhan LIN

         - Gokula Krishnan

         - Kumar Krishna Agrawal

         - Ozan Çağlayan

         - Vincent Michalski

         - affanv14

         - Amjad Almahairi

         - Ray Donnelly

         - Tim Cooijmans

         - happygds

         - mockingjamie

         - Christos Tsirigotis

         - Florian Bordes

         - Ilya Kulikov

         - RadhikaG

         - Taesup (TS) Kim

         - Ying Zhang

         - Anton Chechetka

         - Karthik Karanth

         - Kirill Bobyrev

         - Rebecca N. Palmer

         - Yang Zhang

         - Yaroslav Ganin

         - Jonas Degrave

         - Liwei Cai

         - Lucas Beyer

         - Michael Harradon

         - Morgan Stuart

         - Tim Gasper

         - Xavier Bouthillier

         - p

         - texot

         - Andrés Gottlieb

         - Ben Poole

         - Bhavishya Pohani

         - Carl Thomé

         - David Bau

         - Dimitar Dimitrov

         - Evelyn Mitchell

         - Fei Zhan

         - Fuchai

         - Fábio Perez

         - Gennadiy Tupitsin

         - Gilles Louppe

         - Greg Ciccarelli

         - He

         - Huan Zhang

         - Kaixhin

         - Kevin Keraudren

         - Maltimore

         - Marc-Alexandre Cote

         - Marco

         - Marius F. Killinger

         - Martin Drawitsch

         - Maxim Kochurov

         - Micah Bojrab

         - Neil

         - Nizar Assaf

         - Rithesh Kumar

         - Rizky Luthfianto

         - Robin Millette

         - Roman Ring

         - Sander Dieleman

         - Sebastin Santy

         - Shawn Tan

         - Wazeer Zulfikar

         - Wojciech Głogowski

         - Yann N. Dauphin

         - gw0 [http://gw.tnode.com/]

         - hexahedria

         - hsintone

         - jakirkham

         - joncrall

         - root

         - superantichrist

         - tillahoffmann

         - valtron

         - wazeerzulfikar

         - you-n-g

Keywords: theano math numerical symbolic blas numpy gpu autodiff
differentiation

Platform: Windows

Platform: Linux

Platform: Solaris

Platform: Mac OS-X

Platform: Unix

Classifier: Development Status :: 4 - Beta

Classifier: Intended Audience :: Education

Classifier: Intended Audience :: Science/Research

Classifier: Intended Audience :: Developers

Classifier: License :: OSI Approved :: BSD License

Classifier: Programming Language :: Python

Classifier: Topic :: Software Development :: Code Generators

Classifier: Topic :: Software Development :: Compilers

Classifier: Topic :: Scientific/Engineering :: Mathematics

Classifier: Operating System :: Microsoft :: Windows

Classifier: Operating System :: POSIX

Classifier: Operating System :: Unix

Classifier: Operating System :: MacOS

Classifier: Programming Language :: Python :: 2

Classifier: Programming Language :: Python :: 2.7

Classifier: Programming Language :: Python :: 3

Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
​​

On Sat, Sep 9, 2017 at 10:04 AM, R. David Murray <report@bugs.python.org>
wrote:

>
> R. David Murray added the comment:
>
> Can you figure out what the input to feedparser is in those cases?  If pip
> or whatever is feeding in a None object, then that's not a bug in
> feedparser.  However, if the input is a string (or bytes via the bytes
> interface) and this is heppening, then there's probably something wrong,
> but we'll need to see the actual input to determine what.
>
> ----------
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue31361>
> _______________________________________
>
​