classification
Title: 'str' object has no attribute 'more' [/usr/lib/python3.2/asynchat.py|initiate_send|245]
Type: behavior Stage:
Components: Library (Lib) Versions: Python 3.5, Python 3.4
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: BreamoreBoy, Gryllida, giampaolo.rodola, haypo, jcao219, math_foo, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2011-07-10 00:04 by Gryllida, last changed 2014-07-07 22:07 by haypo. This issue is now closed.

Files
File name Uploaded Description Edit
asynchat_example.py math_foo, 2014-04-16 14:19 example script demonstrating issue 12523
issue12523.patch math_foo, 2014-04-16 16:08 Add logging statement explaining probable cause of error. review
issue12523_r2.patch math_foo, 2014-04-16 19:18 Add error catching to async_chat.push method review
Messages (12)
msg140073 - (view) Author: Gryllida (Gryllida) Date: 2011-07-10 00:04
Asynchat push() function has a bug which prevents it from functioning.

This code worked fine with Python 2.

---------------------------------------------------------------
# https://github.com/jstoker/BasicBot
import asynchat,asyncore,socket
class asynchat_bot(asynchat.async_chat):
def __init__(self, host, port):
asynchat.async_chat.__init__(self)
self.create_socket(socket.AF_INET,socket.SOCK_STREAM)
self.set_terminator('\r\n')
self.data=''
self.remote=(host,port)
self.connect(self.remote)

def handle_connect(self):
self.push('USER BasicBot 8 %s :BasicBot! http://github.com/jstoker/BasicBot\r\nNICK testbot\r\n' % self.remote[0])

def get_data(self):
r=self.data
self.data=''
return r
def collect_incoming_data(self, data):
self.data+=data
def found_terminator(self):
data=self.get_data()
if data[:4] == 'PING':
self.push('PONG %s' % data[5:]+'\r\n')
if '001' in data:
self.push('JOIN #bots\r\n')
if '~hi' in data:
self.push('PRIVMSG #bots :hi.\r\n')
if __name__ == '__main__':
asynchat_bot('127.0.0.1',16667)
asyncore.loop()
---------------------------------------------------------------


In Python 3 however, the exception follows:


---------------------------------------------------------------
~/tests/BasicBot$ python3 asynchat_bot.py
error: uncaptured python exception, closing channel <__main__.asynchat_bot connected at 0xb70078ac> (<class 'AttributeError'>:'str' object has no attribute 'more' [/usr/lib/python3.2/asyncore.py|write|89] [/usr/lib/python3.2/asyncore.py|handle_write_event|462] [/usr/lib/python3.2/asynchat.py|handle_write|194] [/usr/lib/python3.2/asynchat.py|initiate_send|245])
~/tests/BasicBot$ python3 -V
Python 3.2
~/tests/BasicBot$
---------------------------------------------------------------

A comment from Stackoverflow on why it happens:

---------------------------------------------------------------
The error seems to be raised in /usr/lib/python3.2/asynchat.py|initiate_send|245.

def initiate_send(self):
    while self.producer_fifo and self.connected:
        first = self.producer_fifo[0]
        ...
        try:
            data = buffer(first, 0, obs)
        except TypeError:
            data = first.more() <--- here 

Seems like somebody put a string in self.producer_fifo instead of an asyncchat.simple_producer, which is the only class in async*.py with a more() method.
msg140104 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2011-07-11 09:44
Actually, the error is that in Python 3 you should use bytes objects when transmitting/receiving data over the network, not (unicode) strings.
That is, replace '\r\n' with b'\r\n', etc.

Of course, the error message should be made less obscure.
msg216471 - (view) Author: Caelyn McAulay (math_foo) Date: 2014-04-16 14:19
Here is a small script that runs fine under 2.7 but demonstrates the error when run at 3.5.

If, at all the points annotated with '#not bytes :-(', the unicode strings are replaced with bytes objects, the example then successfully runs at 3.5.
msg216485 - (view) Author: Caelyn McAulay (math_foo) Date: 2014-04-16 16:08
I was unable to locate a point in the code where we could be certain that the error was ultimately caused by trying to use (unicode) strings instead of bytes object.

The patch adds a logging statement suggesting what the trouble is when the error is (probably) encountered.

Someone with more experience than me will need to make the call about whether this is a useful addition or not.
msg216549 - (view) Author: Giampaolo Rodola' (giampaolo.rodola) * (Python committer) Date: 2014-04-16 18:42
The part of the code where that is more likely to happen is the push() method.
One might also submit a producer (push_with_producer()) erroneously returning strings on more() though; how to properly fix this one is not clear to me because of the bad asynchat design. I'd be for simply raising an exception in push() as in:

diff --git a/Lib/asynchat.py b/Lib/asynchat.py
--- a/Lib/asynchat.py
+++ b/Lib/asynchat.py
@@ -181,6 +181,8 @@
         self.close()
 
     def push (self, data):
+        if not isinstance(data, bytes):
+            raise TypeError("data must be a bytes object")
         sabs = self.ac_out_buffer_size
         if len(data) > sabs:
             for i in range(0, len(data), sabs):
msg216560 - (view) Author: Caelyn McAulay (math_foo) Date: 2014-04-16 19:18
I like that approach decidedly more than my original one. Suggested revision patch attached.

The sample script will not produce the following:

error: uncaptured python exception, closing channel <__main__.fake_asynchat 127.0.0.1:8000 at 0x7f10a85b4ae8> (<class 'TypeError'>:data must be a bytes object [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|write|91] [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|handle_write_event|460] [/home/caelyn/CPython/async_chat_error/Lib/asyncore.py|handle_connect_event|448] [asynchat_example.py|handle_connect|16] [/home/caelyn/CPython/async_chat_error/Lib/asynchat.py|push|185])

Which is, I think, is a bit more useful.
msg216561 - (view) Author: Caelyn McAulay (math_foo) Date: 2014-04-16 19:20
*will now produce.
msg222376 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-05 21:58
I've no objection to people trying to take this forward but they should be aware that asyncio is recommended for new code.
msg222388 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-07-06 01:20
> I've no objection to people trying to take this forward

We were really looking for your approval, thank you.
msg222401 - (view) Author: Mark Lawrence (BreamoreBoy) * Date: 2014-07-06 10:51
Just close this as asynchat is deprecated from 3.2.
msg222525 - (view) Author: Roundup Robot (python-dev) Date: 2014-07-07 22:01
New changeset f8c9dd2626aa by Victor Stinner in branch '3.4':
Issue #12523: asynchat.async_chat.push() now raises a TypeError if it doesn't
http://hg.python.org/cpython/rev/f8c9dd2626aa

New changeset 4b29d338cc41 by Victor Stinner in branch 'default':
(Merge 3.4) Issue #12523: asynchat.async_chat.push() now raises a TypeError if
http://hg.python.org/cpython/rev/4b29d338cc41
msg222526 - (view) Author: STINNER Victor (haypo) * (Python committer) Date: 2014-07-07 22:07
I fixed the issue. I copied the check from asyncio, _SelectorSocketTransport.write() of asyncio.selector_events for example.
History
Date User Action Args
2014-07-07 22:07:59hayposetstatus: open -> closed

nosy: + haypo
messages: + msg222526

resolution: fixed
2014-07-07 22:01:56python-devsetnosy: + python-dev
messages: + msg222525
2014-07-06 10:51:57BreamoreBoysetmessages: + msg222401
2014-07-06 01:20:58pitrousetmessages: + msg222388
2014-07-05 21:58:12BreamoreBoysetnosy: + BreamoreBoy

messages: + msg222376
versions: + Python 3.4, Python 3.5, - Python 3.2, Python 3.3
2014-04-16 19:20:04math_foosetmessages: + msg216561
2014-04-16 19:18:53math_foosetfiles: + issue12523_r2.patch

messages: + msg216560
2014-04-16 18:42:28giampaolo.rodolasetmessages: + msg216549
2014-04-16 16:08:52math_foosetfiles: + issue12523.patch
keywords: + patch
messages: + msg216485
2014-04-16 14:19:40math_foosetfiles: + asynchat_example.py
nosy: + math_foo
messages: + msg216471

2011-07-11 09:44:17pitrousetnosy: + pitrou

messages: + msg140104
versions: + Python 3.3
2011-07-10 12:51:15r.david.murraysettype: crash -> behavior
2011-07-10 12:47:23r.david.murraysetnosy: + giampaolo.rodola
components: + Library (Lib), - None
2011-07-10 00:53:09jcao219setnosy: + jcao219
2011-07-10 00:04:24Gryllidacreate