Issue35268
This issue tracker has been migrated to GitHub,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2018-11-17 11:52 by Cezary.Wagner, last changed 2022-04-11 14:59 by admin. This issue is now closed.
Messages (8) | |||
---|---|---|---|
msg330028 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-11-17 11:52 | |
I have some problems with asyncio on Windows - it block where it should go. Documentation shows that it should work: https://docs.python.org/3/library/asyncio-stream.html -> StreamReader should return something. See 1st example: import asyncio import sys async def run_stockfish(): STOCKFISH_PATH = r'C:\root\chess\stockfish\stockfish 9\stockfish_9_x64_bmi2.exe' stockfish = await asyncio.subprocess.create_subprocess_exec( STOCKFISH_PATH, stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) stockfish.stdin.write('uci'.encode()) while True: # BUG? - blocks at this line - no output line = await stockfish.stdout.read() print(line) await stockfish.wait() if sys.platform == "win32": asyncio.set_event_loop_policy( asyncio.WindowsProactorEventLoopPolicy()) asyncio.run(run_stockfish(), debug=True) 1st output (nothing): Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32 runfile('C:/Users/Cezary Wagner/PycharmProjects/stockfish-proxy/sandbox/async_proxy/s01_async_stockfish.py', wdir='C:/Users/Cezary Wagner/PycharmProjects/stockfish-proxy/sandbox/async_proxy') 2nd example: import asyncio import sys async def run_stockfish(): STOCKFISH_PATH = r'C:\root\chess\stockfish\stockfish 9\stockfish_9_x64_bmi2.exe' stockfish = await asyncio.subprocess.create_subprocess_exec( STOCKFISH_PATH, stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) stockfish.stdin.write('uci'.encode()) while True: # BUG? - blocks at this line line = await stockfish.stdout.readline() print(line) await stockfish.wait() if sys.platform == "win32": asyncio.set_event_loop_policy( asyncio.WindowsProactorEventLoopPolicy()) asyncio.run(run_stockfish(), debug=True) 2nd output is little better (first line is read): Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32 runfile('C:/Users/Cezary Wagner/PycharmProjects/stockfish-proxy/sandbox/async_proxy/s01_async_stockfish.py', wdir='C:/Users/Cezary Wagner/PycharmProjects/stockfish-proxy/sandbox/async_proxy') b'Stockfish 9 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott\r\n' What should be done in both case (or maybe done): Stockfish 9 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott uci id name Stockfish 9 64 BMI2 id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott option name Debug Log File type string default option name Contempt type spin default 20 min -100 max 100 option name Threads type spin default 1 min 1 max 512 option name Hash type spin default 16 min 1 max 131072 option name Clear Hash type button option name Ponder type check default false option name MultiPV type spin default 1 min 1 max 500 option name Skill Level type spin default 20 min 0 max 20 option name Move Overhead type spin default 30 min 0 max 5000 option name Minimum Thinking Time type spin default 20 min 0 max 5000 option name Slow Mover type spin default 89 min 10 max 1000 option name nodestime type spin default 0 min 0 max 10000 option name UCI_Chess960 type check default false option name SyzygyPath type string default <empty> option name SyzygyProbeDepth type spin default 1 min 1 max 100 option name Syzygy50MoveRule type check default true option name SyzygyProbeLimit type spin default 6 min 0 max 6 uciok |
|||
msg330029 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-11-17 11:56 | |
It looks like StremReader.read() not work and StremReader.readline() in Windows or I am not understand documentation. https://docs.python.org/3/library/asyncio-stream.html coroutine read(n=-1) Read up to n bytes. If n is not provided, or set to -1, read until EOF and return all read bytes. If EOF was received and the internal buffer is empty, return an empty bytes object. coroutine readline() Read one line, where “line” is a sequence of bytes ending with \n. If EOF is received and \n was not found, the method returns partially read data. If EOF is received and the internal buffer is empty, return an empty bytes object. |
|||
msg330090 - (view) | Author: Steve Dower (steve.dower) * | Date: 2018-11-19 11:39 | |
Are you awaiting those calls (and the write())? I can't tell at first glance whether you need to, but it seems like missing await there would cause your problem. |
|||
msg331895 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-12-15 11:02 | |
Stockfish works like that: 1. Run command line and listen stdout. 2. Send some stdin and listen stdin. 3. stdout is asynchronous and continous (there no single output, line is generated every some time). I want to send some stdin than listen stdout response from Stockfish. I tested that first execute of it block all: line = await stockfish.stdout.read() One line is read and application becomes dead. Should show much more lines but it stops in Windows 10. |
|||
msg331896 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-12-15 11:09 | |
See new code example (little changed): It block after first stdout: import asyncio import sys async def run_stockfish(): STOCKFISH_PATH = r'C:\root\chess\stockfish\stockfish 10\stockfish_10_x64_bmi2.exe' stockfish = await asyncio.subprocess.create_subprocess_exec( STOCKFISH_PATH, stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) stockfish.stdin.write('uci'.encode()) while not stockfish.stdout.at_eof(): # BUG? - blocks at this line line = await stockfish.stdout.readline() print(line.decode()) await stockfish.wait() if sys.platform == "win32": asyncio.set_event_loop_policy( asyncio.WindowsProactorEventLoopPolicy()) asyncio.run(run_stockfish(), debug=True) Output: Python 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 14:57:15) [MSC v.1915 64 bit (AMD64)] on win32 runfile('C:/Users/Cezary Wagner/PycharmProjects/cw_chess_uci/sandbox/async_proxy/s01_async_stockfish.py', wdir='C:/Users/Cezary Wagner/PycharmProjects/cw_chess_uci/sandbox/async_proxy') Stockfish 10 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott Valid output is: C:\Users\Cezary Wagner>"C:\root\chess\stockfish\stockfish 10\stockfish_10_x64_bmi2.exe" Stockfish 10 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott uci id name Stockfish 10 64 BMI2 id author T. Romstad, M. Costalba, J. Kiiski, G. Linscott option name Debug Log File type string default option name Contempt type spin default 24 min -100 max 100 option name Analysis Contempt type combo default Both var Off var White var Black var Both option name Threads type spin default 1 min 1 max 512 option name Hash type spin default 16 min 1 max 131072 option name Clear Hash type button option name Ponder type check default false option name MultiPV type spin default 1 min 1 max 500 option name Skill Level type spin default 20 min 0 max 20 option name Move Overhead type spin default 30 min 0 max 5000 option name Minimum Thinking Time type spin default 20 min 0 max 5000 option name Slow Mover type spin default 84 min 10 max 1000 option name nodestime type spin default 0 min 0 max 10000 option name UCI_Chess960 type check default false option name UCI_AnalyseMode type check default false option name SyzygyPath type string default <empty> option name SyzygyProbeDepth type spin default 1 min 1 max 100 option name Syzygy50MoveRule type check default true option name SyzygyProbeLimit type spin default 7 min 0 max 7 uciok |
|||
msg331897 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-12-15 11:12 | |
Another code try - this time I am using task: import asyncio import sys async def process_line_reader(process, on_line=None, on_eof=None): while not process.stdout.at_eof(): # BUG? after first line it becomes dead line = await process.stdout.readline() if on_line is not None: on_line(line.decode()) if on_eof is not None: on_eof() print('eof') async def run_stockfish(): STOCKFISH_PATH = r'C:\root\chess\stockfish\stockfish 10\stockfish_10_x64_bmi2.exe' stockfish = await asyncio.subprocess.create_subprocess_exec( STOCKFISH_PATH, stdin=asyncio.subprocess.PIPE, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) stockfish.stdin.write('uci'.encode()) task = asyncio.create_task(process_line_reader( process=stockfish, on_line=lambda line: print(f'{line}') )) # await task await stockfish.wait() if sys.platform == "win32": asyncio.set_event_loop_policy( asyncio.WindowsProactorEventLoopPolicy()) asyncio.run(run_stockfish(), debug=True) print('done') All is blocked after first line (no print of eof or done): C:\root\Python37-64\python.exe "C:/Users/Cezary Wagner/PycharmProjects/cw_chess_uci/sandbox/async_proxy/s02_async_stockfish.py" Stockfish 10 64 BMI2 by T. Romstad, M. Costalba, J. Kiiski, G. Linscott |
|||
msg331898 - (view) | Author: Andrew Svetlov (asvetlov) * | Date: 2018-12-15 12:00 | |
Sorry, the bug tracker is for working on Python bugs, not on bugs of your programs written with Python. The behavior of subprocesses running is tested intensively, scenarios like you described works well. I don't know sockfish protocol but highly likely the problem is in communicating with the program. For example, are you sure that sending `uci` without endline is enough to start processing the command? Tools like ProcMon https://docs.microsoft.com/en-us/sysinternals/downloads/procmon or WinDbg can help with understanding what data is actually received by a process and what is sent back. |
|||
msg331902 - (view) | Author: Cezary Wagner (Cezary.Wagner) | Date: 2018-12-15 14:32 | |
It just tested it today and I found in parallel same solution as you suggested. Sometimes it happen even your skilled programmer - at least one time in year :) I was just blind. Let's close it since it is invalid report. I just skipped new line at end of 'uci' should be 'uci\n'. Now I am unblocked to test async pipes more and write very nice program. It looks very promising. Thank you for attention. I am sorry for bad report. |
History | |||
---|---|---|---|
Date | User | Action | Args |
2022-04-11 14:59:08 | admin | set | github: 79449 |
2018-12-15 14:36:09 | cheryl.sabella | set | status: open -> closed resolution: not a bug stage: resolved |
2018-12-15 14:32:51 | Cezary.Wagner | set | messages: + msg331902 |
2018-12-15 12:00:20 | asvetlov | set | messages: + msg331898 |
2018-12-15 11:12:37 | Cezary.Wagner | set | messages: + msg331897 |
2018-12-15 11:09:09 | Cezary.Wagner | set | messages: + msg331896 |
2018-12-15 11:02:34 | Cezary.Wagner | set | messages:
+ msg331895 title: Windows asyncio reading continously stdin and stdout Stockfish -> Windows 10 asyncio reading continously stdin and stdout Stockfish |
2018-11-19 11:39:11 | steve.dower | set | messages: + msg330090 |
2018-11-17 11:56:40 | Cezary.Wagner | set | messages: + msg330029 |
2018-11-17 11:52:53 | Cezary.Wagner | create |