classification
Title: sys.stdout.flush and print() hanging
Type: Stage: resolved
Components: Library (Lib) Versions: Python 3.8
process
Status: closed Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: rajeevkchaurasia, serhiy.storchaka
Priority: normal Keywords:

Created on 2021-06-14 04:36 by rajeevkchaurasia, last changed 2021-06-16 06:43 by rajeevkchaurasia. This issue is now closed.

Messages (5)
msg395772 - (view) Author: Rajeev Chaurasia (rajeevkchaurasia) Date: 2021-06-14 04:36
I am running an application using cronjob. This application runs on Linux(X86_64) platform where inside a loop we have print() and sys.stdout.flush() statements. This loop iterates around 500 times and after the loop we have few more sys.stdout.flush() and print() statements. 

When I am running same application on terminal (without cronjob) then it is working fine.

Inside the loop everything is working fine but application is hanging after the loop. I put some debug statements and I found it is hanging on print() and sys.stdout.flush().


Code logic:-

for x in range(500):
  print(x)
  print ("something")
  sys.stdout.flush()
self.logger.info("A")
sys.stdout.flush()     #1 
self.logger.info("B")
print ("Hello")        #2
self.logger.info("C")
print ("Bye")          #3
self.logger.info("D")
-----------------------------------------
"A" is getting printed in log file and nothing after that but-
If I comment #1 then I get "B" as well in the log file.
If I comment #2 then I get "C" as well in the log file
If I comment #3 then I get "D" as well in the log file.

-----------------------------------------
ps results-
[root@scao05adm07 ~]# ps -ef|grep exachk
root     239265  98963  0 Jun11 ?        00:00:00 sh /opt/oracle.ahf/bin/exachk -silentforce -showpass -dball -nocvu -autorun_id DEFAULT
root     239706 239265  0 Jun11 ?        00:00:35 /opt/oracle.ahf/python/bin/python /opt/oracle.ahf/exachk/exachk.py -silentforce -showpass -dball -nocvu -autorun_id DEFAULT
root     277938 239706  0 Jun11 ?        00:00:00 sh /opt/oracle.ahf/exachk/exachk -silentforce -showpass -dball -nocvu -autorun_id DEFAULT -localonly -sudo_remoterun 0
root     278989 277938  0 Jun11 ?        00:12:26 /opt/oracle.ahf/python/bin/python /opt/oracle.ahf/exachk/exachk.py -silentforce -showpass -dball -nocvu -autorun_id DEFAULT -localonly -sudo_remoterun 0
root     281983 278989  0 Jun11 ?        00:00:27 /opt/oracle.ahf/python/bin/python /opt/oracle.ahf/exachk/exachk.py -silentforce -showpass -dball -nocvu -autorun_id DEFAULT -localonly -sudo_remoterun 0

-----------------------------------------
# uname -a
Linux ************ 4.14.35-1902.306.2.2.el7uek.x86_64 #2 SMP Thu Nov 19 18:09:09 PST 2020 x86_64 x86_64 x86_64 GNU/Linux
-----------------------------------------
I believe this issue is related to standard output buffer but I am not able to find a fix for it.

Please help.
-Rajeev
msg395776 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-06-14 06:39
What is self.logger and how do you run your application? Do you redirect its stdout and stderr and to where?
msg395785 - (view) Author: Rajeev Chaurasia (rajeevkchaurasia) Date: 2021-06-14 09:58
self.logger = lib.logger.Log()

We have cronjob entry to call the application everyday
$crontab -e
0 1 * * * /scratch/rajeev/cronjob/exachk.sh

exachk.sh makes call to my python application as-
python3 /opt/oracle.ahf/exachk/exachk.py

Yes I am redirecting stdout to-

sys.stdout = lib.debugger.Tee("/tmp/exachk_screen_output_061321_23211.log")

I am not redirecting stderr anywhere.
msg395793 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2021-06-14 12:43
I do not know what are lib.logger.Log and lib.debugger.Tee, but I guess that lib.logger.Log() creates a logger which writes to sys.stdout, and lib.debugger.Tee() is a file-like object which writes to the specified file and to standard output.

Standard output of your program is not consumed, so your program writes to it until fill the buffer and then hangs. Add >/dev/null 2>/dev/null at the end of the command in cronjob to redirect stdout and stderr. This issue is not specific for Python, you can get the same issue with any program which outputs to stdout or stderr.
msg395907 - (view) Author: Rajeev Chaurasia (rajeevkchaurasia) Date: 2021-06-16 06:43
Thanks a lot Serhiy.
Your suggested fix resolved the issue.
-Rajeev
History
Date User Action Args
2021-06-16 06:43:07rajeevkchaurasiasetstatus: open -> closed

messages: + msg395907
stage: resolved
2021-06-14 12:43:51serhiy.storchakasetmessages: + msg395793
2021-06-14 09:58:43rajeevkchaurasiasetmessages: + msg395785
2021-06-14 06:39:53serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg395776
2021-06-14 04:36:52rajeevkchaurasiacreate