subprocess的管道输出。打开文件

I need to launch a number of long-running processes with subprocess.Popen, and would like to have the stdout and stderr from each automatically piped to separate log files. Each process will run simultaneously for several minutes, and I want two log files (stdout and stderr) per process to be written to as the processes run.

我需要使用subprocess.Popen启动一些长时间运行的进程，并希望将每个进程中的stdout和stderr自动通过管道分离日志文件。每个进程将同时运行几分钟，并且我希望在进程运行时将每个进程写入两个日志文件（stdout和stderr）。

Do I need to continually call p.communicate() on each process in a loop in order to update each log file, or is there some way to invoke the original Popen command so that stdout and stderr are automatically streamed to open file handles?

我是否需要在循环中的每个进程上不断调用p.communicate（）以更新每个日志文件，或者是否有某种方法来调用原始Popen命令以便stdout和stderr自动流式传输以打开文件句柄？

3 个解决方案

#1

Per the docs,

根据文档，

stdin, stdout and stderr specify the executed programs’ standard input, standard output and standard error file handles, respectively. Valid values are PIPE, an existing file descriptor (a positive integer), an existing file object, and None.

stdin，stdout和stderr分别指定执行程序的标准输入，标准输出和标准错误文件句柄。有效值为PIPE，现有文件描述符（正整数），现有文件对象和无。

So just pass the open-for-writing file objects as named arguments stdout= and stderr= and you should be fine!

所以只需将open-for-writing文件对象作为命名参数stdout =和stderr =传递给你，你应该没问题！

#2

You can pass stdout and stderr as parameters to Popen()

您可以将stdout和stderr作为参数传递给Popen（）

subprocess.Popen(self, args, bufsize=0, executable=None, stdin=None, stdout=None,
                 stderr=None, preexec_fn=None, close_fds=False, shell=False,
                 cwd=None, env=None, universal_newlines=False, startupinfo=None, 
                 creationflags=0)

For example

例如

>>> import subprocess
>>> with open("stdout.txt","wb") as out, open("stderr.txt","wb") as err:
...    subprocess.Popen("ls",stdout=out,stderr=err)
... 
<subprocess.Popen object at 0xa3519ec>
>>>

#3

I am simultaneously running two subprocesses, and saving the output from both into a single log file. I have also built in a timeout to handle hung subprocesses. When the output gets too big, the timeout always triggers, and none of the stdout from either subprocess gets saved to the log file. The answer posed by Alex above does not solve it.

我同时运行两个子进程，并将两者的输出保存到一个日志文件中。我还建立了一个超时来处理挂起的子进程。当输出太大时，超时总是触发，并且任何子进程的stdout都不会保存到日志文件中。亚历克斯上面提出的答案并没有解决它。

# Currently open log file.
log = None

# If we send stdout to subprocess.PIPE, the tests with lots of output fill up the pipe and
# make the script hang. So, write the subprocess's stdout directly to the log file.
def run(cmd, logfile):
   #print os.getcwd()
   #print ("Running test: %s" % cmd)
   global log
   p = subprocess.Popen(cmd, shell=True, universal_newlines = True, stderr=subprocess.STDOUT, stdout=logfile)
   log = logfile
   return p


# To make a subprocess capable of timing out
class Alarm(Exception):
   pass

def alarm_handler(signum, frame):
   log.flush()
   raise Alarm


####
## This function runs a given command with the given flags, and records the
## results in a log file. 
####
def runTest(cmd_path, flags, name):

  log = open(name, 'w')

  print >> log, "header"
  log.flush()

  cmd1_ret = run(cmd_path + "command1 " + flags, log)
  log.flush()
  cmd2_ret = run(cmd_path + "command2", log)
  #log.flush()
  sys.stdout.flush()

  start_timer = time.time()  # time how long this took to finish

  signal.signal(signal.SIGALRM, alarm_handler)
  signal.alarm(5)  #seconds

  try:
    cmd1_ret.communicate()

  except Alarm:
    print "myScript.py: Oops, taking too long!"
    kill_string = ("kill -9 %d" % cmd1_ret.pid)
    os.system(kill_string)
    kill_string = ("kill -9 %d" % cmd2_ret.pid)
    os.system(kill_string)
    #sys.exit()

  end_timer = time.time()
  print >> log, "closing message"

  log.close()

#1