annotate MoinMoin/util/SubProcess.py @ 5212:ce70252a3e90

Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below) Due to the usage of multiple pipes (stdout, stderr) to the subprocess (usually a external filter program like pdftotext or antiword), deadlocks could occur (especially when the filter program wrote lots of output to stderr, larger than the buffer). This was (hopefully) fixed as follows: Python >= 2.4: use enhanced subclass of subprocess.Popen that implements a timeout - if the process did not end when the timeout is reached, it will get killed automatically. Note: there is windows code to support the timeout functionality, but only the linux code was tested. On windows with Python < 2.6, you need Python win32 extensions to support process termination. Python 2.3: use popen2 to only get stdout (not stderr) - no support for: * timeout * stderr * rc Please use Python >= 2.4 if you have issues with this. We will drop the popen2 based code after requiring Python 2.4 with moin 1.9.
author Thomas Waldmann <tw AT waldmann-edv DOT de>
date Wed, 07 Oct 2009 18:47:05 +0200
parents
children bf05a6ae61ea
rev   line source
5212
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
1 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
2 Enhanced subprocess.Popen subclass, supporting:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
3 * .communicate() with timeout
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
4 * kill/terminate/send_signal (like in Py 2.6) for Py 2.4 / 2.5
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
5
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
6 Sample usage:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
7 out, err = Popen(...).communicate(input, timeout=300)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
8 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
9
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
10 import os
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
11 import time
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
12 import subprocess
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
13 import threading
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
14 import signal
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
15
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
16 if subprocess.mswindows:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
17 try:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
18 # Python >= 2.6 should have this:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
19 from _subprocess import TerminateProcess
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
20 except ImportError:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
21 # otherwise you need win32 extensions:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
22 from win32process import TerminateProcess
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
23 else:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
24 import select
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
25
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
26 class Popen(subprocess.Popen):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
27 # send_signal, terminate, kill copied from Python 2.6
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
28 # (we want to support Python >= 2.4)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
29 if subprocess.mswindows:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
30 def send_signal(self, sig):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
31 """Send a signal to the process
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
32 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
33 if sig == signal.SIGTERM:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
34 self.terminate()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
35 else:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
36 raise ValueError("Only SIGTERM is supported on Windows")
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
37
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
38 def terminate(self):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
39 """Terminates the process
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
40 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
41 TerminateProcess(self._handle, 1)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
42
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
43 kill = terminate
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
44
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
45 else: # POSIX
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
46 def send_signal(self, sig):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
47 """Send a signal to the process
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
48 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
49 os.kill(self.pid, sig)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
50
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
51 def terminate(self):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
52 """Terminate the process with SIGTERM
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
53 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
54 self.send_signal(signal.SIGTERM)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
55
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
56 def kill(self):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
57 """Kill the process with SIGKILL
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
58 """
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
59 self.send_signal(signal.SIGKILL)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
60
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
61 def communicate(self, input=None, timeout=None):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
62 """Interact with process: Send data to stdin. Read data from
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
63 stdout and stderr, until end-of-file is reached. Wait for
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
64 process to terminate. The optional input argument should be a
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
65 string to be sent to the child process, or None, if no data
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
66 should be sent to the child.
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
67
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
68 communicate() returns a tuple (stdout, stderr)."""
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
69
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
70 self.timeout = timeout
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
71
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
72 # Optimization: If we are only using one pipe, or no pipe at
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
73 # all, using select() or threads is unnecessary.
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
74 if [self.stdin, self.stdout, self.stderr].count(None) >= 2:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
75 stdout = None
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
76 stderr = None
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
77 if self.stdin:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
78 if input:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
79 self._fo_write_no_intr(self.stdin, input)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
80 self.stdin.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
81 elif self.stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
82 stdout = self._fo_read_no_intr(self.stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
83 self.stdout.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
84 elif self.stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
85 stderr = self._fo_read_no_intr(self.stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
86 self.stderr.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
87 self.wait()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
88 return (stdout, stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
89
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
90 return self._communicate(input)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
91
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
92 if subprocess.mswindows:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
93 def _communicate(self, input):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
94 stdout = None # Return
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
95 stderr = None # Return
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
96
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
97 if self.stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
98 stdout = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
99 stdout_thread = threading.Thread(target=self._readerthread,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
100 args=(self.stdout, stdout))
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
101 stdout_thread.setDaemon(True)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
102 stdout_thread.start()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
103 if self.stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
104 stderr = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
105 stderr_thread = threading.Thread(target=self._readerthread,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
106 args=(self.stderr, stderr))
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
107 stderr_thread.setDaemon(True)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
108 stderr_thread.start()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
109
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
110 if self.stdin:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
111 if input is not None:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
112 self.stdin.write(input)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
113 self.stdin.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
114
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
115 if self.stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
116 stdout_thread.join(self.timeout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
117 if self.stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
118 stderr_thread.join(self.timeout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
119
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
120 # if the threads are still alive, that means the thread join timed out
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
121 timed_out = (self.stdout and self.stdout_thread.isAlive() or
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
122 self.stderr and self.stderr_thread.isAlive())
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
123 if timed_out:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
124 self.kill()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
125 else:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
126 self.wait()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
127
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
128 # All data exchanged. Translate lists into strings.
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
129 if stdout is not None:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
130 stdout = stdout[0]
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
131 if stderr is not None:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
132 stderr = stderr[0]
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
133
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
134 # Translate newlines, if requested. We cannot let the file
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
135 # object do the translation: It is based on stdio, which is
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
136 # impossible to combine with select (unless forcing no
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
137 # buffering).
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
138 if self.universal_newlines and hasattr(file, 'newlines'):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
139 if stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
140 stdout = self._translate_newlines(stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
141 if stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
142 stderr = self._translate_newlines(stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
143
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
144 return (stdout, stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
145
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
146 else: # POSIX
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
147 def _communicate(self, input):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
148 timed_out = False
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
149 read_set = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
150 write_set = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
151 stdout = None # Return
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
152 stderr = None # Return
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
153
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
154 if self.stdin:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
155 # Flush stdio buffer. This might block, if the user has
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
156 # been writing to .stdin in an uncontrolled fashion.
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
157 self.stdin.flush()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
158 if input:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
159 write_set.append(self.stdin)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
160 else:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
161 self.stdin.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
162 if self.stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
163 read_set.append(self.stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
164 stdout = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
165 if self.stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
166 read_set.append(self.stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
167 stderr = []
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
168
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
169 input_offset = 0
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
170 while read_set or write_set:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
171 try:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
172 rlist, wlist, xlist = select.select(read_set, write_set, [], self.timeout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
173 except select.error, e:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
174 if e.args[0] == errno.EINTR:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
175 continue
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
176 raise
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
177
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
178 timed_out = not (rlist or wlist or xlist)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
179 if timed_out:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
180 break
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
181
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
182 if self.stdin in wlist:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
183 # When select has indicated that the file is writable,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
184 # we can write up to PIPE_BUF bytes without risk
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
185 # blocking. POSIX defines PIPE_BUF >= 512
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
186 chunk = input[input_offset:input_offset + 512]
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
187 bytes_written = os.write(self.stdin.fileno(), chunk)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
188 input_offset += bytes_written
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
189 if input_offset >= len(input):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
190 self.stdin.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
191 write_set.remove(self.stdin)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
192
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
193 if self.stdout in rlist:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
194 data = os.read(self.stdout.fileno(), 1024)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
195 if data == "":
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
196 self.stdout.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
197 read_set.remove(self.stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
198 stdout.append(data)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
199
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
200 if self.stderr in rlist:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
201 data = os.read(self.stderr.fileno(), 1024)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
202 if data == "":
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
203 self.stderr.close()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
204 read_set.remove(self.stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
205 stderr.append(data)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
206
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
207 # All data exchanged. Translate lists into strings.
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
208 if stdout is not None:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
209 stdout = ''.join(stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
210 if stderr is not None:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
211 stderr = ''.join(stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
212
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
213 # Translate newlines, if requested. We cannot let the file
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
214 # object do the translation: It is based on stdio, which is
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
215 # impossible to combine with select (unless forcing no
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
216 # buffering).
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
217 if self.universal_newlines and hasattr(file, 'newlines'):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
218 if stdout:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
219 stdout = self._translate_newlines(stdout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
220 if stderr:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
221 stderr = self._translate_newlines(stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
222
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
223 if timed_out:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
224 self.kill()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
225 else:
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
226 self.wait()
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
227 return (stdout, stderr)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
228
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
229
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
230 def exec_cmd(cmd, input=None, timeout=None):
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
231 p = Popen(cmd, shell=True,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
232 close_fds=not subprocess.mswindows,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
233 bufsize=1024,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
234 stdin=subprocess.PIPE,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
235 stdout=subprocess.PIPE,
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
236 stderr=subprocess.PIPE)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
237 data, errors = p.communicate(input, timeout=timeout)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
238 return data, errors, p.returncode
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
239
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
240
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
241 if __name__ == '__main__':
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
242 print exec_cmd("python", "import time ; time.sleep(20) ; print 'never!' ;", timeout=10)
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
243 print exec_cmd("python", "import time ; time.sleep(20) ; print '20s gone' ;")
ce70252a3e90 Xapian indexing: fix deadlocks, new MoinMoin.util.SubProcess module (see below)
Thomas Waldmann <tw AT waldmann-edv DOT de>
parents:
diff changeset
244