git.CatPipe: set a buffer size on the subprocess to increase performance.

apenwarr: I modified Carsten's patch slightly, since "line mode" is not
really appropriate.  On my system, this patch (or Carsten's) can read 111
megabytes in 1.7 seconds instead of 2.1 seconds, or 65MB/sec instead of 52
MB/sec.

Signed-off-by: Avery Pennarun <apenwarr@gmail.com>

diff --git a/lib/bup/git.py b/lib/bup/git.py
index 49e98019c2927c023de97a8683a6f4992c08c518..081ae623ead87670c064884d5d2d927e455c792a 100644 (file)
--- a/lib/bup/git.py
+++ b/lib/bup/git.py
@@ -998,6 +998,7 @@ class CatPipe:
                                   stdin=subprocess.PIPE,
                                   stdout=subprocess.PIPE,
                                   close_fds = True,
+                                  bufsize = 4096,
                                   preexec_fn = _gitenv)
 
     def _fast_get(self, id):
@@ -1014,6 +1015,7 @@ class CatPipe:
         assert(not id.startswith('-'))
         self.inprogress = id
         self.p.stdin.write('%s\n' % id)
+        self.p.stdin.flush()
         hdr = self.p.stdout.readline()
         if hdr.endswith(' missing\n'):
             self.inprogress = None