From: Carsten Bormann <cabo@tzi.org>
Date: Thu, 6 Jan 2011 21:38:10 +0000 (-0800)
Subject: git.CatPipe: set a buffer size on the subprocess to increase performance.
X-Git-Url: https://git.michaelhowe.org/gitweb/?a=commitdiff_plain;h=9aa4a3824c3c2ae038f1b9efcc886f74e06d86da;p=packages%2Fb%2Fbup.git

git.CatPipe: set a buffer size on the subprocess to increase performance.

apenwarr: I modified Carsten's patch slightly, since "line mode" is not
really appropriate.  On my system, this patch (or Carsten's) can read 111
megabytes in 1.7 seconds instead of 2.1 seconds, or 65MB/sec instead of 52
MB/sec.

Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
---

diff --git a/lib/bup/git.py b/lib/bup/git.py
index 49e9801..081ae62 100644
--- a/lib/bup/git.py
+++ b/lib/bup/git.py
@@ -998,6 +998,7 @@ class CatPipe:
                                   stdin=subprocess.PIPE,
                                   stdout=subprocess.PIPE,
                                   close_fds = True,
+                                  bufsize = 4096,
                                   preexec_fn = _gitenv)
 
     def _fast_get(self, id):
@@ -1014,6 +1015,7 @@ class CatPipe:
         assert(not id.startswith('-'))
         self.inprogress = id
         self.p.stdin.write('%s\n' % id)
+        self.p.stdin.flush()
         hdr = self.p.stdout.readline()
         if hdr.endswith(' missing\n'):
             self.inprogress = None