Rob Browning [Wed, 31 Jul 2013 16:32:17 +0000 (11:32 -0500)]
_create_via_common_rec: treat rmdir() EEXIST like ENOTEMPTY.
See rmdir(2):
ENOTEMPTY
pathname contains entries other than . and .. ; or,
pathname has .. as its final component. POSIX.1-2001
also allows EEXIST for this condition.
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Thu, 4 Jul 2013 19:12:22 +0000 (14:12 -0500)]
config/configure: add and use bup_find_prog; remove unused MF_PATH_INCLUDEs.
Pull all the common program search code into bup_find_prog() and use
it everywhere. This also makes configure's output a bit more
consistent in style/content.
Make it obvious that at the moment bup doesn't pay any attention to a
PYTHON or GIT environment variable value. i.e. bup currently ignores a
setting like PYTHON=python2.7.
Bup also doesn't use MF_PATH_INCLUDE settings yet, so remove them.
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Sat, 8 Jun 2013 15:25:33 +0000 (10:25 -0500)]
git.py: flush idx_map before close so FILE* operations will see changes.
Flush idx_map (msync()) before closing it, since it doesn't look like
POSIX guarantees that a matching FILE* (i.e. idx_f) will see the
parallel changes if we don't.
From the original report:
After `bup save`, `git fsck` would show messages like
"Packfile index for %s SHA1 mismatch"
This indicated a bad trailing checksum on the pack index file.
Signed-off-by: Rob Browning <rlb@defaultvalue.org> Reported-by: Alan Jenkins <alan.christopher.jenkins@gmail.com> Reviewed-by: Alan Jenkins <alan.christopher.jenkins@gmail.com> Tested-by: Alan Jenkins <alan.christopher.jenkins@gmail.com>
Rob Browning [Sun, 26 May 2013 17:16:40 +0000 (12:16 -0500)]
Remove lib.sh actually-root in favor of t/root-status.
Create a t/root-status command that outputs root, fake, or none and
that uses the same cross-platform logic as helpers.py is_superuser().
Use it everywhere.
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Sun, 26 May 2013 16:05:56 +0000 (11:05 -0500)]
metadata.py: use socket() instead of mknod(...IF_SOCK) on Cygwin.
Apparently os.mknod(...IF_SOCK) fails on Cygwin, but our t/mksock tool
(which uses socket()) works just fine, so use that in
_create_via_common_rec() when on Cygwin.
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Catch errors when restoring Linux attrs into xfs or btrfs
Both xfs and btrfs turn out to return EOPNOTSUPP, rather than ENOTTY,
when we try to apply Linux attrs that they do not support. This patch
improves matters in that it ensures we can restore into these file
systems without crashing, but does not quite address all issues yet:
- if any of the attributes we try to apply is not supported, none of
them will be applied to the file at all
- the user will typically drown in error output; eg. for a backup
originating from an ext4 file system, we can expect endless error
output relating to the extents-flag not being supported on xfs/btrfs
Signed-off-by: Yung-Chin Oei <yungchin@yungchin.nl> Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Gabriel Filion [Tue, 14 May 2013 06:53:05 +0000 (02:53 -0400)]
Documentation: replace ~/.bup with $BUP_DIR in bup-index.md and bup-save.md.
When one is not using the default bupdir placement, then those files
aren't found inside ~/.bup, but wherever "-d /path.." or $BUP_DIR are
pointing to.
Thanks to krichter722@aol.de for mentioning this.
Signed-off-by: Gabriel Filion <gabster@lelutin.ca> Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Fix a timing issue that often caused the test to fail -- the (removed)
file "foo" was marked as invalid in the index when it shouldn't have
been because it was modified less than tmax before the index run.
Insert a "bup tick" just before "bup index" so that foo won't be
marked invalid by the second run.
Don't expect the second "bup save" to fail since foo will now be valid
in the index (and hence ignored), even though it is no longer
available. The removal will be noticed by the next index+save.
When removing directories, the outcome is different: since metadata is
saved per directory, "bup save" notices that the directory is missing
and logs an "errno 2" error, but does not crash, which is what the
tested patch is concerned with. A third index+save run completes
without error.
This also means that without a "bup tick" before an index run, calls,
"bup save" might exit with non-zero exitcode when files/directories
are removed after bup index, but bup is not guaranteed to notice
missing files.
Signed-off-by: Tim Riemenschneider <git@tim-riemenschneider.de> Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Add tests for file and dir removal between index and save.
Make sure the removals don't cause bup save to immediately abort.
Signed-off-by: Tim Riemenschneider <git@tim-riemenschneider.de>
[rlb@defaultvalue.org: move tests to their own file.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Damien Robert [Tue, 20 Nov 2012 20:52:45 +0000 (21:52 +0100)]
save-cmd.py: don't crash when a path disappears between index and save.
Protect calls to metadata.from_path() with a try/catch and defer
errors via add_error() instead of just crashing.
Signed-off-by: Damien Robert <damien.olivier.robert@gmail.com>
[git@tim-riemenschneider.de: rebase to current tmp/pending/meta: since
metadata is (now) stored in the index, only 1 of 2 hunks still
applies (race condition between reading the file for its content and
stat-ing it for metadata).]
[rlb@defaultvalue.org: edit commit message; squash file and dir
removal fixes into this commit; limit the scope of the try/catch to
the from_path() call, and put the remaining code in an else clause.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Sat, 18 Aug 2012 20:35:23 +0000 (15:35 -0500)]
Add support for "bup restore --exclude-rx <pattern> ...".
When --exclude-rx <pattern> is provided to bup restore, don't restore
any path matching <pattern>, which must be a Python regular expression
(http://docs.python.org/library/re.html). The pattern will be
compared against the full path, without anchoring, so "x/y" will match
"ox/yard" or "box/yards". To exclude the contents of /tmp, but not
the directory itself, use "^/tmp/.".
You may check the behavior at runtime by setting BUP_DEBUG=2 in the
environment.
Thanks to Zoran Zaric <zz@zoranzaric.de> for reporting a bug in an
earlier version of this patch.
Signed-off-by: Rob Browning <rlb@defaultvalue.org> Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Rob Browning [Sat, 21 Jul 2012 20:09:47 +0000 (15:09 -0500)]
Add support for "bup index --exclude-rx <pattern> ...".
When --exclude-rx <pattern> is provided to bup index, exclude any path
matching <pattern>, which must be a Python regular expression
(http://docs.python.org/library/re.html). The pattern will be
compared against the full path, without anchoring, so "x/y" will match
"ox/yard" or "box/yards". To exclude the contents of /tmp, but not
the directory itself, use "^/tmp/.".
You may check the behavior at runtime by setting BUP_DEBUG=2 in the
environment.
Thanks to Zoran Zaric <zz@zoranzaric.de> for reporting a bug in an
earlier version of this patch.
Signed-off-by: Rob Browning <rlb@defaultvalue.org> Reviewed-by: Zoran Zaric <zz@zoranzaric.de>
Thomas Haller [Sun, 3 Mar 2013 19:31:57 +0000 (20:31 +0100)]
wvtest.py: add a fallback definition of relpath() for Python < 2.6.
Since bup still supports Python 2.5 and relpath was added to Python in
2.6, add an os.path.relpath replacement, adapted from CPython source
(tag v2.6, file Lib/posixpath.py, hg-commit 95fff5a6a276).
Signed-off-by: Thomas Haller <thom311@gmail.com>
[rlb@defaultvalue.org: update LICENSE and add comment to code about
updating LICENSE.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Signed-off-by: Zak Wilcox <iwilcox@iwilcox.me.uk>
[rlb@defaultvalue.org: change original --ignore-dev to --no-check-device to match tar.
Adjust code to work with current master.
Remove tests -- will be reintroduced shortly.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Gabriel Filion [Sun, 17 Mar 2013 22:24:08 +0000 (18:24 -0400)]
Ignore artefacts from sampledata
9b5ae5f53e introduced the creation of some symlinks to avoid issues on
Cygwin. "make" now leaves the repository in a dirty state since we
forgot to exclude those symlinks.
The symlinks in question are products of the build/test process so we
need to avoid commiting them.
Signed-off-by: Gabriel Filion <gabster@lelutin.ca> Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Jakob Matthes [Sun, 10 Mar 2013 18:10:38 +0000 (19:10 +0100)]
bup drecurse/index: allow non-directory --exclude and --exclude-from paths.
Previously bup only allowed directory exclusions.
Signed-off-by: Jakob Matthes <jakob.matthes@gmail.com> Reviewed-by: Gabriel Filion <lelutin@gmail.com>
[rlb@defaultvalue.org: change test file from g to j to avoid conflict
with subsequent "exclude-from" test; add WVPASS to mkdir so problem
is evident; edit commit message.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Mark J Hewitt [Mon, 1 Oct 2012 16:17:28 +0000 (17:17 +0100)]
Use the correct C types when converting between C and Python in _helpers.c.
Previously, bup might fail when handling large repositories. For
example:
Traceback (most recent call last):
File "/usr/lib/bup/cmd/bup-midx", line 258, in <module>
do_midx_dir(path)
File "/usr/lib/bup/cmd/bup-midx", line 199, in do_midx_dir
all = list(do_midx_group(path, part1)) + part2
File "/usr/lib/bup/cmd/bup-midx", line 216, in do_midx_group
rv = _do_midx(path, None, sublist, gprefix)
File "/usr/lib/bup/cmd/bup-midx", line 123, in _do_midx
count = merge_into(fmap, bits, total, inp)
OverflowError: size does not fit in an int
['/usr/bin/bup', 'midx', '--auto', '--dir',
'/backup/bup/repository/objects/pack']: returned 1
Signed-off-by: Mark J Hewitt <m.hewitt@computer.org>
[rlb@defaultvalue.org: Remove vestigal header includes.
Remove Py_ssize_t definition for Python < 2.5.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
save-cmd.py: insist on having an index; check for the primary index file.
Insist on having an index, and report an error (with a suggestion to
run "bup index") if one isn't found.
Previously (at least as recently as 0.25-rc1), bup save would just
create an empty tree if there was no index. The metadata work
unintentionally broke that behavior.
For now, treat a missing index as an error under the assumption that
it's not likely to be intentional. If a good use case for the
previous behavior is presented, we can always restore it.
Signed-off-by: Jean-Baptiste Denis <jbd@jbdenis.net>
[rlb@defaultvalue.org: adjust commit message and save-cmd.py error message.] Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Sun, 17 Feb 2013 16:33:08 +0000 (10:33 -0600)]
Normalize stat(2) timespecs in _helpers.c.
Previously, the code required timespecs to have a nanosecond value
between 0 and 999999999, because that's what Linux appeared to
produce, but other platforms don't (i.e. Cygwin) -- so normalize the
system values to match expectations.
In particular, look for negative nanosecond values (must be greater
than -10**9), and if found, rewrite the timespec with non-negative
nanoseconds (still less than 10**9).
Thanks to Will Rouesnel <w.rouesnel@gmail.com> for reporting the
problem and posting an initial patch.
Signed-off-by: Rob Browning <rlb@defaultvalue.org>
Ryan Brown [Thu, 7 Mar 2013 02:48:23 +0000 (21:48 -0500)]
Add a force-delete function to t/test.sh
Replace "rm -rf" with force-delete because "rm -rf" will proceed even
if it doesn't succeed. This masked a problem (reported on Fedora)
where the root directory and other system directories had "r-xr-xr-x"
(0555) permissions, causing bup restore to create directories that "rm
-rf" couldn't remove. As a result, subsequent tests would fail.
Signed-off-by: Ryan Brown <ryansb@csh.rit.edu>
[rlb@defaultvalue.org: edited commit message.] Reviewed-by: Rob Browning <rlb@defaultvalue.org>
Rob Browning [Sun, 3 Mar 2013 22:51:14 +0000 (16:51 -0600)]
Fix use of Python assert() with respect to optimization.
Fix a number of places where bup's assertions had material
side-effects, or where other code expected to see the AssertionError,
neither of which happen when optimization is enabled.
Reported-by: Jon Dowland <jmtd@debian.org> Signed-off-by: Rob Browning <rlb@defaultvalue.org>
projects / packages / b / bup.git / log