Andrew Deason [Wed, 10 Mar 2010 16:16:04 +0000 (10:16 -0600)]
Squash pthreaded vos warnings
Trying to build vos in src/tvolser can produce a warning, at least on
amd64_linux26, since we cast to an int of the wrong size in SendFile and
ReceiveFile. Use intptr_t like we do everywhere else instead.
Simon Wilkinson [Wed, 10 Mar 2010 15:40:50 +0000 (15:40 +0000)]
Don't always use the local cell for db clients
Commit d008089a79ef268bbca91d660a840f32cb416865 broke things so that
we were always trying to use the local cell, rather than the one specified
by the user. Fix this so that the command line cell is used, unles -localauth
is specified, which restores the existing behaviour.
Andrew Deason [Tue, 9 Mar 2010 17:43:42 +0000 (11:43 -0600)]
vldb_check: do not ntohl u_chars
vldb_check was ntohl'ing fields like serverNumber when reading in a VL
entry. Since this is a u_char, ntohl'ing will just always result in 0,
not the actual value. Remove the ntohls so we read in VL data correctly.
Andrew Deason [Fri, 19 Feb 2010 23:04:29 +0000 (17:04 -0600)]
Add FSYNC_VerifyCheckout
Add the function FSYNC_VerifyCheckout, which can be used to verify that
a previous checkout of a volume is still known by the fileserver, and
hasn't been forgotten about (by e.g. the fileserver restarting).
Andrew Deason [Tue, 23 Feb 2010 21:43:11 +0000 (15:43 -0600)]
Add DAFS documentation overview for developers
Add some developer documentation for DAFS. This isn't comprehensive or
extremely detailed, but is intended to provide a larger-picture overview
of some of the pieces of DAFS. More detailed docs are intended to exist
in source comments (doxygen and such), and should already be there for
the most part.
Andrew Deason [Fri, 19 Feb 2010 21:17:56 +0000 (15:17 -0600)]
Schedule all salvages via VScheduleSalvage_r
Change I03ecf6302436c35fec705cd6c84a40b7cdbf6f97 allowed non-fileserver
programs to schedule salvages via FSSYNC, making the FSSYNC call
directly in VRequestSalvage_r. This isn't as safe as making the call in
VScheduleSalvage_r (as is done when the fileserver schedules a salvage
via SALVSYNC), since we may not have relinquished all of our handles and
such for the volume by the time the salvager starts.
So instead, make this path a bit more like the fileserver, and make
non-fileserver programs actually make the FSSYNC call in
VScheduleSalvage_r. Consequently, make VScheduleSalvage_r conditionally
hit FSSYNC or SALVSYNC. Add a VCheckSalvage to attach2 failures, so
non-fileserver salvage requests actually get scheduled.
Also, reorganize the attach2 error handling a bit to make this a bit
easier.
Andrew Deason [Thu, 18 Feb 2010 20:22:39 +0000 (14:22 -0600)]
Add FSSYNC debug logging
Add logging for what FSSYNC commands are being received by the
fileserver, so it is possible to see what's going on with FSSYNC. Log at
level 125, since this isn't interesting unless you knows some FSSYNC
details, and it can get quite verbose.
Andrew Deason [Wed, 24 Feb 2010 15:47:03 +0000 (09:47 -0600)]
Move *SYNC string translation out of fssync-debug
Move the routines for translating ProgramType codes, and SYNC command
codes, response codes, etc out of fssync-debug, and into header files
anyone can use. These routines are useful especially for SYNC-related
debug or error logs.
Andrew Deason [Thu, 18 Feb 2010 18:21:45 +0000 (12:21 -0600)]
Do not rely on vol header for V*VolumeHandles_r
VCloseVolumeHandles_r and VReleaseVolumeHandles_r were using V_id to get
the id of the volume; just use vp->hashid so we can call these even if
we lack a header.
also report unload. in event of panic after module is unloaded (which
*shouldn't* happen but i have evidence it might) you can retcon where the
module was
Ben Kaduk [Tue, 9 Mar 2010 04:38:15 +0000 (23:38 -0500)]
Export prototypes for osi_fbsd_{alloc,free} for use in rx
Include prototypes for osi_fbsd_alloc() and osi_fbsd_free() in
osi_machdep.h, since afs_prototypes.h is not included when compiling
the rx code. afs_osi_Alloc_NoSleep is #defined to be osi_fbsd_free,
and is used in the rx code; if the prototype for the latter is not
included, then int is assumed for all parameters and return values,
which breaks the calling convention for 64-bit pointers.
Ben Kaduk [Sat, 6 Mar 2010 20:37:50 +0000 (15:37 -0500)]
Use correct types for UFS devices
For the FreeBSD 5.3 release, UFS (and everything else) took a
"dreaded s/dev_t/struct cdev */" change; we need to keep up
with this when comparing mountpoints' devices.
Ben Kaduk [Sat, 6 Mar 2010 20:20:18 +0000 (15:20 -0500)]
Use the correct API for msleep() in FBSD's afs_osi_TimedSleep()
msleep() does not take a struct timespec*, it takes its timeout
value as a multiple of the kernel's HZ parameter (tuneable at
startup). Since the afs_osi_TimedSleep interface is documented
to take a timeout in milliseconds, we must correct for the (sometimes)
different units (currently HZ defaults to 1000). We prefer
to multiply the timeout by HZ before dividing by 1000 so as to
not lose precision; overflow is assumed to be unlikely.
OpenBSD 4.7 (currently in beta) removes the (dummy) VREF() macro from the
system header files. For at least as far back as OpenBSD 3.6, all this
macro does is expand to a call to a system routine by the same name in
lower case. This patch therefore replaces the single OpenBSD use of
VREF() with a direct call to the routine to which it expanded.
Ben Kaduk [Sat, 6 Mar 2010 19:51:17 +0000 (14:51 -0500)]
Be type correct in osi_ThreadUnique() for FBSD
Formerly, in AFS_FBSD50_ENV, we used curproc for the ThreadUnique
value; however, curproc (#defined as curthread->td_proc) is a
struct proc *, not an actual pid. (As such, it suffers from
a 32/64-bit mismatch on 64-bit systems.) Use the correct value,
curproc->p_pid, instead.
Ben Kaduk [Mon, 1 Mar 2010 01:08:52 +0000 (20:08 -0500)]
FBSD module loads now
Actually declare the global afs_global_owner, in afs_osi.c.
This allows all symbols to resolve, so that the loader will
accept the module. There are still significant issues, though.
Jeffrey Altman [Fri, 5 Mar 2010 14:49:12 +0000 (09:49 -0500)]
Windows: use krb5_get_error_message instead of error_message
krb5_get_error_message() can return more descriptive errors
based upon the context in which the error occurred by extracting
the cached error message from the krb5_context. Since aklog
has a krb5_context, use that instead of error_message().
Use GetProcAddress to load krb5_get_error_message and
krb5_free_error_message function pointers because versions
of KFW prior to 3.2 do not export them.
Simon Wilkinson [Sat, 6 Mar 2010 00:13:06 +0000 (00:13 +0000)]
Linux: Make keyring destructor remove all tokens
The keyring destructor on Linux was only destroying the first
unixuser structure found for the PAG being destroyed. In environments
where each PAG contains tokens for multiple cells, this led to us
leaking unixuser structures, and eventually grinding to a hault due
to the length of the resulting hash chains (and some horrific
O(n-squared) performance in the PAG statistics code).
Add a utility function for the keyring destruction function to call
which will mark as deleted all unixuser structures for a particular
PAG, and use it.
Simon Wilkinson [Sat, 6 Mar 2010 11:21:06 +0000 (11:21 +0000)]
Linux: Fix builds on RHEL4
RHEL4 has a very old 2.6 kernel (2.6.9), which predates the start of
the Linux git tree. When I started using page_offset, I mistakenly assumed
that everything in the initial commit to that tree was available in all
2.6 versions we care about. That isn't the case, sadly.
Secondly, the new readpage code uses zero_user_segments, which has only
been available in the mainline kernel since 2.6.25 (RHEL5 appears to have
a backport)
Implement local wrappers for both of these functions when configure can't
find them in the kernel we're building for.
These functions have been created independently of the Linux tree.
page_offset is a copy of the code we used before we replaced it.
zero_user_segments() is a first-principles implementation
of the function (which zeros a pair of memory ranges within a single page)
Marc Dionne [Fri, 5 Mar 2010 22:54:42 +0000 (17:54 -0500)]
Linux: replace invalidate_inode_pages
This helper has been deprecated for a while, and gets removed
with 2.6.34.
Replace it with invalidate_remote_inode, which has been around for
all of 2.6's life, according to Chaskiel in RT #124377.
Jeffrey Altman [Fri, 5 Mar 2010 14:43:13 +0000 (09:43 -0500)]
Windows: Add krb5 error message functions to loadfuncs header
Add krb5_get_error_message and krb5_free_error_message
to loadfuncs-krb5.h so that they can be used in conjunction
with LoadFuncs to obtain function pointers when available.
Jeffrey Altman [Thu, 4 Mar 2010 21:23:36 +0000 (16:23 -0500)]
Windows: reset local mount point count during freelance re-initialization
When a re-initialization is taking place, be sure to reset
cm_noLocalMountPoints to 0 in case someone deletes the "Freelance"
registry key out from underneath the service.
Marc Dionne [Fri, 26 Feb 2010 22:00:58 +0000 (17:00 -0500)]
Remove duplicate make targets in tubik, cleanup dependencies
Eliminate a few duplicate targets in the tubik Makefile, and
clean up some dependencies that could cause the same source file
to be compiled by different threads simultaneously when building
in parallel. At least on my current gcc this would consistently
result in a gcc internal error.
The duplicate targets generate make warnings.
Derrick Brashear [Sat, 27 Feb 2010 20:30:06 +0000 (15:30 -0500)]
BOP_MOVE and userspace move EXDEV helper
turns background daemons into afsdb helper-like processes, which can
exit and do work. for macos, add BOP_MOVE and implement mv in afsd:
(macos EXDEV move is cp+rm, literally)
run cp + rm and pass the return back in. if it fails, just give
the client the error it had already
Claudio Bisegni [Mon, 1 Mar 2010 19:41:48 +0000 (20:41 +0100)]
OSXPreferencePane
checkAfsStatusForStartup method modification for search /afs volume for determinate if afs is on has been transfered into checkAfsStatus. checkAfsStatusForStartup method is used to check when afs start axitn system startup. Anyway these are only workaround we must use osx api or afs api to make this job.
Export rx_SetConnSecondsUntilNatPing from afsrpc.dll.
Add NatPingInterval registry value to
HKLM\SYSTEM\CurrentControlSet\Services\TransarcAFSDaemon\Parameters
to permit Nat Ping to be enabled. The default value is 0 seconds.
Derrick Brashear [Thu, 25 Feb 2010 03:34:28 +0000 (22:34 -0500)]
rx lowlevel nat ping
for rfc 4787, do a minimal impact nat ping. this uses an rx
"version request" reply debug packet, which will simply be discarded by the
receiver, to keep the port mapping open.
Andrew Deason [Thu, 18 Feb 2010 17:36:10 +0000 (11:36 -0600)]
Convert ObtainSalvageLock to the VLockFile API
Make ObtainSalvageLock use the VLockFile API, to consolidate
platform-specific locking code, and to make it possible to acquire a
shared lock on the salvage lock file. Create the ObtainSharedSalvageLock
function to acquire such a lock.
Derrick Brashear [Wed, 24 Feb 2010 05:30:28 +0000 (00:30 -0500)]
darwin afscall syscall should set retval
because of how the ioctl based syscall works, passing back a real
return value should be done this way, as it is for the other other
afs syscall subcalls. matters for userspace handlers.
Rainer Toebbicke [Wed, 24 Feb 2010 11:02:08 +0000 (12:02 +0100)]
Do not leak information on partial success in RXAFS_InlineBulkStatus
Initialize all RXAFS_InlineBulkStatus return parameters in case portions
remain unset on FetchStatus failures. Also ensure VolSync info is set even if
the first FetchStatus fails.
Derrick Brashear [Wed, 24 Feb 2010 16:13:23 +0000 (11:13 -0500)]
remove dropbox attribute leak fix
change 6ec18461649d3e4f44b2476f886bcc480c456500, avoid leaking stat info,
appears to have side effects on at least macos, crashing the system
coreservicesd. revoke the change until the exact behavior we want is
better understood.
Andrew Deason [Thu, 18 Feb 2010 17:42:27 +0000 (11:42 -0600)]
VLockFile: Do not close fd on lock failure
When we fail to acquire a lock in _VLockFd, do not close the fd if we
fail to acquire the lock for any reason, since VLockFile does that for
us already. There also may be other locks on that file, and we clearly
do not want to release them when just one lock fails. VLockFile takes
care of the necessary refcounting and fd closing, so don't do it in
_VLockFd too.
Chaz Chandler [Tue, 23 Feb 2010 22:20:39 +0000 (17:20 -0500)]
add limits.h to sysincludes.h on IRIX
xdr_mem.c wasn't including limits.h on IRIX when KERNEL was declared, so i
INT_MAX was undefined. this adds limits.h to sysincludes.h so it's included
for all other places where it might be needed.
Anders Kaseorg [Sat, 20 Feb 2010 18:28:59 +0000 (13:28 -0500)]
Linux: Use kbuild to check for UML
<linux/autoconf.h> is automatically included by kbuild; you aren’t
supposed to include it directly. That breaks in 2.6.33 because
<linux/autoconf.h> is moving to <generated/autoconf.h>.
Anders Kaseorg [Sun, 21 Feb 2010 01:28:42 +0000 (20:28 -0500)]
Linux: Fix conversion of whole-file locks
An overflow bug in commit 49b7bbdd3b45df694fadbef48f9ed99d9bfe07b9
caused whole-file locks to be treated as byte-range locks, which fail
to be propagated to other machines.
Fix this by setting l_len = 0 for locks that range to the end of the
file.
Andrew Deason [Wed, 27 Jan 2010 21:08:34 +0000 (15:08 -0600)]
Abstract /vicepX header traversal
The code for examining all of the headers on a vice partition now exists
in two places: the VGC scanner, and the salvager. Create the
VWalkVolumeHeaders function that contains the common logic so it's in
one place, and take a couple of function callbacks for the differences.
Andrew Deason [Mon, 15 Feb 2010 22:22:56 +0000 (16:22 -0600)]
Allow GetSomeSpace_r to select an optimal host
Previously GetSomeSpace_r would never find an 'ideal' host for which to
clear callbacks, since lih0_r and lih1_r required a non-NULL rock to do
anything. Remove the requirement for the passed-in host rock to be
non-NULL, and make lih*_r more threadsafe, by passing in a parameter
struct for the rock.
Also attempt to make the GSS_r code a bit more clear with some
descriptive variable names and such.
Andrew Deason [Tue, 16 Feb 2010 17:08:38 +0000 (11:08 -0600)]
h_TossStuff_r: make sure host does not go away
When h_TossStuff_r h_NBLock_r's a host, it is not only possible for
someone else to grab a hold on the host, but in theory it's also
possible for someone to hold a host, release it, and for the host to be
deleted again (assuming some callers hold HOSTDELETED hosts, which they
should not be doing).
To make this safety check a bit more robust, hold the host in
h_TossStuff_r before h_NBLock_r'ing, to ensure that it does not go away
while we're waiting for H_LOCK.
Andrew Deason [Mon, 22 Feb 2010 17:56:07 +0000 (11:56 -0600)]
Prevent deadlock if fs_stateSave panics
fs_stateSave can call ShutDownAndCore(PANIC) if it detects host list
corruption. Right now this will deadlock the shutdown thread, since
we're still holding a read lock on the FS state when we do so, and
ShutDownAndCore will attempt to acquire a write lock. Do two things to
make this a bit better:
-- Unlock the FS state before calling fs_stateSave, sine we're no
longer looking at the FS state (the 'state' in fs_stateSave refers to
the host list and callback lists, not the state locked by
FS_STATE_*LOCK)
-- If ShutDownAndCore detects that it was called inside of
ShutDownAndCore, skip shutting down the volume and host packages, since
they have already been shut down
Andrew Deason [Fri, 12 Feb 2010 23:44:31 +0000 (17:44 -0600)]
Check for HOSTDELETED before h_Hold_r
A few places h_Hold_r a host and later drop and reacquire H_LOCK without
checking if the hostFlags contains HOSTDELETED. This can cause a race
with h_TossStuff_r where we later reference a host that is about to be
freed or already has been freed.
Add checks for HOSTDELETED in these places, and skip over the deleted
hosts.
Andrew Deason [Fri, 12 Feb 2010 22:30:44 +0000 (16:30 -0600)]
Correct the h_Enumerate_r hostList safety check
Ide1e5aca7c2c4a4af3f62bc07821db694f2f9999 added safety checks for a few
traversals through hostList, including the traversal in h_Enumerate_r.
Unfortunately, h_Enumerate_r may not hold H_LOCK over its entire
traversal (h_Release_r can drop and reacquire it), so the value of
hostCount is not guaranteed to stay the same.
A host may be deleted during the loop, or right near the end, decreasing
hostCount to below our current running 'count' of hosts, triggering the
panic unnecessarily.
Also, we break out of the loop if the H_ENUMERATE_ISSET_BAIL test
succeeds, making the panic also incorrectly trigger then.
So instead, remember the value of hostCount, and ensure that we've
actually exceeded that count in the post-loop check.
Andrew Deason [Mon, 15 Feb 2010 16:55:33 +0000 (10:55 -0600)]
h_TossStuff_r: check held-ness after lock
h_TossStuff_r checks if a host is held or locked by another thread
before trying to delete the host. Unfortunately, it checks if it is
locked before checking if it is held, and the lock check drops H_LOCK.
Thus, another thread could hold the host while we don't have H_LOCK, and
we could delete a host that is being held.
Although it is a bug if any thread holds a host that is being deleted,
some instances of this still exist, so make the check more robust.
Reverse the order of the tests, so we detect if someone held the host
while the lock check dropped H_LOCK.
Also log when this happens, as it indicates a bug occurring.
Derrick Brashear [Sun, 21 Feb 2010 22:15:26 +0000 (17:15 -0500)]
idledead client dont interrupt writes
allowing failover is nonsensical, there's nothing to fail to. keep trying to
write. if we want to change this, we need to handle it better than just this.
Simon Wilkinson [Wed, 17 Feb 2010 13:46:23 +0000 (13:46 +0000)]
Unix: Fix objdir builds
Make some changes to the Unix build system so that object directory
builds work again. In particular, change the way that we call
xcodebuild on Darwin, so that source and binary can live in different
places.
Andrew Deason [Wed, 17 Feb 2010 18:56:51 +0000 (12:56 -0600)]
Include stdint.h when using intptr_t
Some platforms such as OpenBSD require including stdint.h before you can
use intptr_t. So, include stdint.h (if it is available) whenever we use
intptr_t.
Andrew Deason [Wed, 2 Dec 2009 19:37:27 +0000 (13:37 -0600)]
salvager: avoid needing temp files to stay around
The salvager makes use of a couple of temporary files to store some
information while doing a salvage. Instead of referring to these files
by path name everywhere, pass around file handles instead. That way we
can unlink the files immediately, and they will be deleted on close.
This removes one of the roadblocks to allowing multiple salvages on the
same partition to occur at once (since otherwise other salvagers would
remove the temporary files on startup), and also makes it much less
likely that old temporary files will be left lying around in the first
place.
Andrew Deason [Thu, 4 Feb 2010 22:47:19 +0000 (16:47 -0600)]
DAFS: Maintain viced volume group hierarchy cache
When salvaging a volume (with DAFS or not), it is required to read the
volume headers of all volumes on the partition, so we know what volumes
are in the same volume group as the salvaged volume. Currently with
DAFS, this requirement can make demand-salvages very slow, since each
demand-salvage must read each volume header on the partition.
So, instead of having each demand-salvage read the volume headers
itself, have a demand-salvage request the required volume group
hierarchy information from the fileserver. The fileserver will scan the
partition's volume headers, and will keep the hierarchy cached in
memory. Any modifications to this hierarchy from volume
creation/deletion will update this volume group cache (VGC) via FSSYNC
commands.
This results in a dramatic salvaging speedup when many demand-salvages
are requested, and eliminates the cases where DAFS salvaging can be
significantly slower than non-DAFS salvaging.
Andrew Deason [Thu, 4 Feb 2010 22:08:48 +0000 (16:08 -0600)]
Add a general file-level locking API
Add an API for acquiring and releasing file-level locks. This abstracts
away the platform-specific aspects of file locking, and also makes the
locking work when several threads attempt to acquire a lock within the
same process.
This commit just adds the functions and datatypes; nothing actually uses
them. The users will be added in (hopefully nearby) future commits.
projects / packages / o / openafs.git / log