Andrew Deason [Wed, 1 Sep 2010 20:10:56 +0000 (15:10 -0500)]
ubik: Record the last write tid in writeTidCounter
ubik is currently tracking writeTidCounter for write transactions
separately from regular transactions (assigned from tidCounter).
Specifically, tidCounter is incremented twice for each transaction,
but writeTidCounter is incremented twice only for write transactions.
As a result, writeTidCounter and tidCounter tend to drift far apart.
This is a problem, since the tid for DISK_* calls uses the transaction
id of the current transaction (based on tidCounter), and VOTE_Beacon
uses writeTidCounter for its transaction id. So, in effect, the tid in
VOTE_Beacon is completely bogus and unrelated to the transaction id of
the actual current write transaction. This can cause valid write
transactions to become invalidated when tidCounter becomes negative,
since VOTE_Beacon will send a positive tid, and if there is a current
in-flight write transaction with a negative tid, SVOTE_Beacon will
deem the transactions inequal and will abort the write transaction.
So instead, record the transaction id counter for the last write
transaction in writeTidCounter. This way, when we call VOTE_Beacon, we
will use the correct transaction id counter for the current write
transaction, and SVOTE_Beacon on the remote site will not invalidate
the transaction.
Andrew Deason [Wed, 10 Nov 2010 21:35:17 +0000 (15:35 -0600)]
afscp: Add -s option
Add an -s option to afscp, to specify an amount of time to sleep in
the middle of a read or write operation. This can be helpful in
simulating a slow client.
Permit a popular path through afs_linux_dentry_revalidate to pass without
taking a lock which it actually does not need. This affects multi-core
software-build nodes in particular, where serialization and high stat() counts
restricts useful processing to a single core.
Andrew Deason [Tue, 12 Oct 2010 22:46:36 +0000 (17:46 -0500)]
viced: Add options for interrupting clients
Add the -offline-timeout and -offline-shutdown-timeout options to the
fileserver, to implement interrupting clients accessing volumes we are
trying to take the volume offline. Document the new options.
Currently this is only implemented for read operations. Implementing
this for write operations and callback breaks will require more work.
This also removes the VGetVolumeTimed interface from the volume
package, since the fileserver was the only user and with this change
the fileserver now uses the VGetVolumeWithCall interface.
Andrew Deason [Fri, 29 Oct 2010 16:29:37 +0000 (11:29 -0500)]
vol: Interrupt RX calls accessing offlining vols
When we are waiting for a volume to go offline, only wait a certain
amount of time for it to go offline before we interrupt all RX calls
associated with that volume. This amount of time is configurable in
the new offline_timeout and offline_shutdown_timeout volume package
option fields.
Andrew Deason [Fri, 29 Oct 2010 16:14:49 +0000 (11:14 -0500)]
vol: Add interfaces for registering RX calls
Add VGetVolumeWithCall and VPutVolumeWithCall, to associate RX calls
with volume heavyweight references. Also add the interrupt_rxcall
field to the volume package options structure.
This also adds the VIsGoingOffline function, so a caller can tell when
a volume is going offline.
Andrew Deason [Wed, 27 Oct 2010 22:34:40 +0000 (17:34 -0500)]
vol: Add VGetVolumeTimed
Replace the VGetVolumeNoWait interface with the more general
VGetVolumeTimed interface, which allows for waiting for offlining
volume for arbitrary amounts of time (instead of just "waiting
forever" or "not waiting at all"). Also add VOL_CV_TIMEDWAIT and
VTimedWaitStateChange_r as necessary to implement this.
Andrew Deason [Thu, 30 Sep 2010 17:39:22 +0000 (12:39 -0500)]
viced: Allow checkout of VOL_STATE_DELETED volumes
The fileserver allows nonexistent volumes to be checked out. Since
VOL_STATE_DELETED volumes conceptually do not exist, we should also
allow them to be checked out.
Change-Id: Ibc0f761569b339579acff03a3cbd9f795f783e0a
Reviewed-on: http://gerrit.openafs.org/2873 Reviewed-by: Tom Keiser <tkeiser@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Andrew Deason [Fri, 12 Nov 2010 16:03:43 +0000 (10:03 -0600)]
doc: Fix fileserver synopsis
The fileserver synopsis was missing the leading 'fileserver' before
the options list. This causes the options list to not be interpreted
as POD, and so you get a lot of ugly unprocessed POD markup in the man
page. Fix that.
Andrew Deason [Thu, 11 Nov 2010 20:45:04 +0000 (14:45 -0600)]
vos release: Avoid full dump on all sites
Commit 9fed169b1c5c823fd96cea94daf712b2cf06c901 attempts to remove any
remote RO site that has RO_DONTUSE set. However, we set RO_DONTUSE on
all sites earlier in the release process if this is a full release,
resulting in all sites getting a full dump.
Correct this by remembering in 'origflags' what flags each site has
before we messed with the vldb entry.
Thanks to Mike Meffie.
Change-Id: I8c9582c9f7aaee92637ddc1ad11aed96fd69686b
Reviewed-on: http://gerrit.openafs.org/3296 Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Michael Meffie <mmeffie@sinenomine.net> Tested-by: Michael Meffie <mmeffie@sinenomine.net>
Andrew Deason [Thu, 29 Jul 2010 22:02:38 +0000 (17:02 -0500)]
Fix util test dependencies
The test programs in tests/util did not depend on libutil.a or
libtap.a. So, if libutil.a changed, they were not relinked. Add the
dependencies so correcting a part of libutil will cause the tests to
actually reflect the change.
Andrew Deason [Wed, 29 Sep 2010 16:48:44 +0000 (11:48 -0500)]
salvager: Do not break cbks when salvaging parts
When salvaging whole partitions / whole servers, the fileserver is
assumed to not be running. So only break callbacks if we are salvaging
a single volume. If we are salvaging a whole partition, do not tell
the fileserver to break callbacks on changed volumes, since the
fileserver will not be around and we will just hang.
Tom Keiser [Thu, 14 Oct 2010 05:24:03 +0000 (01:24 -0400)]
vol: make namei_ListAFSSubDirs deal with multiple/bad linktables
The salvager ends up deadlocking when multiple linktables exist
in the same volume group special directory. The issue is that we
open and flock all discovered linktables, but only close out the
last one found. Consequently, when our child scans the linktables
again, we deadlock against the locked and leaked descriptor(s) our
parent left around before forking.
While we have so far been unable to root-cause the actual creation
of spurious linktables, this patch will at least stop the salvager
from deadlocking against itself when this occurs.
Marc Dionne [Sun, 7 Nov 2010 18:14:55 +0000 (13:14 -0500)]
Cache bypass: make readpage deal with reads at end of file
When a file's size is an exact multiple of the page size, the vfs
will issue a readpage for an extra page at the end, for which there
is no data. Deal with it here instead of letting it trickle down
to the background daemon, which will issue an unnecessary read to the
server, and maybe get confused because there is no data.
Change-Id: If86ed785c4e9d50edb1a960142f726fa869c50fc
Reviewed-on: http://gerrit.openafs.org/3281 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Andrew Deason [Thu, 30 Sep 2010 16:34:59 +0000 (11:34 -0500)]
volser: Do not FSYNC_VOL_DONE temporary volumes
Currently, the volserver just looks at the destroyMe field in a volume
header to see if it should tell the fileserver a volume has been
deleted (FSYNC_VOL_DONE opcode). However, temporary volumes created by
the volserver (such as new volumes or clones) have destroyMe set, but
are obviously not deleted, and so the fileserver should just be told
FSYNC_VOL_LEAVE_OFF about these volumes instead.
Note that this commit slightly changes the meaning of the
vp->needsPutBack field. Now it must be set to VOL_PUTBACK_DELETE to
tell the fileserver that the volume has been deleted, and set to
VOL_PUTBACK if it just needs to be given back to the fileserver.
Marc Dionne [Sun, 7 Nov 2010 18:35:36 +0000 (13:35 -0500)]
Cache bypass: adjust read size for non-contiguous readpages
The bypass code's readpages does deal with a non-contiguous list
of pages, but it doesn't adjust the size of the read it requests
from the background daemon accordingly. As a result we'll ask the
server for pages we were not asked to read.
Change-Id: I5468a4a68f4f415961298144f5bb581faeda540a
Reviewed-on: http://gerrit.openafs.org/3282 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: BuildBot <buildbot@rampaginggeek.com>
Andrew Deason [Fri, 5 Nov 2010 21:48:28 +0000 (16:48 -0500)]
vol: Do not give back not-checked-out vols
VAttachVolumeByName_r has logic to give back a volume over FSSYNC if
we checked out a volume but failed to attach it for whatever reason.
However, the logic used for determining if the volume was checked out
or not is a bit inaccurate (even moreso than the comments imply),
potentially causing us to VOL_ON volumes that don't exist at all.
Instead of trying to guess based on various conditions whether or not
we checked out the volume, keep track of a variable that is only set
when we actually checkout the volume from the fileserver. Then only
give back the volume if it is set.
Marc Dionne [Sun, 7 Nov 2010 20:06:16 +0000 (15:06 -0500)]
Cache bypass: release and unlock pages when we get 0-length reply
In some cases, such as reading past the end of file as known to the
server, fetchdata will get a 0 length reply. Deal gracefully by
unlocking and releasing any pages reserved by readpages. Failure
to do so here leads to deadlocks later as we can exit with some pages
still locked.
Change-Id: Ic9e496c8c64d5e94f2cf9976d160ef5873f27249
Reviewed-on: http://gerrit.openafs.org/3283 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Simon Wilkinson [Fri, 1 Oct 2010 08:36:06 +0000 (09:36 +0100)]
rx: Reorganise includes
RX files were including the same header set in three different places,
once for user-land builds, once for kernel builds and once for ukernel.
The duplication was a bit pointless, and really frustrating when adding
new headers. So, reorganise the includes so that we only list headers
that are used in all three builds in one location.
Also take the opportunity to indent the #ifdefs so that it is clear
what is going on, and to remove some more of AFS_OSF_ENV and
AFS_AUX_ENV from kernel builds.
Michael Meffie [Thu, 4 Nov 2010 13:26:25 +0000 (09:26 -0400)]
avoid private stdio fields in waitkey
Use the stdio_ext functions provided by solaris and glibc
instead of directly accessing private stdio FILE structure
members. This is needed for 64-bit solaris builds and is more
portable in general since the FILE structure is meant to be
opaque. Remove the duplicated code in the pthreaded butc
package.
Andrew Deason [Fri, 5 Nov 2010 19:34:05 +0000 (14:34 -0500)]
DAFS: Do not let VScheduleSalvage_r free vp
In VScheduleSalvage_r, we were calling VCancelReservation_r, which has
the possibility of free'ing vp. Since we still use vp after doing this
(and since we're already inside VCancelReservation_r to begin with),
we must not free vp. Instead, just decrement nWaiters without
triggering any of the dtor code in VCancelReservation_r. This is safe
as long as all VScheduleSalvage_r callers ensure that they check to
free the vp if necessary, which they all do.
RedHat: Remove potpourri.h from RedHat SPEC file manifest
potpourri.h is no longer needed, so it should be removed from the
list of files to be installed. The missing file was causing the
RPM package build process to fail.
Marc Dionne [Fri, 5 Nov 2010 00:49:41 +0000 (20:49 -0400)]
Cache bypass: Fix oops in bypass transition functions
The FCSBypass flag might change between the time we check it before
entering afs_TransitionToCaching or afs_TransitionToBypass and when
we check it again within the functions.
Instead of panicing, just exit if someone beat us to it. Also move
the checks within the write lock region to make sure the code
doesn't get run multiple times.
Andrew Deason [Thu, 4 Nov 2010 16:37:31 +0000 (11:37 -0500)]
tsm: Make explicit rules for stem-changing targets
If the stem changes for generating a .o from a .c file,
implicit/suffix rules will not apply. That is, if we specify the
dependencies for 'bar.o' in a Makefile as so:
bar.o: foo.c
This will not match an implicit .c.o rule, since the filename stem
changes. We must add an explicit AFS_CCRULE invocation to make the
rule do anything.
This fixes this problem for the src/tsm41 directory. It may exist
elsewhere, but it's difficult to find unless the build breaks.
Andrew Deason [Thu, 4 Nov 2010 15:57:30 +0000 (10:57 -0500)]
UKERNEL: f_fsid is a struct on AIX
The statvfs f_fsid member is a struct with two fields on AIX when
_ALL_SOURCE is defined (which we define to get various extensions). So
in the AIX case, use it as such a struct instead of a single integer.
Andrew Deason [Thu, 4 Nov 2010 15:30:31 +0000 (10:30 -0500)]
Use termios.h for winsize test where available
The configure test for 'winsize' existence was using sys/termios.h,
but some platforms (AIX) have termios.h but no sys/termios.h. So, use
termios.h instead where available.
Hartmut Reuter [Tue, 2 Nov 2010 11:15:42 +0000 (12:15 +0100)]
Make osi_fetchstore.c protocol independent
For future use of OSD and vicep-access osi_fetchstore.c should not depend on
the rx-fileserver-protocol but call instead the routines pointed to by ops.
Some code beautyfication in afs_fetchstore.c to use nBytes instead of code.
New global variable afs_protocols in afs_fetchstore.c which will be used
in RXOSD/VICEP-ACCESS programs in the future.
as discovered by Benjamin Kaduk, we were usually holding rx_refcnt_mutex
but briefly, and here we held it longer, and thus around acquiring freepktQ
mutex. undo it by simply setting STATE_RESET sooner as newcall does.
Jeffrey Altman [Tue, 2 Nov 2010 20:16:20 +0000 (16:16 -0400)]
Windows: Do not leak cm_volume_t objects from the LRU queue
During cm_volume_t object recycling the object is removed
from the LRU to ensure that a single object is not recycled
by multiple threads at the same time. Before cm_FindVolumeByName()
exits the object must be re-inserted into the LRU if it is not
present.
Jeffrey Altman [Thu, 28 Oct 2010 04:37:03 +0000 (00:37 -0400)]
vol: attach2 must always return with VOL_LOCK held
attach2() is required to return with the VOL_LOCK held
even though it is called without it. This must be true
for error conditions as well. Not all error paths are
obtaining the VOL_LOCK before returning. Add out paths for
lock held and lock unheld error cases.
Andrew Deason [Fri, 2 Jul 2010 21:57:42 +0000 (16:57 -0500)]
DAFS: Fix demand-salvages of attached volumes
Currently, when an error is encountered for an attached volume, we
call VRequestSalvage_r, which makes the volume go into the
VOL_STATE_SALVAGING state. This state implies that the volume is
offline, however, which is not necessarily the case if we're calling
VRequestSalvage_r from, for example, VAllocVnode_r or VUpdateVolume_r.
So now, make a new state called VOL_STATE_SALVAGE_REQ to indicate when
a salvage has been requested but the volume is not offline yet (and
thus is not yet ready to give to the salvager). If VCheckSalvage finds
a volume in this state, it offlines the volume first. The FSSYNC
VOL_OFF handler now checks for this state, and if we're giving the
volume to the salvager, we wait for the volume to exit that state.
VRequestSalvage_r also gains a new flag, VOL_SALVAGE_NO_OFFLINE. This
is to ensure that the existing salvaging code paths for unattached
volumes does not change (for when VRequesetSalvage_r is called from
attach2). If this flag is passed, we do what we used to do, which is
just salvage the volume without offlining it.
Andrew Deason [Mon, 1 Nov 2010 20:34:26 +0000 (15:34 -0500)]
Cleanup VOffline log message for non-DAFS
Commit fd592c7674d4aa44dda90998b54d7b56947f6ed8 fixed the 'Volume X
(Y) is now offline' message for DAFS, but the same problem persists
for non-DAFS. Fix the non-DAFS case.
Simon Wilkinson [Fri, 29 Oct 2010 11:10:16 +0000 (12:10 +0100)]
Add libroken as its own library
Include libroken as a library in its own right, so that the whole
of the code can benefit from it. This change purely adds libroken
for the Unix build system. It doesn't replace those pieces of
libroken in hcrypto or util, or enable it for Windows.
There is also the option of using a system-install libroken, if one is
found at configure time.
*) If --with-libroken=yes, or is not supplied than a system library
will be used if suitable. Otherwise, we'll use the internal
libroken
*) If --with-libroken=/path/to/installation then the libroken at
that path will be used. If there is no libroken there, or it
is not suitable, an error will be returned
*) If --with-libroken=internal then the internal libroken is used,
regardless of what is present on the system.
We deliberately do not provide installed headers for the internal
libroken. If other applications wish to make use of libroken, then
they should use the Heimdal one, rather than piggybacking on ours.
Phillip Moore [Tue, 19 Oct 2010 16:17:20 +0000 (12:17 -0400)]
Fix fs bypassthreshold to accept a size of -1 to disable
The fs bypassthreshold command assumes a value of -1 means the feature
is disabled, but the CLI refused to accept this argument, since it is
not strictly a digit (according ti isdigit()). This patch accepts the
string -1, and makes it possible to both enable AND disable this
feature.
Phillip Moore [Tue, 19 Oct 2010 15:31:47 +0000 (11:31 -0400)]
fs getfid defaults to '.', like other path-related commands
I noticed that all of the other commands that accept a list of paths
use the SetDotDefault() function to default to ".", when no arguments
are given. This patch adds that call to getfid, making it more
consistent with similar commands.
Phillip Moore [Tue, 19 Oct 2010 15:23:46 +0000 (11:23 -0400)]
fs getfid output changed for consistency with Windows implementation
This patch removes the redundant volume ID from the output of fs
getfid, and replaces it with the cell name, which is what the Windows
implementation provides.
Phillip Moore [Tue, 19 Oct 2010 12:24:41 +0000 (08:24 -0400)]
Makes fs getfid error handling consistent with other fs commands
This patch makes the fs getfid command print errors for paths that
can't be handled correctly, instead of quietly ignoring them, and it
also returns an error code if any such paths are encountered. This
makes the behavior consistent with other fs commands, such as
listquota, whereis, etc.
Ben Kaduk [Fri, 29 Oct 2010 07:18:02 +0000 (03:18 -0400)]
FBSD: correct and simplify vcache eviction routines
osi_VM_FlushVCache and osi_TryEvictVCache were both attempting
to be wrappers around vgone(), with some checks before hand.
Implement the latter in terms of the former to prevent
code duplication and propagation of incorrect code.
Additionally, correct the locking around vgone(). The
vnode lock must be held, and we must also increase the vnode's
hold count so that it does not disappear out from under us.
As we need the interlock to check the usecount, keep it
locked until we lock the vnode lock, for extra protection.
As an added bonus, we no longer try to call vgonel(), which
is not an exported symbol and merely happened to work due
to the current kernel linker implementation.
Remove some stale comments.
With this change, a parallel buildworld completes on
my four-core machine.
Ben Kaduk [Fri, 29 Oct 2010 16:01:04 +0000 (12:01 -0400)]
FBSD: lock interlock around v_usecount accesses
The FreeBSD vnode locking strategy requires that the vnode
interlock be held for all accesses to v_usecount, such as those
used by our VREFCOUNT and VREFCOUNT_GT macros. Conveniently,
a wrapper function is provided that takes the lock around its
access of the element, vrefcnt(). Use it for our macros.
afs_osi_Alloc_NoSleep() is no longer used by the SOLARIS or IRIX
clients. It is used by the *BSD code in rx, so just let those
platforms define/prototype it in their osi_machdep.h
Rod Widdowson [Fri, 29 Oct 2010 17:01:13 +0000 (18:01 +0100)]
windows: terminate multi_sz correctly
CreateProcess requires a null-terminated list of null-terminated strings
as an environment parameter.
A missing level of indirection was causing the final null to be
missed, meaning that if bosserver ran from somewhere which had an
environment the create process would fail.
Jeffrey Altman [Thu, 28 Oct 2010 05:19:17 +0000 (01:19 -0400)]
Windows: Finish converting vol apps to pthread only
The src/vol directory on Windows is one of the rare examples
where a single directory builds both lwp and pthreaded versions
of libraries and executables. With this patchset the executables
are fully converted from lwp to pthread. This requires that
afsrpc.dll include the pthread implementations of the threadname,
fasttime, and lock implementations from the LWP directory.
The inclusion within afsrpc.dll permits the dviced and
dvolser directories to avoid rebuilding those object modules.
Jeffrey Altman [Thu, 28 Oct 2010 04:43:26 +0000 (00:43 -0400)]
vol: Use OSI_NULLSOCKET and not -1 to indicate invalid fssync fd
The FSync file descriptor is an osi_socket which has an invalid
value of OSI_NULLSOCKET which is not necessarily -1. Be sure to
compare against OSI_NULLSOCKET and not -1 when checking an invalid
value.
Jeffrey Altman [Thu, 28 Oct 2010 04:40:32 +0000 (00:40 -0400)]
vol: Always use INVALID_FD to indicate an invalid fd
file descriptors on Windows are not ints and therefore
cannot be safely compared against -1. Always use INVALID_FD
which is -1 on UNIX and INVALID_HANDLE_VALUE on Windows.
Simon Wilkinson [Fri, 29 Oct 2010 11:40:31 +0000 (12:40 +0100)]
shlib-build: Add ignore option
Add an option to shlib-build to ignore missing symbols in the map file.
This is already the default on some platforms, but others (such as
Darwin) require that all symbols in the mapfile be present in the
objects. This is a pain for libraries such as libroken, which will
have different symbols on different platforms.
Specifying -i adds the necessary magic to Darwin's ld to relax this
check. Changes may also be necessary for other platforms, but I
don't currently have those available for testing.
Marc Dionne [Wed, 27 Oct 2010 00:33:41 +0000 (20:33 -0400)]
bucoord: parallel make fix
Fix an instance of a Makefile rule with multiple targets.
This can cause a parallel make to fail when two instances of
compile_et compete to write the same output files.
Spotted by a build failure with a corrupt bc.h header.
Rod Widdowson [Thu, 28 Oct 2010 17:34:41 +0000 (18:34 +0100)]
Windows: fix built in touch
Recent versions of windows add a whole bunch of attributes above
A_ARCH. (FILE_ATTRIBUTE_NOT_CONTENT_INDEXED was what bit be but
encryption of compression would do it).
This makes ~_A_ARCH not a good choice for testing nonwritability
of a file - so files with these new attributes just get silently ignored.
Using an explicit mask is much better. So do that.
Simon Wilkinson [Sun, 24 Oct 2010 10:50:25 +0000 (11:50 +0100)]
Improve commit messages for git imports
Improve the quality of the commit messages produced by git import
by adding an explicit author (obtained from the $module-author file),
and by including a list of all of the upstream changes that are being
imported.
Derrick Brashear [Mon, 18 Oct 2010 03:39:47 +0000 (23:39 -0400)]
down with assert, up with osi_Assert
because NDEBUG breaks things which happen inside an assert,
be done with that. instead, call osi_Assert wherever possible.
doesn't work for code which builds before rx; those cases we handle
by ensuring no operations happen inside the assert(). side effect:
move all pthread operations wrapped in asserts to MUTEX_mumble and
CV_mumble calls where those exist, so the assertions happen all in
one set of macroes.
Simon Wilkinson [Mon, 25 Oct 2010 09:14:12 +0000 (10:14 +0100)]
rx: Don't let timeouts force fast recovery
The current RX implementation goes into fast recovery whenever a
timeout occurs. This is incredibly wasteful, particularly on fast
connections. So, remove this in favour of TCP style behaviour.
projects / packages / o / openafs.git / log