Andrew Deason [Wed, 24 Nov 2010 14:36:05 +0000 (09:36 -0500)]
ubik: Replay the transaction log label correctly
Commit eec0d94f519b3e27f255b9b7a637df043951424e fixed the transaction
replay log code to correctly identify valid transaction logs on
little-endian systems, but missed ntohl'ing the database label read in
a LOGEND opcode. Fix that, so the database is labelled correctly when
replayed from a transaction log.
And while we're here, actually pass a struct ubik_version* to
adbase->setlabel, to make it a little more clear what's happening.
Andrew Deason [Wed, 1 Sep 2010 20:10:56 +0000 (15:10 -0500)]
ubik: Record the last write tid in writeTidCounter
ubik is currently tracking writeTidCounter for write transactions
separately from regular transactions (assigned from tidCounter).
Specifically, tidCounter is incremented twice for each transaction,
but writeTidCounter is incremented twice only for write transactions.
As a result, writeTidCounter and tidCounter tend to drift far apart.
This is a problem, since the tid for DISK_* calls uses the transaction
id of the current transaction (based on tidCounter), and VOTE_Beacon
uses writeTidCounter for its transaction id. So, in effect, the tid in
VOTE_Beacon is completely bogus and unrelated to the transaction id of
the actual current write transaction. This can cause valid write
transactions to become invalidated when tidCounter becomes negative,
since VOTE_Beacon will send a positive tid, and if there is a current
in-flight write transaction with a negative tid, SVOTE_Beacon will
deem the transactions inequal and will abort the write transaction.
So instead, record the transaction id counter for the last write
transaction in writeTidCounter. This way, when we call VOTE_Beacon, we
will use the correct transaction id counter for the current write
transaction, and SVOTE_Beacon on the remote site will not invalidate
the transaction.
Andrew Deason [Wed, 8 Sep 2010 19:32:35 +0000 (14:32 -0500)]
DAFS: raise vhashsize limit
Raise the maximum specifiable vhashsize to 28 (from 14). Specifying a
vhashsize over 14 can be reasonable if you expect to have a few
million volumes on a fileserver.
Andrew Deason [Tue, 8 Mar 2011 22:59:32 +0000 (16:59 -0600)]
SOLARIS: Perform daemon syscalls as kernel threads
Add AFS_SUN5_ENV to the list of platforms where AFS_DAEMONOP_ENV is
defined. Implement the necessary functionality so we spawn kernel
threads when a daemon syscall is called. Remove the rxk_Listener
wrapper, since it will be called in a separate thread via the
afs_DaemonOp interface.
Andrew Deason [Tue, 8 Mar 2011 21:37:17 +0000 (15:37 -0600)]
libafs: Consolidate afs_DaemonOp code
Create the AFS_DAEMONOP_ENV define to simplify the logic of when we
perform afs_DaemonOp-y code paths. Also create the daemonOp_common
function, to perform common pre-fork operations that are common
between platforms.
Ben Kaduk [Sat, 8 Oct 2011 21:16:26 +0000 (17:16 -0400)]
FBSD: deal with kernel API rename
Upstream decided to rename the kernel functions that implement
syscalls to have a sys_prefix (including afs3_syscall!).
We use a couple of them, so we need to conditionalize accordingly.
Unfortunately, __FreeBSD_version was not bumped with the change,
so we use something close to it and hope it's close enough.
allow cloning of any volume to any volume with same parent ID
remove checks to disallow cloning of ro volumes to rw volumes,
which allows cloning of any volume within the same parent ID
grouping, including allowing destruction of newer version of the
volumes.
remove check for disallowing clones of backup or ro volumes
removes the if-statement ensuring that the volume being cloned is
not a backup volume, nor a read-only volume. This allows clones
from any type of volume to a given volume. Parent volume meta-data
is maintained, only the cloneId value changes.
Andrew Deason [Mon, 29 Aug 2011 22:41:31 +0000 (17:41 -0500)]
DAFS: Remove VOL_SALVAGE_INVALIDATE_HEADER
Currently VRequestSalvage_r takes a flag,
VOL_SALVAGE_INVALIDATE_HEADER, which causes the header for the
specified volume to be freed (via FreeVolumeHeader). This is almost
never safe to do, since there may be other users of the specified
volume that can be accessing the volume header at the same time.
There is also no reason to invalidate the header at the time of the
VRequestSalvage_r call, since the header must be invalidated when we
detach the volume (other utilities may change header information). So,
if there are any problems in the future because we do not invalidate
the header at the time of VRequestSalvage_r, it is the fault of the
detachment/offlining logic.
So, remove VOL_SALVAGE_INVALIDATE_HEADER and all of its users. Take
this opportunity to correctly document the VRequestSalvage_r headers
in the VRequestSalvage_r comment, as it was previously missing the
VOL_SALVAGE_NO_OFFLINE flag.
Michael Meffie [Thu, 13 Oct 2011 16:23:35 +0000 (12:23 -0400)]
DAFS: fssync online requires a partition name argument
fssync-debug online silently fails when run without a partition name.
Check for the required partition name on the server side and the client
side. Report errors back to the client when the server side fails to
pre-attach the volume.
Andrew Deason [Tue, 11 Oct 2011 15:51:14 +0000 (10:51 -0500)]
volser: Remove ExtractVolId
volser was using its own function to extract a volume ID from a
filename string, and was using atol to do so. The ato* family of
functions can have problems with larger volume IDs, not to mention a
lack of error checking, so don't use it. Since we already have the
function VolumeNumber in the vol package to do the very same thing,
just use that instead.
Andrew Deason [Mon, 3 Oct 2011 18:10:44 +0000 (13:10 -0500)]
viced: Check for HOSTDELETED in stillborn check
h_FindClient_r checks the connection rock for a client object twice.
First it sees if we already have a client object, and if we don't, we
effectively create one (or find a suitable one). Then we check again,
to see if someone else set the rock while we were creating a client
structure.
Currently, the first check checks if client->host->hostFlags has
HOSTDELETED set, but the second check does not. So, if the host
associated with the client has been deleted by someone else, currently
we will unnecessarily log a "stillborn client" message, and we will
continue to use the deleted host. If the host continues to be held by
someone, we will run into the same situation repeatedly on future
requests until all of the host references go away.
To fix this, also ignore HOSTDELETED clients when performing the
stillborn race check.
Andrew Deason [Fri, 14 Oct 2011 16:32:34 +0000 (11:32 -0500)]
vos offline: Bring volume back online for -busy
vos offline is supposed to bring a volume back online from "busy"
status before exiting, as volumes should not be in "busy" status for
extended periods of time. This was being enforced by required that
-sleep be specified; however, -sleep only results in the volume being
brought back online if a non-zero sleep time was specified. So, make
sure the volume is brought back online if -busy was specified.
do set errors when we bomb out early
do not unlock and return early when we happen to do a correct zero
length read
do set errors the kernel can deal with if we're feeding a page routine
Simon Wilkinson [Sun, 23 Oct 2011 23:07:33 +0000 (19:07 -0400)]
rpm: Turn on debugging
Now that we build with a blank CFLAGS line, we need to make sure and
actually turn on debugging in the build system, so that our debuginfo
files are vaguely useful
Simon Wilkinson [Wed, 12 Oct 2011 13:50:18 +0000 (09:50 -0400)]
rx: ackall handling
If we ACKALL a stream, then we're sending a hard ACK for all of the
packets in the stream. We shouldn't send that hard ACK, and then a
load of soft ACKs for packets that don't actually exist.
Andrew Deason [Fri, 12 Aug 2011 19:50:26 +0000 (14:50 -0500)]
LINUX: Revert group changes on keyring failure
On Linux kernels that support keyrings, when we setpag we try to add
the PAG to the session keyring and to the supplemental group list.
Currently, if we fail to add the PAG to the keyring (which may happen
due to key quotas, or possibly other reasons), we return failure but
the group list is still modified with the new PAG in it.
Therefore, if the keyring-based approach fails, the new PAG may still
be in use, but there are no keyring keys associated with that PAG, so
the PAG may never get destroyed. This can cause a large number of PAGs
to accumulate over time, causing performance problems.
So, change this so that, in the event that keyring installation fails,
we revert the group list back to what it was before we touched it.
Also mark all unixusers with the new PAG as expired, in case one got
created during processing. Thus, the new PAG never gets used.
Andrew Deason [Thu, 20 Oct 2011 21:57:14 +0000 (16:57 -0500)]
viced: Do not swallow errors on StoreData recovery
When we encounter any error in the StoreData fetch/store loop, we
reset the disk usage to ensure it remains correct, even in the face of
unexpected errors. However, when we do so, we use the errorCode from
VAdjustDiskUsage as our return value; if it is 0, we return success,
ignoring the error that got us in this code path in the first place.
Instead, keep track of a temporary errorCode for the disk usage
adjustment, and do not override our return value if there was no error
in the disk usage numbers.
Simon Wilkinson [Wed, 12 Oct 2011 17:04:28 +0000 (13:04 -0400)]
ukernel: add morepackets check in listener
Make the listener loop actually check for more packets needed,
like kernel, pthreads and lwp. Only checking for new packets every
20 seconds isn't sufficient on today's networks!
Simon Wilkinson [Mon, 10 Oct 2011 22:19:13 +0000 (17:19 -0500)]
docs: Refer to dafs binaries by their real names
(Most of) the dafs binaries are called da(something). Update the
example in the dafileserver documentation so that we call the binaries
by the names that they are actually installed with on the system.
Reviewed-on: http://gerrit.openafs.org/5654 Tested-by: Simon Wilkinson <sxw@inf.ed.ac.uk> Reviewed-by: Derrick Brashear <shadow@dementix.org>
(cherry picked from commit c37a14697df291f641b4fa5a32826054d92210b0)
Simon Wilkinson [Mon, 10 Oct 2011 21:09:40 +0000 (22:09 +0100)]
ptserver: Don't check for noauth before rebuilding
The ptserver database building scripts would check to see if the server
was running from a bosserver with the noauth flag set before performing
a database rebuild.
This means that you can't start ptserver normally, and then configure
the database using pts -localauth, which is the preferred method for
configuring new cells.
Remove the check for noauth. This is slightly risky, as it means that a
corrupt database could be completely erased upon restart. However, we
already check that the dbheader (65k) is entirely blank - which will
protect us against any single page corruption errors.
Reviewed-on: http://gerrit.openafs.org/5653 Tested-by: Simon Wilkinson <sxw@inf.ed.ac.uk> Reviewed-by: Derrick Brashear <shadow@dementix.org>
(cherry picked from commit 7a4498850814ea524f55de0b84d6b0ae0b4834bb)
Hartmut Reuter [Wed, 5 Oct 2011 14:06:05 +0000 (10:06 -0400)]
vol_split: avoid using stale open directory vnodes
we could in case of multiple splits end up using a stale open
vnode for a directory; attempt to close and thus force-reopen
any fdhandles backing ihandles.
Andrew Deason [Wed, 28 Sep 2011 20:02:48 +0000 (15:02 -0500)]
vol: Only check "logging" on vice partitions
We don't care about non-vicepX partitions, so move part of the UFS
"logging" check into VCheckPartition. This API should probably redone
so the "am I a vicepX partition" check is done completely separately,
but for now, this will do.
Reviewed-on: http://gerrit.openafs.org/5514 Reviewed-by: Derrick Brashear <shadow@dementix.org> Tested-by: Andrew Deason <adeason@sinenomine.net> Tested-by: BuildBot <buildbot@rampaginggeek.com>
(cherry picked from commit 82481c6f4464ad978ea56c687205bdfcd3eebffd)
Rod Widdowson [Thu, 29 Sep 2011 14:34:48 +0000 (15:34 +0100)]
FSSYNC-Client: Consistent use of partition name
Over time the FSSYNC code has collected examples where the partition
path is passed rather than the partition name. In Unix this is the
same (/vicepX), but on windows the path is the DOS device (C:).
This checkin changes FSSYNC client code to always use the partition
name.
This checkin does not address FSSYNC server or SALVSYNC.
Andrew Deason [Thu, 29 Sep 2011 19:49:53 +0000 (14:49 -0500)]
DAFS: Do not serialize state for invalid hosts
When we serialize host information for DAFS during shutdown, we have
no guarantee that the host is in a valid state when we look at it.
This can result in a host being saved to disk when we are waiting for
the host to respond to an RPC, and so the information about the host
is invalid. For example, we can save a host that has the
HWHO_INPROGRESS flag set, and when it is restored later, this can
cause odd behavior since the flag is set but no thread is actually
waiting for the host to respond.
So instead, during state serialization, try to determine if a host may
be in an invalid state, and simply skip the host if it may.
Andrew Deason [Thu, 29 Sep 2011 21:04:54 +0000 (16:04 -0500)]
DAFS: Skip hosts with invalid flags on restore
Host entries with HWHO_INPROGRESS set or ALTADDR unset do not have
valid state, since those flags indicate that the fileserver was in the
middle of identifying the host when the host struct was serialized.
Skip entries from the on-disk host data that have such invalid flags
set when restoring state, so we do not load invalid data.
Patchset 718f85a8b69a78ac77beb5c8471af20657be2a53 contained
a small typo that prevents the SetSPrefs pioctl processing from
functioning in all cases. fs setserverprefs continues to work
for non-DB preference lists because fs.c pokeServers() attempts
to try the old SetSPrefs33 pioctl in the non-DB server case.
Andrew Deason [Thu, 29 Sep 2011 20:22:35 +0000 (15:22 -0500)]
DAFS: Add explicit 'valid' field for index maps
The CB, FE, and host serialization structures were just using the
relevant indices to determine whether or not an entry mapping and old
index to a new index was populated with actual data. For host
structures, this really isn't sufficient, since our index can be 0,
and the structure is calloc'd, so the index in the structure could
also be 0.
Add a flag explicitly stating whether or not the structure has been
filled in, to make this unambiguous.
Michael Meffie [Thu, 22 Sep 2011 13:13:18 +0000 (09:13 -0400)]
bozo: report bosserver -rxbind address
Create a file on bosserver startup called bosserver.rxbind in the
server local directory which contains an address local scripts may
use to contact the bosserver.
When bosserver is started with the -rxbind option, write the address
selected from the intersection of the interfaces, NetInfo, and
NetRestrict configuration to the bosserver.rxbind file, otherwise
write the loopback address 127.0.0.1.
Update the RedHat init script to use the new bosserver.rxbind file.
Michael Meffie [Wed, 24 Nov 2010 01:21:50 +0000 (20:21 -0500)]
bozo: bosserver -pidfiles option
Add an option to bosserver to create pidfiles for long running
processes for simple, fs, and dafs bnode types, as well as the
bosserver process. The pidfiles are located in the server local
directory by default, or in the path specifed by the -pidfiles
command-line option.
Marc Dionne [Thu, 22 Sep 2011 00:31:33 +0000 (20:31 -0400)]
fs: fix setserverprefs where long is larger than afs_int32
Make the fscanf and scanf format specifiers match the type of the
target variable. This prevents trying to store a long int into an
afs_int32 variable which may be smaller, and overwriting neighbouring
data on the stack.
The effect on a 64-bit Linux system was that the high bits of "rank"
would overwrite the first 4 bytes of the scanned host name, causing
setserverprefs to fail with:
: couldn't resolve name.
Marc Dionne [Thu, 29 Sep 2011 01:15:32 +0000 (21:15 -0400)]
rx: add post RPC procedure capability
Add the ability to specify a procedure that will be called after
the end of each RPC for a service. This is similar to the
existing afterProc, except that it gets called after the RPC
has ended (after EndCall).
rx_SetPostProc and rx_GetPostProc are provided to set and retrieve
a postProc for a specified service.
Andrew Deason [Thu, 29 Sep 2011 17:04:07 +0000 (12:04 -0500)]
afs: Remove afs1_readdir
The function afs1_readdir is not referenced anywhere. Remove it, as it
currently causes compilation errors (a typo with the
origOffset/orginOffset variable name).
Michael Meffie [Tue, 20 Sep 2011 20:21:50 +0000 (16:21 -0400)]
salvager: fix filesystem path truncation
Fix filesystem path truncation for vice partitions beyond /vicepz.
Commit 3d7388b7 moved the filesystem path to a fixed size buffer,
however the path of the filesystem to be salvaged is truncated for
partitions with two character ids (e.g. /vicepaa), in which case
the salvager will salvager the wrong partition, or abort if the
truncated path is not present.
shutdown_rx, unlike rx_Finalize, kills the socket before the conns.
since we call osi_NetSend directly, we lose. just do a simple
check for rxinit_status, and exit immediately before sending if rx
is not up.
Andrew Deason [Wed, 24 Aug 2011 17:48:19 +0000 (12:48 -0500)]
ihandle: Fix IH_REALLYCLOSE for positional I/O
Currently, ih_fdclose (which is called by IH_REALLYCLOSE), goes
through every FD_HANDLE_OPEN FdHandle_t and closes it. If it finds
handles that are FD_HANDLE_INUSE, it skips those and sets a flag on
the parent IHandle_t. For non-positional I/O, any future opens cannot
use these _INUSE handles, since _INUSE handles cannot be reused, and
the handle will be actually closed when it is FDH_CLOSE'd.
For positional I/O, the situation is different. Multiple threads can
use the same _INUSE FdHandle_t, and so there is nothing currently
stopping a thread from IH_OPEN'ing an ihandle that has been
IH_REALLYCLOSE'd, and getting back an FdHandle_t that existed before
the IH_REALLYCLOSE was issued. This is important, since IH_REALLYCLOSE
is used on files that are deleted, and future IH_OPENs for the same
inode must not use the cached file descriptor. Getting this wrong can
cause data loss, since it can cause us to read from or write to a file
descriptor referring to a deleted file, when we instead should open a
new copy of that file.
To fix this, we create a new FdHandle_t state called
FD_HANDLE_CLOSING, which is set in IH_REALLYCLOSE if we encounter an
FD_HANDLE_INUSE FdHandle_t. In IH_OPEN, we always skip
FD_HANDLE_CLOSING handles, so we can never get back a cached file
descriptor from before an IH_REALLYCLOSE call.
Russ Allbery [Tue, 23 Aug 2011 19:50:55 +0000 (12:50 -0700)]
Generate stub header files for h/*.h files included in libuafs
Previously, the libuafs build created a symlink from h to
/usr/include/sys so that files included under h/* by kernel source
files could be found in the normal system header location. However,
this assumption about the system header location is no longer valid.
Debian and Ubuntu systems with multiarch have arch-specific include
paths so that the same host can be used to build 32-bit and 64-bit
binaries with different system headers, and those include paths are
automatically searched by the compiler. This means some standard
headers are no longer found directly in /usr/include/sys but are
instead found in /usr/include/<arch>/sys.
Using a stripped-down version of similar code for building the kernel
module on Linux, create an h directory containing stub header files
that just include the relevant system <sys/*.h> header file instead.
This allows the compiler to implement its normal internal header
search algorithm.
Also remove all the other symlinks, such as sys, netinet, etc., that
just pointed to the same directories under /usr/include. We can assume
the normal compiler search algorithm will find these headers without
requiring this assistance.
Fixes breakage on freebsd for missing malloc.h, reported by GAWollman,
and, since roken.h already includes stdlib.h to pull in malloc, is no
longer necessary
Andrew Deason [Tue, 26 Apr 2011 19:44:46 +0000 (14:44 -0500)]
Build libafscp when we lack kerberos
Currently, venus fails to build without kerberos, since the
dependencies for afsio always include afscp.h, which does not exist
when we do not build libafscp. To fix this the easy way, and since
libafscp is still very useful without kerberos, allow libafscp to
build without kerberos support (which limits it to anonymous
connections only).
afsio is a utility for file transfer to and from AFS file space
without the help of the AFS client/cache manager. Using libafscp,
this (partially rewritten) version of afsio is able to accomplish
(1) authenticated access to an AFS path or FID (an existing
KerberosV ticket is required), (2) fall back on unauthenticated
("anonymous") access if authentication (token acquisition) fails,
and (3) work independtly of the AFS cache manager (afsd need not
be running, though CellServDB and ThisCell are currently required).
issues:
1) libvldbint and libafsint are not compiled pthreaded. we link in
what we need. this should be changed when we are all-pthreaded.
2) venus is not a pthreaded-directory otherwise. same deal:
in an all-pthreaded universe, undo the bodge that we do here.
3) venus is not an all-krb5 directory either. slight ick.
This patch is intended to bring libafscp into accordance with the
current OpenAFS coding standards while also fixing a few small
issues. Apologies in advance for the numerous whitespace changes.
libafscp provides the ability to accomplish many of the functions of an AFS
client without a running afsd cache manager. It is being introduced into
the OpenAFS source tree in order to improve the capabilities of several
utilities but may have additional benefits for testing clients and servers
and for use on platforms which do not have afsd support.
projects / packages / o / openafs.git / log