Michael Meffie [Tue, 17 Apr 2012 02:29:24 +0000 (22:29 -0400)]
bozo: increase salvage instance poll rate
Increase the bos client poll rate of the salvager temporary bnode
instance status, from every 5 seconds to 1 second. This reduces the
minimum time bos salvage takes, from 5 seconds to 1 second, which
can add up when doing a large number of volume salvages.
Change-Id: Ia0f48bfabae9442ab0f1b4a6f43df34699892f66
Reviewed-on: http://gerrit.openafs.org/7231 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Tom Keiser <tkeiser@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementix.org>
High security mode for integrated logon never was high security.
It use was deprecated in the 1.5 series and it has no use at all
in the afs redirector world. Remove it.
The get cache params output is supposed to include two values:
. the size of the cache
. the size of the cache in use
Windows no longer has a concept of an unused cache buffer. All
buffers are inserted onto the freelist and are available for
recycling when the AFSCache file is created. Instead of reporting
the used cache space as 0K, report it as the full cache in use.
It is likely to disturb users less.
Andrew Deason [Fri, 27 Apr 2012 17:59:25 +0000 (12:59 -0500)]
vol: A GOING_OFFLINE volume should yield VOFFLINE
Currently, GetVolume treats a volume in the VOL_STATE_GOING_OFFLINE
state the same as VOL_STATE_SHUTTING_DOWN, and so returns VNOVOL for a
GOING_OFFLINE volume, but these states are very different.
GOING_OFFLINE indicates that a volume should soon be in the UNATTACHED
state, so we should treat GOING_OFFLINE the same as UNATTACHED for
returning errors to the user. For UNATTACHED, we return specialStatus
if it's set, or VOFFLINE otherwise; so, just do the same for
GOING_OFFLINE.
The variable bPurgeExtents was not being set when a DV change
was detected in AFSValidateEntry(). This resulted in the purge
being skipped and old data being left in the cache.
Windows: Directory validation should purge data changes immediately
During AFSEnumerateDirectory() and AFSVerifyDirectoryContent() calls
use AFSPerformObjectInvalidate() instead of AFSInvalidateObject()
to trigger the data purge. This is necessary to avoid a race as
AFSInvalidateObject() will queue a work request that will be performed
after the metadata is updated.
Windows: Flag purge on close if CcPurgeCacheSection fails
CcPurgeCacheSection can fail. If it does, remember that the
purge still needs to be performed by setting the
AFS_FCB_FLAG_PURGE_ON_CLOSE flag on the File Control Block.
Simon Wilkinson [Sun, 22 Apr 2012 17:19:07 +0000 (18:19 +0100)]
tests: More fixes for the vos test
The vos test wasn't running correctly from runtests, as it contained
a relative path which assumed that the CWD was tests/volser, rather
than tests/
Modify this to use the BUILD environment variable when invoked from
runtests, and also add an exit after the exec(), so that if we do
fail to launch the binary we don't have two processes both running
the same code.
Windows: Add global root to name array if share name
If the share name was resolved by querying the service instead
of finding the entry in the root.afs root directory, construct
a name array in AFSParseName() that includes the AFSGlobalRoot
above the resolved share root directory.
In AFSBackupEntry, check for the case where two volume root entries
appear in sequence without an intervening mount point.
Simon Wilkinson [Sat, 21 Apr 2012 06:43:59 +0000 (07:43 +0100)]
ptserver: Refactor per-call ubik initialisation
The way in which the ubik database is initialised is identical for
all read transactions, and for all write transactions. Rather than
duplicating this code in each call handler, pull it out into two
helper functions - ReadPreamble and WritePreamble.
Simon Wilkinson [Sat, 21 Apr 2012 19:55:23 +0000 (20:55 +0100)]
util: Completely remove get_krbrlm
Commit d85ece0977e043154b7d8f5aef5f4cd972771e8e added a new
mechanism for determining whether a realm is local or not, and
susequent commits removed all in-tree calls to the now-legacy
functions in get_krbrlm.c
To avoid confusion, just remove all of these legacy functions, as
we don't want to end up supporting two ways of doing this
operation.
This change is not suitable for pullup to a stable release.
Simon Wilkinson [Wed, 18 Apr 2012 11:46:31 +0000 (12:46 +0100)]
tests: Add a RX functionality test
Use the rxperf performance testing tools to add a couple of simple
RX tests. The first moves 1Mbyte of data backwards and forwards 30
times. The second starts 30 threads, which each move 1MByte of data
once.
This is by no means an exhaustive test of RX, but the single and
multi-threaded invocations should provide a useful smoke test if
things get very broken.
Simon Wilkinson [Tue, 17 Apr 2012 22:19:17 +0000 (23:19 +0100)]
rxperf: Move into the tools directory
Move the 'rxperf' RX performance testing utility out of the
src/rx/test directory, and into the slightly more visible top level
src/tools/ directory
As this is the first time that rxperf has been built as part of the
default build, make a number of changes so that it will build on all
of our supported platforms.
Simon Wilkinson [Wed, 18 Apr 2012 11:44:43 +0000 (12:44 +0100)]
tests: Explicitly include DES in superuser test
When the hcrypto/des header was removed from our installed headers, it
wasn't added back in to the superuser test. Add it now, so that the test
can build.
Simon Wilkinson [Wed, 18 Apr 2012 11:35:10 +0000 (12:35 +0100)]
Mac OS: Fixed shared library symbol issues
Some of our shared libraries (in particular, roken) build with different
symbols in them depending on the exact configuration options for a
particular platform. This means that not all of the symbols in the map
file may be present within the library. On Mac OS X we have been working
around this by using the "-flat_namespace,-undefined,suppress" linker
options.
However, with Lion this no longer works, as the linker still expects to
find the symbol in the library whose mapfile indicated that it was
present. So, for example, we end up with errors like:
dyld: Symbol not found: _errx
Referenced from: openafs.git/tests/rx/../../src/tools/rxperf/rxperf
Expected in: openafs.git/lib/librokenafs.dylib.1.1
... despite errx actually being provided by the system libraries.
The fix to this is to use the default two level namespace, and change
our behaviour for undefined symbols to 'dynamic_lookup', rather than
'suppress'
Michael Meffie [Mon, 5 Mar 2012 15:47:45 +0000 (10:47 -0500)]
audit: remove static local realms
Remove the static list of local realms and use the
auth interace to do the local realm check. A callback
function is registered by the servers to avoid a circular
dependency between audit and auth.
Simon Wilkinson [Fri, 13 Apr 2012 13:49:59 +0000 (14:49 +0100)]
rx: Use native 64bit data counters
Modify the peer, call and rpc_stats structures to use native 64 bit
types for the bytesSent and bytesRcvd data counters. All of our
platforms support native 64bit quantities now, so there's absolutely
no value in rolling our own.
Windows: Drop Fcb Resource across SetEOF and SetAllocation
If the file size or allocation is being altered, we must hold
the PagingResource and drop the Fcb Resource. Dropping the
Fcb resource is necessary to avoid a deadlock with TrendMicro's
filter if the size is set to zero and acquiring the PagingResource
is necessary to prevent races now that the Fcb Resource is no
longer held.
Instead of calling CcPurgeCacheSection() in AFSProcessOverwriteSupersede()
as part of the file length truncation to zero, call CcSetFileSizes().
Wait to call CcSetFileSizes() until after the Fcb->Resource has been
dropped but while the Fcb->Header.PagingIoResource is still held.
Make sure that file sizes are restored in the Fcb->Header if the
afsd_service rejects the file update.
Michael Meffie [Tue, 28 Feb 2012 13:50:33 +0000 (08:50 -0500)]
auth: local realms configuration
Add krb.conf and krb.excl support to the auth cell configuration
library. Provide a function to determine if the user is local to the
cell. Provide a function to set the local realms during application
initialization. These changes are intended to replace the functions
afs_krb_get_lrealm and afs_is_foreign_ticket_name.
Simon Wilkinson [Fri, 13 Apr 2012 18:14:44 +0000 (19:14 +0100)]
rx: Remove surplus call to FindPeer
When stats are enabled, rxi_ReadPacket calls FindPeer immediately
the packet is received from the wire. The peer structure that it
gets is used solely to increment a counter, and then thrown away.
Given that FindPeer requires a lock, and a hash lookup, this is
really inefficent.
Instead, delay the compilation of statistics until rxi_ReceivePacket.
Call FindPeer for version and debug packets which have no associated
connection otherwise wait until we have found the packet's connection,
and use the peer which is linked from there.
Andrew Deason [Thu, 29 Mar 2012 15:30:47 +0000 (10:30 -0500)]
rx: dec rx_nWaiting on clearing RX_CALL_WAIT_PROC
Currently, a couple of callers (rxi_ResetCall, and
rxi_AttachServerProc) will decrement rx_nWaiting only if
RX_CALL_WAIT_PROC is set for a call, and the call is on a queue
(presumably rx_incomingCallQueue). This can cause an imbalance in
rx_nWaiting if these code paths are reached when, in another thread,
rx_GetCall has removed the call from its queue, but it has not yet
cleared RX_CALL_WAIT_PROC (this can happen while it is waiting for
call->lock). In this situation, rx_GetCall will remove the call from
its queue, wait, and e.g. rxi_ResetCall will clear RX_CALL_WAIT_PROC;
neither will decrement rx_nWaiting.
This is possible if a new call is started on a call channel with an
extant call that is waiting for a thread; we will rxi_ResetCall in
rxi_ReceivePacket, but rx_GetCall may be running at the same time.
This race may also be possible via rxi_AttachServerProc via
rxi_UpdatePeerReach -> TryAttach -> rxi_AttachServerProc while
rx_GetCall is running, but I'm not sure.
To avoid this, decrement rx_nWaiting based on RX_CALL_WAIT_PROC alone,
regardless of whether or not the call is on a queue. This mirrors the
incrementing rx_nWaiting behavior, where rx_nWaiting is only
incremented if RX_CALL_WAIT_PROC is unset for a call, so this should
guarantee that rx_nWaiting does not become unbalanced.
In rxi_ReceivePacket, if the packet is for a client connection
and there is no call allocated, the conn->conn_call_lock was
leaked. Introduced by 95c38dff3740d7e24971ceb5875c06e7abfce102.
Michael Meffie [Mon, 20 Feb 2012 20:26:09 +0000 (15:26 -0500)]
auth: get local cell internal function
Add an internal function to retrieve the local cell
name when the auth mutex is already held. Implement
afsconf_GetLocalCell in terms of the new internal
function.
Instead of attempting to CreateFile(OPEN_EXISTING) the _._AFS_IOCTL_._
file, try using GetFileAttributes() to obtain the attributes of the
file. The file doesn't exist in the directory listing but the
afs redirector and the afs smb server will respond to the request.
Perhaps the SMB servers in the broken printers will fail it.
When verifying a directory entry we must update the metadata
for a file therefore we must also purge the data if the version
number changed. This is not optional.
Simon Wilkinson [Sun, 8 Jan 2012 11:45:57 +0000 (11:45 +0000)]
rx: Rework code which pulls packet from recv queue
Both rxi_ReadProc and rxi_FillReadVector contained copies of the
same code to pull a packet out of the receive queue, and turn it
into the call's currentPacket. Abstract this out into a single common
function, so we're not maintaining the same code in two different
places.
Simon Wilkinson [Fri, 6 Jan 2012 10:12:26 +0000 (10:12 +0000)]
rx: Tidy up currentPacket handling
Instead of making a copy of the call->currentPacket variable in our
read/write routines, reference it directly. Make it clear that
currentPacket is used solely by the application thread, and remove
a number of mistaken comments that suggest otherwise.
Simon Wilkinson [Thu, 8 Sep 2011 10:26:39 +0000 (11:26 +0100)]
Remove redundant header includes
Remove includes of system headers where roken.h already takes care of
including them. This simplifies the source tree, reduces the amount of
work done by the compiler, and ensures that all of our headers are
included with the correct guards
The list of files to edit was generated with the following script:
list=`grep include external/heimdal/roken/roken.h.in \
| sed -e's/#include//g' | sort | uniq`; \
for A in `find . -name *.c | xargs grep -l roken.h \
| grep -v external/ | grep -v WINNT/`; do \
found=0; \
for B in $list; do \
if grep "$B" $A > /dev/null; then \
echo "$A : $B"; \
found=1; \
fi; \
done; \
if [ $found == 1 ] ; then mvim -f $A; fi; \
done
Andrew Deason [Thu, 22 Mar 2012 22:54:12 +0000 (17:54 -0500)]
salvager: Trust inode-based special data over OGM
Currently the salvaging code looks for special inodes, and infers the
volume id and inode type from the OGM data in each special inode file.
However, we can already derive this information from the inode number
itself for the special inode, so if they disagree, use the values
based off of the inode number and correct the OGM data.
The inode number should be more likely to be correct, since that is
how we look up the special inode from the header when attaching the
volume. It is also impossible to get special inode files with the same
name, so this ensures we don't get duplicates. And for people that go
snooping around /vicepX/AFSIDat even though we tell them not to, it
seems more likely that they go around 'chmod'ing or 'chown'ing rather
than 'mv'ing.
This change avoids an abort in the salvaging code when the OGM data is
wrong. If we trust the OGM data when it is incorrect, we assume the
special inode file is for a different volume. So when we go to
recreate one of the special files for the volume we're actually
working with, the IH_CREATE fails (from EEXIST) and so we abort.
Andrew Deason [Fri, 23 Mar 2012 18:02:22 +0000 (13:02 -0500)]
namei: Abstract out OGM functions a bit more
Add GetWinOGM and SetWinOGM for getting and setting the
Windows-equivalent of the Unix OGM data. Make those and CheckOGM use
GetFileTime/SetFileTime so we can operate just via an FD_t, without
needing the full pathname. Modify the NT namei_icreate to use
SetWinOGM.
Michael Meffie [Wed, 29 Feb 2012 14:40:14 +0000 (09:40 -0500)]
Build: link libaudit before util
libaudit depends on util symbols, so link it first to avoid
unresolved symbols. Include libaudit in the list of libs
instead of making it a special case.
If an error was detected when processing RDR_CleanupFileEntry()
do not perform the file deletion. Just return the error as
attempting the deletion could result in an exception.
Simon Wilkinson [Sun, 23 Oct 2011 15:21:52 +0000 (16:21 +0100)]
Rework the ugen_* interface
Remodel the ugen_* ubik initialisation interface so that more complex
sets of security flags can be passed through it. A number of new
functions are provided, all of which take the AFSCONF_* security flags
defined in libauth:
*) ugen_ClientInitCell is used to initialise a ubik client structure
when the caller already has afsconf_dir and afsconf_cell
structures for the cell being contacted
*) ugen_ClientInitServer is used to initialise a client which will
contact a specific server within a given cell, rather than
selecting one from the pool
*) ugen_ClientInitFlags is used to initialise a client given a
cellname, and a path to a configuration directory. It is is
closest to the legacy ugen_ClientInit function.
All of the in tree callers are updated to use this new interface
Marc Dionne [Sun, 22 Jan 2012 15:03:43 +0000 (10:03 -0500)]
viced: Make Update_TargetVnodeStatus usable by replication code
Add a remote flag, skip rights related code for the remote case.
This will need to be revisited later; in the case that what ends
up on disk is slightly different than the InStatus passed in, the
changes should be preserved to pass to the replicas.
Marc Dionne [Sun, 22 Jan 2012 14:34:28 +0000 (09:34 -0500)]
viced: Make GetVolumePackageWithCall usable by replication code
Add a remote flag to GetVolumePackageWithCall to make it usable for
remote replication RPCs. For the remote case the checks related
to access rights are skipped.
Marc Dionne [Sat, 21 Jan 2012 19:42:01 +0000 (14:42 -0500)]
vol: Allow VAllocVnode of specific vnodes
Add parameters to VAllocVnode to allow the caller to specifiy the
vnode and unique numbers to use. This will be used by the RW
replication code to keep vnode numbers in sync between the master
volume and the replicas.
Adapted from code by Vishal Powar and Derrick Brashear.
Simon Wilkinson [Thu, 5 Apr 2012 20:00:32 +0000 (16:00 -0400)]
rxkad: Remove DES include from header file
Remove the hcrypto/des.h include directory from the rxkad_prototypes
header file. We ship rxkad_prototypes.h, but we don't ship the hcrypto
headers. Having an installed header depend upon an uninstalled one is
a pain when building out-of-tree code like perl-AFS.
Simon Wilkinson [Thu, 5 Apr 2012 19:58:11 +0000 (15:58 -0400)]
rxkad: Move conversion functions to own file
Move the various key conversion functions into thir own header file,
rather than rxkad_prototypes.h. This means we can isolate the dependency
on the hcrypto DES headers (which we don't install)
Simon Wilkinson [Fri, 6 Apr 2012 22:56:30 +0000 (23:56 +0100)]
shlibafsauthent: Add a load more objects
The perl-AFS module needs to link against a pthreaded, PIC, library
that contains all of the routines that it provides perl shims for.
At the moment, libafsrpc_pic and libafsauthent_pic are the only such
libraries that OpenAFS provides.
Add all of the necessary objects to libafsauthent_pic such that the
2.6.2 version of perl-AFS can build with a minimal set of patches.
Minimise the damage by only making these available through the static
version of the libafsauthent_pic library.
Simon Wilkinson [Fri, 6 Apr 2012 22:54:10 +0000 (23:54 +0100)]
opr: Install the queue.h header
The opr queue definition is required by a structure in the installed
cellconfig.h header. So that header can still be used by out-of-tree
applications, install the queue.h header too.
Refactor AFSValidateEntry to avoid obtaining the
ObjectInformation->Fcb->Resource when it isn't necessary.
This will avoid contention and improve performance.
The only time that the Fcb->Resource is required is when
the object requires verification, the object is a FILE,
and the object was successfully evaluated.
Even with this reorganization there is a small window
of opportunity for a deadlock to occur if a CcPurgeCacheSection()
which is called with the Fcb->Resource held triggers a filter
driver to issue a CreateFile and in between the two operations
an invalidate object is received.
Simon Wilkinson [Fri, 9 Sep 2011 06:46:01 +0000 (07:46 +0100)]
Generalise the new background daemon code
Move the new background daemon code out from under the AFS_DARWIN80_ENV
ifdef into its own AFS_NEW_BKG define. This lets us use it for things
other than the Darwin mv helper, on other platforms.
Based on code originally written by Derrick Brashear
Andrew Deason [Mon, 9 Apr 2012 22:16:42 +0000 (17:16 -0500)]
vos: Do not try to remove backup volume id 0
Currently we always try to delete a BK volume if we're deleting the
RW. If the BK volume id is 0, this is never going to do anything, so
don't try to delete it.
Simon Wilkinson [Sun, 1 Apr 2012 16:27:04 +0000 (12:27 -0400)]
rx: Remove packet burst elements from peer struct
The peer structure contains a number of elements left over from an
earlier, burst based, congestion control strategy. As OpenAFS has
never implemented this strategy, just remove these elements from
the structure, and store 0 values for these elements in the debug
packets.
Simon Wilkinson [Fri, 25 Nov 2011 20:51:40 +0000 (20:51 +0000)]
rx: Remove the unused packet skew code
We don't (and haven't, since the release of OpenAFS) use the packet
skew calculations for anything. However, maintaining them requires
taking locks in some critical parts of the transmission code path.
For both speed, and maintainability, reasons remove the skew code
from the tree - it's in git if we ever want to go back to it.
projects / packages / o / openafs.git / log