Contains DKMS robustness fixes, improvements to the defaults for the
module build, and cleanup of the openafs-client init script. Updates
the build system for the new demand-attach binary naming and for the
changes to supported configure options. Fixes some issues with
afs-newcell. Forces disabling of the Linux syscall probing in kernel
module builds, since no supported Debian kernel allows this and it
causes problems. Update debhelper to V8, which allows simplification
of debian/rules and debian/module/rules.
Michael Meffie [Thu, 23 Sep 2010 14:15:57 +0000 (10:15 -0400)]
scout: display fetch and store counts as unsigned
Fetches and stores are already defined as unsigned, so format
them as unsigned values when displaying in scout. This fixes
the bug where scout shows those counts as negative values on
busy servers which have been running for a while.
Simon Wilkinson [Thu, 23 Sep 2010 16:41:47 +0000 (17:41 +0100)]
rx: Big windows make us sad
The commit which took our Window size to 128 caused rxperf to run
40 times slower than before. All of the recent rx improvements have
reduced this to being around 2x slower than before, but we're still
not ready for large window sizes.
As 1.6 is nearing release, reset back to the old, fast, window size
of 32. We can revist this as further performance improvements and
restructuring happen on master.
Simon Wilkinson [Sat, 2 Oct 2010 03:17:56 +0000 (23:17 -0400)]
rx: Reduce dependence on call->lock
This patch reduces our dependence on call->lock, by allowing more
of the reader thread to run lock free. Doing so requires that
call->mode only be set by the reader thread. As a result, call->mode
can only be set to RX_CALL_ERROR by rxi_CallError(). The mode is
set to RX_CALL_ERROR by the reader thread immediately after regaining
the call->lock when it has been dropped.
Simon Wilkinson [Tue, 5 Oct 2010 00:20:32 +0000 (01:20 +0100)]
hcrypto: Fix builds on Irix
The recent hcrypto pullup added a depedence on the errx() function,
which isn't present on Irix. Solve this by pulling in a load more
of libroken, in order to provide this function.
In the long term, libroken should get split out into its own
directory, and the ability to use a previously installed
libroken should be added. For now, this will hopefully get Irix on
its feet again.
Simon Wilkinson [Tue, 5 Oct 2010 08:01:00 +0000 (09:01 +0100)]
Irix: Make compiler less chatty
Supress a few of our errors from the Irix compiler and linker, so its
output is a little less verbose.
This change suppresses the function declared but not used and
multiple declaration errors that we get due to our static_inline fudge
and the paramater declared but not used errors.
Other error suppression is possible - you just need the number
immediately after the 'cc-' in the build logs to say which number to
add to the -woff line.
Simon Wilkinson [Thu, 30 Sep 2010 12:58:26 +0000 (13:58 +0100)]
Kill AFS_64BIT_ENV
The AFS_64BIT_ENV was being pretty much universally defined. So,
remove the definition, and the conditional code that it controlled.
From this point on, all platforms are assumed to be capable of
handling 64bit values.
Simon Wilkinson [Mon, 4 Oct 2010 19:22:50 +0000 (20:22 +0100)]
Heimdal: Fix 32bit build problems
The earlier inclusion of sha512 from Heimdal broke the build on
32bit platforms, because this file doesn't flag 64bit constants as
being such.
The correct place for this fix is in Heimdal, and it will be replaced
with a fix from Heimdal as soon as one is available. For now though,
this gets our build going again.
Andrew Deason [Sun, 3 Oct 2010 23:27:19 +0000 (18:27 -0500)]
vol: Log ignored dirs that look like partitions
If we see a /vicep*-like directory when we VAttachPartitions, and we
ignore it because it lacks an AlwaysAttach, log that we ignored it.
This may make things less confusing to admins that just try to create
a /vicep* directory and don't know why it doesn't work.
Simon Wilkinson [Mon, 4 Oct 2010 11:33:24 +0000 (12:33 +0100)]
configure: Restore saved CFLAGS
When we test for whether the C compiler can take the
-fno-strength-reduce flag, we add the flag to CFLAGS to do so.
However, we were not restoring the old value of this flag when we
completed the test, and so we were always setting -fno-strength-reduce
in the userspace compile.
Previously, this was harmless, as we always overwrote CFLAGS, but if
we're moving to a world where we honour the user's setting of CFLAGS,
we need to not leak changes in this way.
Andrew Deason [Fri, 10 Sep 2010 20:52:34 +0000 (15:52 -0500)]
vos release: Force full dump on RO_DONTUSE sites
When releasing a volume, currently we perform an incremental dump on
RO_DONTUSE sites if the volume exists on the remote site. Since
RO_DONTUSE implies that we do not expect a site to exist there, delete
the extant volume if we find one and force a full dump.
If we perform an incremental dump, we run the risk of incrementally
dumping to a temporary volume that the administrator is not aware of,
and doesn't have any actual data but has a last update time late
enough that it may be missing some data after the incremental dump. So
to avoid that, force a full dump every time.
sin_family isnt sent over the network and therefore doesnt need
htons(). sin_family is essential the same as domain, and no one
does socket(htons(AF_INET), ...)
Windows: Ensure that cm_NameI errors are acted upon promptly
There are many cases in the SMB server where an error from cm_NameI()
was either ignored or not acted upon until several other operations
are performed that could result in the same error being repeated.
This is a mistake which did not have negative side effects until
additional checks for callback status were added recently.
At present, if a CM_ERROR_ACCESS error is returned and ignored,
subsequent attempts to operate on the same cm_scache_t will result
in additional queries to the file server that will also end in an
abort response. This can trigger the file server to delay responses
to the client.
This patchset ensures that all cm_NameI() errors are acted upon
promptly.
Windows: Fix Parent(path) computation to permit mp and symlink creation
The parent path computation was leaving trailing slashes on the
path names which prevented the creation of mount points and symlinks
when UNC paths were used that contained mount points.
Jeffrey Altman [Sat, 2 Oct 2010 03:47:11 +0000 (23:47 -0400)]
Rx: raise rx_minPeerTimeout to 20ms
At 2ms it is possible for the packet we are sending to be resent
just about immediately as the retryTime computation occurs before
the send takes place and not afterwards. If the network send blocks,
the retryTime may have already expired.
We do not want rx_minPeerTimeout to be too large though because the
value will end up being used as the backoff time period when the
actual RTT for the connection is less than the rx_minPeerTimeout
value. Extensive testing shows 20ms to be an adequate compromise
that avoids the vast majority of unnecessary resends without
unnecessarily slowing down the connection if a packet is in fact
lost.
Rx: When call receive is done, send ack all packet
When all of the packets for a call have been received,
immediately send an ack all packet to the peer. This permits
the peer to free the contents of the transmit queue and cancel
all pending resend events.
Rx: protect rx_conn and rx_call refCount field with rx_refcnt_mutex
Add a new global mutex rx_refcnt_mutex to protect the conn->refCount
and call->refCount in place of relying upon the conn->conn_data_lock
and the call->lock.
This will relieve some lock contention with rx_ReceivePacket().
Jeffrey Altman [Sat, 2 Oct 2010 04:49:38 +0000 (00:49 -0400)]
Windows: Pass Volume Root Fid to cm_Analyze after RXAFS_GetVolumeStatus
RXAFS_GetVolumeStatus can return VNOVOL, VMOVED, etc. In order to
process them and update volume state a fid must be passed to cm_Analyze().
Use the volume root fid.
Andrew Deason [Thu, 11 Mar 2010 16:39:56 +0000 (10:39 -0600)]
Provide an abstract work queue object
Add some routines for specifying chunks of work to be done. The idea
is to be able to pass these to different threads, and specify
dependencies between them, wait on them completing, etc.
This adds the afs_wq* family of functions. Originally written by Tom
Keiser.
Andrew Deason [Tue, 14 Sep 2010 14:45:10 +0000 (10:45 -0400)]
DAFS: Raise LogLevel for per-chain vol stats
Only report detailed per-chain volume statistics on shutdown/SIGXCPU
if LogLevel is 125 (or 25 for smaller per-chain stats). If a
fileserver is configured with a large -vhashsize, printing out stats
for each chain can take awhile and use up a nontrivial amount of disk
space for logging, so only print out these stats if we're asked for
them.
configure: --with-linux-kernel-packaging should default to disabled
the test for this build feature is reversed. by default, the value for
with_linux_kernel_packaging will not be defined which makes the existing
test pick MPS='SP' instead of LINUX_WHICH_MODULES. based on the configure
help messages, this would appear to be an opt-in not an opt-out.
...
Optional Packages:
...
--with-linux-kernel-packaging
use standard naming conventions to aid Linux kernel
build packaging (disables MPS, sets the kernel
module name to openafs.ko, and installs kernel
modules into the standard Linux location)
...
Simon Wilkinson [Tue, 28 Sep 2010 23:21:43 +0000 (00:21 +0100)]
rx: Don't have different args for rxi_FreeCall
rxi_FreeCall changes its number of arguments depending on whether
locks are enabled or not. That's a little bit nasty to read, so just
change it so that it always takes two arguments, and ignores the
second when it doesn't need it.
Simon Wilkinson [Tue, 28 Sep 2010 23:11:53 +0000 (00:11 +0100)]
rx: Make statistics interface use Atomics
Make the rx_statistics statistics gathering infrastructure use
atomics for all of its counters. This significantly reduces
lock contention. However, it also (potentially) changes the format
of the rx_stats variable which has been used by callers in the past.
To simplify this process, and to aid with future changes, we remove
direct access to rx_stats. Instead, two additional API functions
rx_GetStatistics and rx_FreeStatistics are provided. These give the
caller access to an 'normal' rx_statistics structure.
Tom Keiser has suggested that we should explore using thread-local
statistics structures, and just aggregating them when we are asked
to report. This is a fine idea, and is equally possible with the
new interface that this patch introduces.
Simon Wilkinson [Tue, 28 Sep 2010 22:48:50 +0000 (23:48 +0100)]
rx: Add rx_NewThreadId function
The fileserver and the fsync server were locking an internal RX
mutex, and incrementing an internal counter in order to obtain fake
pthread thread IDs. Instead of letting them muck around in the
internals of RX, provide an API that can be called to obtain a
ThreadId counter, and use that API throughout the code.
Simon Wilkinson [Tue, 28 Sep 2010 22:37:54 +0000 (23:37 +0100)]
rx: Add atomic operations code
Add support for an atomic type, and atomic operators for RX. This
builds on work which has already been done for Windows, where
InterlockedOperations are used for statistics gathering.
A new opaque type, rx_atomic_t is introduced so that normal arithmetic
operations will fail on atomic data.
An implementation using native atomic methods is provided for Darwin,
Solaris and Windows. A native kernel implementation is used for Linux.
Where OpenAFS is built with a sufficiently modern gcc, gcc's atomic
primitives will be used. Sadly, gcc's builtin operations are not
available for i386, they will only be used with builds the set
-march=i486 (or later).
Otherwise, we fall back to a single mutex which protects all atomic
operations.
Simon Wilkinson [Sun, 26 Sep 2010 14:48:54 +0000 (15:48 +0100)]
RX: Tidy reader data locking
Data which is accessed only by the reader thread doesn't need to be
protected by call->lock
Remove the call->lock protection where it isn't required, which makes
certain read/write calls lock free.
Stop rx_ResetCall from manipulating reader thread data. This data will
be zero'd and cleared when the reader thread calls rx_EndCall, and
doesn't need to be reset by the Listener thread.
The change which made rx_ResetCall reset reader thread information
was originally part of 559ea99b. It caused race conditions that were
fixed by adding additional lock protection in d0cc6e, 4dadd2 and 423ab97e. This commit reverts portions of all of those changes. It
is safe to not clear the iovc in ResetCall because any NewCall must
be balanced by a corresponding EndCall in the reader thread, and
EndCall does the appropriate freeing of reader elements.
Ben Kaduk [Wed, 29 Sep 2010 00:03:25 +0000 (20:03 -0400)]
More FBSD syscall tweaking
We're now properly registered in syscalls.master for HEAD
(i.e. proto-9.0) and RELENG_8 (proto-8.2), which means that
afs3_syscall is prototyped in sys/sysproto.h . Accordingly,
don't declare it in afs_prototypes.h for those cases.
Also add FBSD82_ENV checks for the new syscall-registration code,
and cast afs3_syscall to sy_call_t* for the sysent structure.
Simon Wilkinson [Mon, 27 Sep 2010 22:50:23 +0000 (23:50 +0100)]
rx: Limit window size to max acks
The RX ack packet can only acknowledge 255 packets at once. In the
current implementation, this limits our maximum window size to 255,
as we can't acknowledge any packets we receive outside of that window
size.
The previous value, 350ms, is historical. Now that networks are
so much faster, an artificially high timeout value when backed off
results in an extremely long delay before communication can resume.
Rx: Do not hold call lock across memcpy in rx_ReadProc/rx_WriteProc
1.4.x does not hold the call lock across memcpy operations in
rx_ReadProc, rx_ReadProc32, rx_WriteProc, rx_WriteProc32. The
claim is that the call curpos, curlen, and nLeft fields which
refer to the current packet being processed will not be touched
by any other thread. Therefore it is safe to drop the call lock
to permit another thread to add packets to the call while the memcpy
is performed in parallel.
This patchset continues to hold the call lock longer than the
original implementation but does drop it for the length of time
it takes to copy data from the packet buffer to the application
buffer.
Windows: Export additional RX debugging variables from afsrpc.dll
Export
rxi_nRecvFrags @2008 DATA
rxi_nSendFrags @2009 DATA
rx_initReceiveWindow @2010 DATA
rx_initSendWindow @2011 DATA
rx_intentionallyDroppedPacketsPer100 @2012 DATA
rx_intentionallyDroppedOnReadPer100 @2013 DATA
so they can be referenced from pthreaded builds of src/rx/test tools.
Exported variables must be present in both FREE and CHECKED builds.
Rx: PrintTheseStats should not be dependent on RXDEBUG
When RXDEBUG is not defined, PrintTheseStats generates an error
even though the statistics are in fact available. The global
variable rx_packetTypes was not being defined without RXDEBUG.
Make rx_packetTypes defined always and permit statistics to
always be printed.
The global dataPacketsReSent statistic should be the sum of all
peer->reSends and dataPacketsSent should not include the count of
resent packets. Prior to this patchset, dataPacketsSent included
the resent packets and dataPacketsReSent was computed as the number
of requests for Ack instead of the number of packets resent.
If a packet is missing, the peer timeout is backed off to provide
a new starting point for timeout computation. The backoff state
must be stored in the peer object to ensure that multiple failures
do not result in more than one backoff before a successfully received
packet is available for recomputation.
Rx: only compute peer bytes sent and received if rx_stats_active
Computing the bytes sent and received is an expensive operation.
If rx statistics collection has been disabled we should not collect
the peer data. The most expensive operation is the rx_FindPeer()
call that is performed during rxi_ReadPacket(). rxi_ReadPacket()
is processed by the rx listener thread which must be as fast as
possible.
rxi_ReceiveAckPacket can acquire and drop the conn_data_lock several
times and acquires and drops the peer_lock unnecessarily. This patchset
adds a variable to track whether the conn_data_lock is held in order
to avoid the need to drop it and reacquire it based upon conditional
operations. It also relocates the peer->maxPacketSize computations
in order to consolidate the work performed under the peer_lock.
rxperf made assumptions that it was built against LWP, used buffer
sizes for read/write that were too small, made use of non-portable
types, and set signal handlers that are unsupported.
Andrew Deason [Wed, 15 Sep 2010 16:19:33 +0000 (12:19 -0400)]
libafs: Fix pioctl get/putInt alignment issues
We don't know if the buffer for pioctl data is aligned to anything, so
we can't just dereference the given pointer as an int or anything
else. So, just memcpy the data in for ints and such; conveniently,
afs_pd_getBytes and afs_pd_putBytes can do this for us, so just use
that.
Marc Dionne [Fri, 10 Sep 2010 23:55:39 +0000 (19:55 -0400)]
vlserver: Set but not used variables
Remove some variables that are set but never used in the vlserver
directory:
- n1,n2,n3 and n4 in vlclient.c appear to have never been used even
in the original IBM code
- some variables in vldb_check.c that are no longer used after some
recent changes
Andrew Deason [Tue, 14 Sep 2010 16:15:22 +0000 (12:15 -0400)]
volser: Delete timed-out temporary volumes
When a transaction times out on a volume, delete the volume if it is a
temporary volume (destroyMe is set). This prevents half-created
volumes from accumulating, which can take up space and screw up
certain vol ops in some versions.
Marc Dionne [Sat, 11 Sep 2010 17:23:11 +0000 (13:23 -0400)]
butc: Set but unused variables
Remove unused variable taskId
writeData() systematically returns 0, so make it void and adapt
call sites that assigned the return value but never used it.
Also move the function up in the file to avoid the need for a
forward declaration, and make it static since it's only used here.
Marc Dionne [Thu, 1 Jul 2010 15:38:20 +0000 (11:38 -0400)]
Linux: normalize error return for emulated syscalls
pagsh and other code expect setpag() and pioctl() to behave like
a regular syscall or pioctl, that is to return -1 on error, with
errno set to the specific error code.
On Linux, the underlying emulation does a straight return of any
error code it gets from the ioctl, and errors are not properly
caught by the callers.
As an example, pagsh won't detect an error from setpag such as
exceeding a keyring quota limit. With this patch, the user
will see this:
$ pagsh
setpag: Disk quota exceeded
sh-4.1$
The code in proc_afs_syscall is modified to set errno to the error
code and to set errorcode to -1 in case of error.
proc_afs_sycall is reindented while we're changing code there.
minimize the impact of Rx packet tracking. in particular, do no
extra queue scans, which means the rest of the state which tracks
where a packet is now isn't of use. make it possible to re-enable.
Simon Wilkinson [Wed, 8 Sep 2010 07:22:57 +0000 (08:22 +0100)]
Add config.log to gitignore globally
When you end up explicitly regenerating chosen Makefiles by
running ../../config.status Makefile, the working tree ends up littered
with config.log files.
Currently, we only ignore config.log in the top directory - extend this
so that it's ignored across the tree.
Marc Dionne [Fri, 10 Sep 2010 01:02:05 +0000 (21:02 -0400)]
Warning fix for gcc 4.5 "operation may be undefined" warnings
The inc_header_word and set_header_word macros make repeated use of their
argument, which triggers many (~30) warnings with gcc 4.5, like this one:
./ptutils.c:473:6: warning: operation on ‘cheader.foreigncount’ may be undefined
Removing the cast to afs_int32 in the macros gets rid of the warning,
and should be safe since we're just getting a small positive integer value
- the offset of the member in the structure - and passing it to the
pr_Write function which expects an afs_int32.
projects / packages / o / openafs.git / log