Marc Dionne [Sun, 15 May 2011 00:57:12 +0000 (20:57 -0400)]
Linux: fix reading files larger than the chunk size
Commit 2571b6285d5da8ef62ab38c3a938258ddd7bac4e fixed an issue with
the use of tmpfs as a disk cache and ftruncate() on files in AFS.
But it introduced a problem reading larger files as reported in
RT ticket 129880.
What should be compared against the current cache file size is the
offset into the current chunk, not the overall offset for the whole
file.
Russ Allbery [Sat, 14 May 2011 18:52:17 +0000 (11:52 -0700)]
Update to Debian source format 3.0 (quilt)
* Change to Debian source format 3.0 (quilt). Force a single Debian
patch for simplicity since the packaging is maintained in Git using
branches, and include a patch header explaining why.
Marc Dionne [Sat, 14 May 2011 17:19:52 +0000 (13:19 -0400)]
Linux: fix permission op test for certain compilers
Some compilers complain that _inode is used uninitialised here.
Since this test requires -Werror, it causes the test to fail
and our permission op to be used in RCU mode, leading to lockups.
Initialise it to make the compilers happy.
Fixes a lockup seen on kernels 2.6.38+ on Gentoo and Debian.
Simon Wilkinson [Tue, 3 May 2011 09:03:22 +0000 (10:03 +0100)]
Fix com_err renaming
The pull up of 352c3e366c2e0a8de4550547192276e62afd2cc1 to the 1.6
branch missed replacing occurences of com_err with afscom_err in
the sgistuff and pam directories. Fix this so that Irix builds
again.
Andrew Deason [Tue, 15 Feb 2011 18:04:32 +0000 (12:04 -0600)]
libafs: Drop xvcache for AllocCBR
Normally when we AllocCBR, we are holding xvcache write-locked, since
it is called from FlushVCache. Before a309e274632993c5aeec04c6e090f5ac95837a40, when AllocCBR needs to flush
CBRs due to a lack of space, we hit the net, giving up callbacks on
fileservers.
This can cause a problem if one of those fileservers needs to contact
us in order to complete that request, since the callback service
thread may be waiting for xvcache, causing a deadlock (that is
eventually broken by network timeouts).
To avoid this, drop xvcache if AllocCBR looks like it does not have
sufficient space. Fix all callers of afs_FlushVCache to handle the
case where we sleep, since with this change, afs_FlushVCache can sleep
on all platforms.
This partially reverts a309e274632993c5aeec04c6e090f5ac95837a40, as it
contains an alternative method of avoiding the xvcache lock in this
situation. This commit restores much of the code path to be much more
similar to how it used to be, except that it allows for dropping
xvcache for AllocCBR. This should make any change to our prior
behavior smaller/simpler, and thus safer and more consistent with
existing clients. This reintroduces the hard limit to how much space
we allocate for CBRs, although the part of a309e274632993c5aeec04c6e090f5ac95837a40 that raised this limit is
retained.
Andrew Deason [Thu, 28 Apr 2011 16:48:18 +0000 (11:48 -0500)]
libafs: Use vcount, not maxvcount to trim vcaches
Every five minutes we afs_ShakeLooseVCaches to try and return the
number of vcaches in use down to the originally configured -stat level
(when we are using dynamic vcaches). We should calculate how many
vcaches to flush based on the number of currently active vcaches
(afs_vcount), not the peak number (afs_maxvcount). Otherwise, once we
exceed the configured -stat level, we will always keep trying to flush
numerous vcaches, even if we barely have any vcaches in use.
Simon Wilkinson [Mon, 25 Apr 2011 18:18:39 +0000 (14:18 -0400)]
Linux: Don't read pages beyond the cache eof
If we attempt to read past the end of the current cache file (for
example, when we're extending the file with ftruncate), don't force
the backend filesystem to populate that page with non-existent data.
This will hopefully fix a bus error when using tmpfs as a backing
cache.
Andrew Deason [Wed, 27 Apr 2011 21:24:46 +0000 (16:24 -0500)]
viced: Improve deleted client log messages
Change the information logged related to deleted clients to be a
little more useful. In particular this includes adding the client and
host refcounts, to help see if the cause is a reference count leak.
Andrew Deason [Wed, 27 Apr 2011 19:23:43 +0000 (14:23 -0500)]
viced: Avoid ref leak on origin callback break
When breaking a callback, sometimes we send a callback to the host
that performed the callback-inducing operation. When we do this,
currently BreakCallBack gives the origin host structure to
MultiBreakCallBack_r, which avoids releasing that host after the
callback is broken.
However, BreakCallBack obtains a reference to every host to which it
delivers a callback, even if it is the origin host, so a reference is
leaked. Fix this by not ever passing a host to MultiBreakCallBack_r,
and just have MultiBreakCallBack_r release a reference for every host
to which it delivers a callback break.
Andrew Deason [Wed, 27 Apr 2011 18:51:23 +0000 (13:51 -0500)]
viced: Transfer host ref in h_FindClient_r
In h_FindClient_r, we can change which client structure we're dealing
with if we find a different client struct in the Rx conn-specific
data. We adjust the refcounts for the client structures themselves,
but not the associated hosts. While the host structures should be the
same most of the time, we are not guaranteed that, so adjust their
refcounts as well.
Andrew Deason [Sat, 23 Apr 2011 21:52:30 +0000 (16:52 -0500)]
viced: Release all hosts in h_Enumerate*
h_Enumerate and h_Enumerate_r were not releasing all of the holds they
obtained when the callback function caused the enumeration to bail
early. Correct them so all host holds are released.
Andrew Deason [Sat, 23 Apr 2011 21:44:41 +0000 (16:44 -0500)]
viced: Print a warning when using a deleted client
We should never get a deleted client back from GetClient. Log a
message if we do, to explain why access may suddenly appear to fail,
and assist in determining why.
Note that we still try to service the request, since the accessing
user may still have enough access to do whatever was requested.
Jeffrey Altman [Sun, 1 May 2011 04:11:13 +0000 (00:11 -0400)]
Windows: Fix caching of non-existing vols
In cm_UpdateVolumeLocation() the conditional that would
trigger the immediate return of CM_ERROR_NOSUCHVOLUME
was backwards which prevented the caching from working.
cm_CheckOfflineVolumes() is called by the daemon thread
to reset the status of offline volumes. Non-existing
volumes are by definition offline and cannot be brought
online. Therefore, the cm_CheckOfflineVolumes() function
should skip volumes with the CM_VOLUMEFLAG_NOEXIST flag
set.
Ben Kaduk [Sun, 19 Dec 2010 04:52:43 +0000 (23:52 -0500)]
Rename libcom_err to libafscom_err
We no longer provide a compatible libcom_err, and in fact
we renamed the symbols in our libcom_err several years ago
to reflect this fact.
When we build on a system where KRB5_LIBS includes
-lkrb5 -lcom_err , the new Unix build system will pick up
our libcom_err (as $(AFS_LDFLAGS) is the first argument in
AFS_LDRULE and pulls in a linker search path for our libcom_err)
which does not provide all the needed symbols for libkrb5.
Fully rename our libcom_err away to avoid these conflicts.
Marc Dionne [Wed, 30 Mar 2011 22:32:04 +0000 (18:32 -0400)]
Linux: Fix fallout from path_lookup commit
Fix a few issues with the recent commit to deal withg the removal
of path_lookup, spotted on RHEL 5:
- the configure tests needs fs.h to be included before namei.h, to
get the definition of struct inode
- we need to avoid the use of struct path unless its needed; on
older kernels the structure doesn't exist
Andrew Deason [Sat, 23 Apr 2011 21:25:00 +0000 (16:25 -0500)]
viced: Fix host enumeration flags
Do not give uninitialized flags values to h_Enumerate callback
functions. In fact, do not give a flags value to h_Enumerate or
h_Enumerate_r callback functions at all, since they are not actually
used.
Fix host enumeration callback functions to just return 0 or the
relevant flags, instead of basing the return value off of the given
flags value. Update MultiBreakVolumeCallBack_r to use the correct
return values, since it currently tries to use the old meanings of the
host enumeration return values.
If the Kerberos v5 library cannot be loaded (pkrb5_init_context
equal to NULL) return a reasonable error code instead of
returning success and doing nothing.
Windows: NPLogonNotify provide password in all cases
When calling KFW_AFS_get_cred() from NPLogonNotify()
always provide the user password. Do not count on a
credential cache existing from a previous call.
Andrew Deason [Fri, 15 Apr 2011 16:18:37 +0000 (11:18 -0500)]
AIX51: Fix PAGs
On AIX 5.1 and later, we set a process' PAG by using the AIX PAG
mechanism (and not by group ids), but we were determining what PAG a
process was in by the group list. Instead use the PAG identifier.
This effectively reverts 277c37f48c8126ba9cb986ffc7361fcb98e2bbf2, but
it puts the kcred_getpag call in a different place that makes more
sense in the current PAG code organization.
Simon Wilkinson [Thu, 21 Apr 2011 15:07:05 +0000 (16:07 +0100)]
Linux: Restrict # of cbrs we allocate at once
With commit a309e274632993c5aeec04c6e090f5ac95837a40, we changed the
number of CBRs that we allocate in a chunk from 300 to 1024. However,
this change takes the amount of memory requried to allocate a chunk
of CBRs above PAGE_SIZE on Linux. This changes the allocator that we
use from kmalloc to vmalloc. Whilst we can, and do, prevent kmalloc
from flushing filesystem pages when we invoke it, we don't have a
similar level of control over vmalloc. 1.6 only: limit it back to 300
elsewhere.
In one reported case, clients deadlock whilst attempting to allocate
this memory, in a call stack that looks something like:
Simon Wilkinson [Mon, 25 Apr 2011 12:56:38 +0000 (13:56 +0100)]
Windows: Remove duplicate file
The 'Streamfiles.txt' file had been committed with both that name,
and an all lower case name. This makes git very sad on systems with
case insensitive filenames.
In cm_ReadMountPoint and cm_HandleLink the variable 'thyper'
represets the 'offset' at which cm_GetData should fetch data.
Rename 'thyper' to 'offset' and fix a coding error caused by
misinterpreting the variable purpose.
cm_GetData() drops the cm_scache_t rw lock which permits other
threads to access the data while it is in an inconsistent state.
Avoid the race by using a stack allocated temporary buffer to
receive the data from cm_GetData(). Only copy the data into
the mountPointStringp buffer under the rwlock.
Andrew Deason [Thu, 21 Apr 2011 19:24:45 +0000 (14:24 -0500)]
aklog: Return token when performing 524 conversion
We weren't actually returning a token and username from
rxkad_get_converted_token. Do so.
This is a 1.6-specific change. This issue was fixed on master when
aklog was changed to use the new SetTokenEx family of pioctls in
commit 53837416cbed3ba4d11f63015e1f13800519f2ed.
Andrew Deason [Tue, 12 Apr 2011 21:30:55 +0000 (16:30 -0500)]
libafs: Do not specify an Rx busy channel error
Do not specify an error for Rx busy channels in libafs, so we do not
get notified of busy channel notifications. The current code just
specifies RX_CALL_TIMEOUT for the busy channel error, which has
problems (in particular, for RXAFS calls that are not wrapped in an
afs_Analyze loop). So, disable this behavior.
This is a 1.6-specific change. The master branch will instead get
fixes for this functionality, and it will get turned on in 1.6 again
as appropriate.
OpenBSD: Complete implementation of afs_osi_TimedSleep
The OpenBSD version of afs_osi_TimedSleep() is missing the required
afs_event structure and afs_getevent routine. This update adds them
(by borrowing a copy of the code from the FreeBSD implementation).
Marc Dionne [Sun, 27 Mar 2011 15:20:17 +0000 (11:20 -0400)]
Linux: 2.6.39: deal with BKL removal
For 2.6 kernels the OpenAFS code is already BKL free, but the
corresponding header file smp_lock.h is still included in various
places. This header no longer exists, so remove all occurrences
outside of LINUX24.
Russ Allbery [Thu, 31 Mar 2011 21:08:43 +0000 (14:08 -0700)]
Declare dependency between openafs-client and openafs-fileserver inits
* Declare optional dependencies in the openafs-client init script on
openafs-fileserver so that, if the same host is running both a
fileserver and a client, the fileserver starts before and stops after
the client.
Marc Dionne [Sun, 27 Mar 2011 15:20:17 +0000 (11:20 -0400)]
Linux: 2.6.39: deal with BKL removal
For 2.6 kernels the OpenAFS code is already BKL free, but the
corresponding header file smp_lock.h is still included in various
places. This header no longer exists, so remove all occurrences
outside of LINUX24.
Andrew Deason [Fri, 25 Mar 2011 20:54:01 +0000 (15:54 -0500)]
viced: REALLYCLOSE origfdP after CoW
In StoreData_RXStyle, we save a pointer to the original FdHandle_t if
we need to CopyOnWrite the target vnode, for the purposes of possibly
copying additional data later on.
After the CopyOnWrite call, this points to an inode that is not in the
current volume, and is thus less likely to be accessed. In CopyOnWrite
itself, the original file handle is FDH_REALLYCLOSE'd because of this,
so the handle does not remain in the cache. Do the same in
StoreData_RXStyle, so origfdP is always FDH_REALLYCLOSE'd after we
have called CopyOnWrite.
Ben Kaduk [Fri, 17 Dec 2010 06:16:30 +0000 (01:16 -0500)]
new files for FreeBSD packaging at 1.6.0pre3
Packaging for use with FreeBSD's Ports Collection.
The directory layout is flattened, and a couple of files that do not
make sense here are not included.
The afsd rc script lives in packaging/ as it is not runnable in
its committed form, it requires variable substitution that occurs
at install-time from the Ports Collection.
Ben Kaduk [Mon, 22 Nov 2010 04:32:37 +0000 (23:32 -0500)]
Remove outdated rc file for afsd on FBSD
The semantics of rc scripts have changed drastically since this
was written. It will need to be rewritten from scratch; the
new version will live in src/packaging/FreeBSD as afsd.in, as it
is included with the port packaging in that form.
Jeffrey Altman [Wed, 16 Mar 2011 06:17:38 +0000 (02:17 -0400)]
Windows: avoid recursive cm_CheckOfflineVolume
Add a new cm_req_t flag CM_REQ_OFFLINE_VOL_CHK which is used
to prevent cm_Analyze() from performing recursive
cm_CheckOfflineVolume operations that will exhaust the stack.
Andrew Deason [Wed, 23 Mar 2011 16:07:03 +0000 (11:07 -0500)]
DAFS: Do not VDeregisterVolOp_r while exclusive
We should wait for a volume to transition out of an exclusive state
before calling VDeregisterVolOp_r on a volume, since some code may be
examining the vol op outside of VOL_LOCK in an exclusive state. We
should be doing this anyway before performing volume state checks,
since we may be trying to e.g. attach the volume at the same time.
Reviewed-on: http://gerrit.openafs.org/4287 Tested-by: Andrew Deason <adeason@sinenomine.net> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit d4ebebd0c3f89b65e93f0da6d53665271e6d2a61)
Russ Allbery [Thu, 24 Mar 2011 19:56:56 +0000 (12:56 -0700)]
Correctly document the AFS client setuid defaults
AFS no longer honors setuid status by default. Update the admin
guide documentation appropriately and add a warning recommending
against enabling setuid status given the limitations of the current
AFS protocol.
Reformat this section of the admin guide to make it easier to
maintain.
Simon Wilkinson [Wed, 23 Mar 2011 15:35:35 +0000 (15:35 +0000)]
rpm: Build srpms without %dist in their name
On RHEL6, %dist is set for all RPMS and SRPMS built on the system. This
leads to makesrpm.pl building SRPMS with .el6 in their release. As we
endeavour to use the same SRPM on all of our Fedora platforms, this is
less than ideal, so change makesrpm.pl to squash the setting of %dist
before building the RPM.
Reviewed-on: http://gerrit.openafs.org/4285 Reviewed-by: Alexander Ivan Redinger <aredinger@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit b0921071dd1b4d460d406985bba9619561444055)
Andrew Deason [Fri, 11 Mar 2011 00:37:44 +0000 (18:37 -0600)]
vol: Correct VolumeNumber for large volume IDs
VolumeNumber was using atoi to convert a volume header name to a
volume ID. This can return just -1 for volume IDs larger than 2^31-1,
though, so use strtoul instead.
Andrew Deason [Wed, 16 Mar 2011 19:44:56 +0000 (14:44 -0500)]
salvager: Fix volume parsing on 64-bit
When an unsigned long is wider than an afs_uint32, comparing the
afs_uint32 vid to ULONG_MAX is always going to be false (which the
compiler can warn us about). Fix this by storing to an unsigned long,
and converting to a volume id after ensuring that the result is not
too large.
Andrew Deason [Thu, 17 Mar 2011 15:43:23 +0000 (10:43 -0500)]
viced: Actually print client CPS
Client CPS was not getting output on SIGXCPU like the rest of the
client information, since the conditional in the CPS loop was never
true. Fix this so we traverse the CPS entries correctly.
Reviewed-on: http://gerrit.openafs.org/4256 Reviewed-by: Simon Wilkinson <sxw@inf.ed.ac.uk> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 5df51a82947f0ff60d57f551c5faa136ad7f6e0f)
DAFS: allow salvager to detect whether FSYNC server is DAFS
in order to enable the right commands and print the right errors,
query for DAFS-only FSYNC op.
doesn't use bozo InstanceInfo as it's possible you can run fileserver
outside of bos.
Simon Wilkinson [Thu, 17 Mar 2011 19:28:23 +0000 (19:28 +0000)]
rpms: Make makesrpm.pl cater for new rpm names
Starting from 1.6.0, our specfile no longer adds a stray '1.' to the
start of the release field. makesrpm.pl was failing to deal with
these new-style names and therefore not actually making an rpm.
Fix this by conditionalising on the OpenAFS version when selecting
which rpm to copy.
Reviewed-on: http://gerrit.openafs.org/4258 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Michael Meffie <mmeffie@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit af854b4e2f19e5f1a83b686e99ca3dd6122af942)
Simon Wilkinson [Sun, 20 Mar 2011 21:13:09 +0000 (21:13 +0000)]
util: Fix exec_alt
exec_alt was failing its tests on Mac OS X (but passing them on Linux).
It turns out that this is because it was failing to NULL terminate the
string that it creates in construct_alt(), which copies in the
characters from argv0, prefix and suffix, but never copies in a trailing
NULL.
Amend the code so that the trailing NULL from suffix is used to
terminate the string.
Reviewed-on: http://gerrit.openafs.org/4267 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 8c418408016ff8d6be9f85c4666a49954f61dbd0)
Andrew Deason [Thu, 30 Sep 2010 17:39:22 +0000 (12:39 -0500)]
viced: Allow checkout of VOL_STATE_DELETED volumes
The fileserver allows nonexistent volumes to be checked out. Since
VOL_STATE_DELETED volumes conceptually do not exist, we should also
allow them to be checked out.
Reviewed-on: http://gerrit.openafs.org/2873 Reviewed-by: Tom Keiser <tkeiser@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 88fc9ada4cfaee354199480916d139400ee0ef5b)