Andrew Deason [Mon, 23 Dec 2013 18:32:28 +0000 (13:32 -0500)]
RedHat: Munge future kernel versions
We currently look for "fc1?" (that is, fc10 through fc19) when trying
to munge the kernel version in some ways. This broke on Fedora 20,
since 20 obviously does not match "fc1?". Similarly, we look
specifically for "el6" for RHEL6 versioning quirks, but these will
break on RHEL7 and beyond.
Change the version checks so that this will work all the way through
Fedora 99 and RHEL 9. That won't work forever, but it will keep us
working for a few versions if the versioning quirks do not change.
Benjamin Kaduk [Thu, 9 Jan 2014 17:13:27 +0000 (12:13 -0500)]
ktc: fix up initializer for local_tokens
The old initializer was incomplete (initializing only one of the four
fields in the struct), which prompted warnings from clang
(-Wmissing-field-initializers):
../../../openafs/src/auth/ktc.c:149:2: warning: missing field 'server'
initializer [-Wmissing-field-initializers]
Since the variable is at file scope, it will be initialized to all
zeros anyway, and there is no need for an explicit initializer.
Jeffrey Altman [Thu, 9 Jan 2014 14:57:33 +0000 (09:57 -0500)]
Windows: Mark Irp Pending before Deferring
After CcDeferWrite() is called we no longer have access to the
current Irp. If we mark it deferred after calling CcDeferWrite()
we might mark the wrong thing.
Jeffrey Altman [Tue, 7 Jan 2014 15:57:01 +0000 (10:57 -0500)]
Windows: cm_ConnByServer fix search for replication
Separate connection objects are maintained for use when accessing
replicated and single source volumes. If the matching connection
type cannot be found while holding the cm_connLock shared a second
search is performed after the lock is upgraded to an exclusive lock.
This second connection search was not enforcing the replication criteria.
Jeffrey Altman [Tue, 7 Jan 2014 15:53:37 +0000 (10:53 -0500)]
Windows: cm_connLock not required for cm_GetUCell
In cm_ConnByServer() there is no need to hold the cm_connLock across
the cm_GetUCell() call. Obtain the cm_ucell_t object before the
cm_connLock is obtained.
Andrew Deason [Tue, 10 Dec 2013 23:02:34 +0000 (17:02 -0600)]
cellconfig: Do not use 'long' for dbserver IPs
A few places in this file assume that our dbserver IP addresses are
"long"s. A long int can be 8 bytes on some platforms, but we know
these IP addresses are all 4-byte integers. In the rare instances
where we have the maximum number of dbservers, this can overwrite a
bit of extra memory. This can also result in a misaligned access on
platforms such as SPARC v9, since the elements of he->h_addr_list are
not guaranteed to be 8-byte aligned.
So instead, treat these as 4-byte integers. For copying out of
he->h_addr_list, also use a memcpy anyway to be safe, since we are not
guaranteed alignment.
Arne Wiebalck [Fri, 13 Dec 2013 10:46:04 +0000 (11:46 +0100)]
make openafs uninstallable even if /afs is missing
The preuninstall scriptlet of the openafs RPM removes /afs. If, for
whatever reason, that directory does not exist, the scriptlet will
fail and hence break the deinstallation of the openafs package. The
proposed patch makes the scriptlet evaluate to true even if the /afs
has been removed by some other means and allows the package to be
uninstalled.
Andrew Deason [Tue, 17 Dec 2013 23:30:26 +0000 (17:30 -0600)]
LINUX: Use sock_create_kern where available
Currently, we use sock_create to create our Rx socket. This means that
accesses to that socket (sendmsg, recvmsg) are subject to SELinux
restrictions. For all recvmsg accesses and some sendmsg accesses, this
doesn't matter, since the access will be performed by one of our
kernel threads (running as kernel_t or something similar, which is
unrestricted). Such as: the rx listener, a background daemon, the rx
event thread, etc.
However, sometimes we do run in the context of a normal user process.
For some RPCs like FetchStatus, we tend to run the RPC in the
accessing user thread, which can result in us sendmsg()ing the data
packets with the initial arguments in the user thread. We can also
send delayed ACKs via rx_EndCall, and possibly a variety of other
scenarios.
In any of these situations when we are sendmsg()ing from a user
thread, SELinux can prevent us from sending to the socket, if the
calling user thread context is not able to write to an afs_t
udp_socket. This will result in packets not being sent immediately,
but the packets will be resent later, so access will work, but appear
very slow. This can easily happen for processes that are specifically
constrained by SELinux; for example, webservers are often constrained,
even if most of the rest of the system is not. This can be noticed by
seeing the 'resends' and 'sendFailed' counters rising in 'rxdebug
-rxstat', as well as noticing SELinux access failures if 'dontaudit'
rules are ignored.
To avoid this, use sock_create_kern to create the Rx socket, to
indicate that this is a socket for use by kernel code, and not
accessible by a user. This should cause us to bypass any LSM
restrictions (SELinux, AppArmor, etc). Add a configure check for this,
since this function has not always existed, according to
<https://lists.openafs.org/pipermail/openafs-devel/2004-June/010651.html>
Change-Id: I77e7f87e93be4d750d398e01dc1634efd80657bc
Reviewed-on: http://gerrit.openafs.org/10594 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Michael Meffie <mmeffie@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@your-file-system.com>
Andrew Deason [Tue, 17 Dec 2013 23:27:53 +0000 (17:27 -0600)]
rx: Remove obsolete comment
This comment refers to the fact that we used to be just checking for
SELinux to see if we should pass that extra argument. Ever since
commit cb1b41b159b98881f66319d7f65d941ba9fab911, we do have a better
test for this.
Ken Hornstein [Thu, 5 Dec 2013 18:57:36 +0000 (13:57 -0500)]
Remove extra whitespace from macro invocations
On MacOS X 10.9, the compiler has switched to LLVM and as a consequence
generates an error if there is a space between a macro invocation and
the starting left parenthesis.
Based on code originally done by Matt Haught <dmhaught@ncsu.edu>.
Jeffrey Altman [Thu, 5 Dec 2013 05:41:10 +0000 (00:41 -0500)]
Windows: RXAFS_GetVolumeStatus no PRSFS_READ check
Since d2d591caf2c9b4cf2ebae708cc9b4c8b78ca5a5a the file server no
longer performs a PRSFS_READ access check for the GetVolumeStatus RPC.
The cache manager should no longer test for PRSFS_READ as a means of
avoiding RPCs that are known to fail.
Jeffrey Altman [Wed, 27 Nov 2013 17:26:44 +0000 (12:26 -0500)]
Windows: RDR capture Cc/Mm exceptions do not break
All of the Cc and Mm functions are wrapped in try/except blocks.
The purpose is to ensure that Cc and Mm do not return an error as
an exception which could result in the afs redirector failing to
release a resource. Instead of calling the AFS exception handler
just handle the exception with EXCEPTION_EXECUTE_HANDLER. This permits
the __except block to capture the exception code.
The AFS exception handler will throw its own exception if the
AFSDebugFlags AFS_DBG_BUGCHECK_EXCEPTION bit is set. This is helpful when
debugging exceptions thrown by errors in the afs redirector code. It is
not helpful when a Cc function throws an exception. For example,
CcReadCopy() will throw STATUS_DELETE_FILE as an exception if an attempt
to read from a deleted file is initiated. This should simply fail the
read operation not BSOD the system.
Jeffrey Altman [Tue, 26 Nov 2013 15:52:45 +0000 (10:52 -0500)]
Windows: Rationalize Freelance vs "fs flush*"
Background:
cm_scache_t objects representing Freelance volume (cell=-1, volume=-1)
are special because they are populated from the Freelance mountpoint
and symlink tables. These tables are in turn generated from the
registry. The tables are regenerated on-demand after the execution of
cm_noteLocalMountPointChange() which increments cm_data.fakeDirVersion
which becomes the new data version value for the (-1.-1.1.1) directory
object.
The next time that cm_GetSCache() is called for a Freelance object
the fake root directory is rebuilt by cm_InitFakeRootDir(). Since the
vnode values are not persistent with regards to directory entry names the
FileId unique is used to distinguish the various versions.
cm_data.fakeUnique is incremented with each call to cm_InitFakeRootDir().
Each time cm_noteLocalMountPointChange() is executed the afs redirector is
notified of the data version change which will force the redirector to
rebuild its view of the directory the next time a path evaluation requires
evaluation of the root (\afs). In other words, on the next request.
If cm_noteLocalMountPointChange() is executed multiple times there is the
possibility of a race between the redirector and the service. When the
race is lost the redirector receives an invalidation event for -1.-1.1.1
as it is in the process of rebuilding the directory contents. The
redirector ends up believing it has the most recent data version when it
doesn't but the service no longer has Freelance mountpoint and symlink
tables representing the requested data version. Hence, the mountpoints
and symlinks end up as CM_SCACHETYPE_INVALID.
fs flushfile and fs flushvolume both had explicit checks to prevent
flushing Freelance objects because each call to cm_FlushFile() on a
Freelance object would execute cm_noteLocalMountPointChange() triggering
the race.
The Problem:
fs flushall is not executed on a specific object (volume or file).
Therefore there was no explicit check to prevent execution against
Freelance objects. For each cm_scache_t in the cache cm_FlushFile() is
processed. If there are N Freelance mountpoints and symlinks, there will
be N+1 calls to cm_noteLocalMountPointChange() in quick succession. Not
only does this risk losing the race described above but it is extremely
wasteful as the Freelance tables may be repeatedly regenerated.
This Patchset:
This patchset re-organizes the Freelance processing in the flush code
paths. cm_FlushFile() and cm_FlushVolume() can simply no longer be
successfully executed against a Freelance object. Both will return
CM_ERROR_NOACCESS.
"fs flush <file>" is not permitted against Freelance objects.
"fs flushvolume <path>" will execute cm_noteLocalMountPointChange() once if
the path is a Freelance object.
"fs flushall" continues to execute cm_FlushFile() on all cm_scache_t
objects. The calls on Freelance object will fail. After all cm_scache_t
objects are flushed then cm_noteLocalMountPointChange() will be executed
once to force the Freelance directory to be rebuilt.
This patchset does not address the race but significantly reduces the
likelihood the race will be lost.
Stephan Wiesand [Thu, 21 Nov 2013 14:01:29 +0000 (15:01 +0100)]
Linux: Fix build for older kernels w/o bool
Commit b7f4f2023b2b3e1aac46715176940fb50cc75265 broke builds against
older kernels which don't have bool defined in linux/types.h . Fix
this by using unsigned char instead of bool for the static inline
functions.
Change-Id: Icbb82446ef66edd2650f33135ed6ccd2b8a920b2
Reviewed-on: http://gerrit.openafs.org/10483 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Anders Kaseorg <andersk@mit.edu> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com>
Ken Hornstein [Wed, 20 Nov 2013 18:37:52 +0000 (13:37 -0500)]
Support for changes to OS X Mavericks VNOP_SYMLINK() function.
Add support for an extra argument to afs_symlink() to return the
newly-created symlink vnode if requested (this is needed on OS X
Mavericks). On OS X Mavericks return the newly-created symlink vnode in
the symlink vnops functions, on all other platforms ignore it.
It turns out that technically OS X has required the symlink to be
created for a while, but code inside of symlink() would call namei() on
the symlink name if the returned vnode point was NULL. The difference
is that on Mavericks the Manditory Access Control Framework has been
enabled, and that turns on some extra code which unconditionally calls
vnode_mount() on the returned vnode pointer, which ends up causing a
panic
Jeffrey Altman [Fri, 15 Nov 2013 23:32:37 +0000 (17:32 -0600)]
Windows: cm_FindVolumeByFID
cm_GetVolumeByFID() does not query the vldb if the volume group
is not known to the cache manager. cm_FindVolumeByFID() is to
be used in cases where the volume group data must be known for the
operation to successfully complete.
Jeffrey Altman [Tue, 12 Nov 2013 14:58:44 +0000 (09:58 -0500)]
Windows: Fix out of range pointer validation
The ACL, Stat, and Volume pointer validation checks did not take
into account that NULL is a valid pointer value. As a result the
cache validation failed.
Calling GetSystemTime() followed by SystemTimeToFileTime() performs
two format translations which can be avoided by using
GetSystemTimeAsFileTime() directly.
Anders Kaseorg [Thu, 7 Nov 2013 20:37:25 +0000 (15:37 -0500)]
Linux: Get rid of !STRUCT_KEY_UID_IS_KUID_T case
On the few kernel versions before struct key.uid was converted to
kuid_t (v3.7-rc1~147^2~76), it was not possible to enable both
CONFIG_KEYS and CONFIG_UIDGID_STRICT_TYPE_CHECKS, so this case was
impossible. That’s good, because it also had a typo in its
implementation (and was confusing to deal with correctly).
Change-Id: I4ecd164ed3604558ed4419bf6f9d531bd5d1a9ff Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Reviewed-on: http://gerrit.openafs.org/10443 Reviewed-by: Derrick Brashear <shadow@your-file-system.com> Reviewed-by: Stephan Wiesand <stephan.wiesand@desy.de> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Tested-by: BuildBot <buildbot@rampaginggeek.com>
WARNING: CPU: 3 PID: 657 at /build/buildd/linux-3.12.0/lib/kobject.c:196 kobject_add_internal+0x1f4/0x300()
kobject_add_internal failed for afs with -EEXIST, don't try to register things with the same name in the same directory.
and leads to general system instability. This can be reproduced by
starting AFS twice with an empty cache, dynroot disabled, and no
network.
Change-Id: I8ec1ed365c5b3cf60bd34af0aca94e0c496bcaa3 Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Reviewed-on: http://gerrit.openafs.org/10448 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com>
Andrew Deason [Thu, 7 Nov 2013 06:05:39 +0000 (00:05 -0600)]
doc: backup manpage fixes
- Consistently specify -dryrun and -n across various subcommands.
Many did not list -n, some listed -n but no -dryrun, and some
listed -noexecute instead.
- backup_volrestore: Add missing option -usedump
- backup_deletedump: Add missing options -groupid, -dbonly, -force,
and -portoffset
Andrew Deason [Thu, 7 Nov 2013 06:03:17 +0000 (00:03 -0600)]
backup deletedump: Change -port to -portoffset
Use -portoffset instead of -port, for consistency with all of the
other backup suite commands. Leave -port in as an alias, for backwards
compatibility. Currently -port will mean -portoffset anyway, since
it's an unambiguous abbreviation, but put in the alias explicitly,
just in case some other option comes along starting with -port.
Andrew Deason [Tue, 5 Nov 2013 17:00:21 +0000 (11:00 -0600)]
backup: Display general help on -help
Currently, 'backup' tries to guess if we are running 'backup
interactive' before libcmd actually parses our arguments. This is
tricky, since we run 'backup interactive' if no explicit subcommand is
actually given. One consequence of this is that currently, running
'backup -help' just displays the help for the 'backup interactive'
command, not the help output for 'backup' itself.
The current heuristics for guessing at whether we are running 'backup
interactive' or not are a bit fishy, but at least for now, just make
sure -help works. This should still ensure any other behavior is
unchanged, but just 'backup -help' now works like other command
suites.
Andrew Deason [Wed, 12 Jun 2013 22:48:46 +0000 (17:48 -0500)]
doc: Add 'checkman' tool
Add the 'checkman' script, which compares a command's "-help" output
to the options actually documented in its manpage. This command is
certainly not perfect, and may contain false negatives and false
positives. It is not (currently) intended to be run as an automated
check, but is meant to assist a human manually checking the
correctness of man pages. An error reported by 'checkman' does not
necessarily indicate something that should actually be changed.
Andrew Deason [Fri, 14 Jun 2013 20:37:27 +0000 (15:37 -0500)]
bos: Remove MR-AFS commands and options
The blockscanner and unblockscanner commands, as well as many options
to "bos salvage", were only of use to MR-AFS. MR-AFS is not used
anywhere anymore, and these commands and options were largely
undocumented, so get rid of them. See
<https://lists.openafs.org/pipermail/openafs-info/2013-January/039215.html>.
Thanks to Hartmut Reuter for providing information about this.
Andrew Deason [Thu, 31 Oct 2013 04:28:16 +0000 (23:28 -0500)]
volserver: Restore -allow-dotted-principals
Commit cd3492d0 converted volserver command-line parsing to use
libcmd. However, it accidentally also changed the
-allow-dotted-principals option to -dotted. Change it back to
-allow-dotted-principals for consistency with previous versions, as
well as other server processes.
Note that currently there are no public releases of OpenAFS containing cd3492d0, so no public release has contained the -dotted option.
Anders Kaseorg [Thu, 31 Oct 2013 13:11:59 +0000 (09:11 -0400)]
Linux: Fix build with CONFIG_UIDGID_STRICT_TYPE_CHECKS (user namespaces)
With CONFIG_UIDGID_STRICT_TYPE_CHECKS (a dependency of user namespace
support, CONFIG_USER_NS) turned on, uid_t and kuid_t are different
types, as are gid_t and kgid_t, and we need to use namespace-dependent
functions to convert between them.
We can’t use init_user_ns as the namespace because it’s GPL-only, so
instead we grab the current user_ns at module load time.
This is required to support kernels with user namespace support. We
don’t yet have full support for independent AFS use by different users
in a multiuser container; that will need to wait for future work.
Change-Id: Icc03f9098dd25b483d406db5167264ba960cdcb7 Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Reviewed-on: http://gerrit.openafs.org/10386 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com>
Anders Kaseorg [Tue, 5 Nov 2013 06:11:15 +0000 (01:11 -0500)]
afs_linux_pag_from_groups: Stop checking for NOGROUP sentinel
Linux hasn’t used NOGROUP as a sentinel like this since before kernel
2.1.12, and OpenAFS hasn’t used it on Linux since commit 109927bf6f54b58b76ac48ba41c2012c74937fed (Remove pre-Linux 2.6
support).
Change-Id: I0b18de8e5d9b6cd9b20da43ed050163c2d8651ff Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Reviewed-on: http://gerrit.openafs.org/10426 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com>
Andrew Deason [Wed, 30 Oct 2013 02:11:54 +0000 (21:11 -0500)]
Exit successfully on -help
Running a command with -help is not an error. cmd_Dispatch handles
this correctly, but several server processes call cmd_Parse directly,
and exit with failure on -help. Make them exit successfully instead.
Andrew Deason [Mon, 4 Nov 2013 00:59:08 +0000 (18:59 -0600)]
Do not hide -enable_{peer,process}_stats
Both afsd and kaserver accepted the -enable_peer_stats and
-enable_process_stats options, but they did not include the options in
their usage message. We already document these flags in the manpages,
so also include them in the usage message; they are not a secret.
Andrew Deason [Thu, 5 Sep 2013 05:48:02 +0000 (00:48 -0500)]
doc: Symlink all dafssync-debug pages
For every fssync-debug subcommand, provide a symlink for the
associated dafssync-debug subcommand. This way, running e.g.
"man dafssync-debug_attach" will actually give you a manpage, instead
of needing to specifically run "man fssync-debug_attach".
Andrew Deason [Mon, 4 Nov 2013 16:04:42 +0000 (10:04 -0600)]
doc: Fix livesys output formatting
If we indent text here, the formatting codes are not interpreted, and
the text is output "raw". So currently, we actually see
"I<system_type>" in this section, which is a bit confusing.
Saying the actual output with string substitutions and stuff here
doesn't seem very helpful when the output doesn't have any constant
text in it. Just describe what the output is instead; an example
immediately follows if this is unclear.
Andrew Deason [Sun, 30 Jun 2013 02:06:51 +0000 (21:06 -0500)]
doc: Make all vos pages =include common options
Many manpages for "vos" subcommands were not using the "common" vos
fragments, and instead were just repeating the information directly in
their manpage. Make them all use the "common" vos fragment to avoid
duplication.
Andrew Deason [Mon, 4 Nov 2013 01:04:30 +0000 (19:04 -0600)]
doc: bosserver manpage fixes
Fix -noauth documentation. The current documentation is referring to
the client-side -noauth option, but this is actually the server-side
-noauth option, which is very different.
Andrew Deason [Fri, 14 Jun 2013 20:33:19 +0000 (15:33 -0500)]
doc: buserver manpage fixes
- Add missing -ubikbuffers
- Remove -enable_peer_stats and -enable_process_stats; we don't
actually accept these options
- Fix synopsis formatting, so option arguments are on the same line
as the option name
- Fix the -noauth documentation. The current text is referring to the
client-side -noauth option, but this is actually the server-side
-noauth option, which is very different.
Andrew Deason [Sun, 3 Nov 2013 23:58:02 +0000 (17:58 -0600)]
doc: Fix udebug -port bullet list
The formatting gets screwed up if we have multiple =item tags together
like this. To just have each one be a bullet point, just have a bare
=item before each one, without a "tag" or "key" for the =item.
Ken Dreyer [Mon, 14 Oct 2013 17:39:02 +0000 (11:39 -0600)]
doc: add linked cells description to man pages
The man pages previously described linking DCE cells to AFS cells.
OpenAFS and YFS also allow linking between two AFS cells. Update the
description of linked cells in CellServDB(5), aklog(1), and
fs_newcell(1) to refer to AFS instead of DCE.
Add a linked cell example to the CellServDB man page with an
explanation.
The afs_CacheTruncateDeamon() thread will not sleep until both
'afs_CacheTooFull' and 'afs_WaitForCacheDrain' are true but the
thread will stop freeing space in the cache when 'afs_CacheTooFull'
is true which prevents 'afs_WaitForCacheDrain' from ever becoming
true if it is not already.
Make the conditional for doing work include 'afs_WaitForCacheDrain'.
afs_WakeCacheWaitersIfDrained is called as the last statement
of both afs_DiscardDCache and afs_FreeDCache. There is no need
to perform the same check again before exiting afs_FlushDCache.
For consistency with afs_FlushDCache and afs_DiscardDCache
include afs_blocksDiscarded in the free space test. When afs_FreeDCache
is called it should be zero.
Jeffrey Altman [Mon, 28 Oct 2013 05:06:47 +0000 (01:06 -0400)]
Windows: RDRLib Worker Thread shutdown
The thread waiting to unload the library is polling the state of
the AFS_WORKER_INITIALIZED flag in the PoolContext->State field
for each worker. Ensure that the thread performs no actions other than
PsTerminateSystemThread() after clearing the AFS_WORKER_INITIALIZED flag.
Jeffrey Altman [Tue, 29 Oct 2013 12:24:06 +0000 (08:24 -0400)]
Windows: Uninitialized TargetNameLength/Offset
In RDR_PopulateCurrentEntry it is possible for TargetNameLength and
TargetNameOffset to be uninitialized resulting in stack garbage being
returned to the redirector. This can result in a blue screen.
Jeffrey Altman [Fri, 18 Oct 2013 23:26:56 +0000 (19:26 -0400)]
Windows: cm_NewSCache skip in hash recycled entries
If cm_RecycleSCache returns an in-hash entry it means that either
it wasn't recycled properly or somehow we raced this entry with
another thread. Just skip it and keep searching.