Rod Widdowson [Sun, 25 Aug 2013 16:16:39 +0000 (09:16 -0700)]
Windows: Pin the Cc FileObject during section create.
This means that if we purge the data cache while the section is being
created then the MJ_CLOSE will not happen until we unpin the FO.
Thus we can drop any embarsssing locks prior to the close and
meddling antivirus products can do odd stuff in the close path.
Note that there may not be a file object, but in that case there
will be no close on the purge since any CcInitialize operations
will wait on us dropping the SOP lock exe - hence the SOP cannot
be set up.
Also note that this only applies to the data section,
but we do not purge the image section.
Refactor AFSPerformObjectInvalidate so that all of the non-DIRECT_IO
processing variables are in the Extents processing section. Remove
all references to Extents processing from the DIRECT_IO block.
Jeffrey Altman [Thu, 22 Aug 2013 21:50:39 +0000 (17:50 -0400)]
Windows: Refactor AFSVerifyEntry AFSValidateEntry
Inside a big switch statement it is hard to follow when there
are multiple 'break' exits within a 'case'. Reorganize the code
so that there is only a single exit for the FILE type. Unnecessary
blocks are removed as well.
Section Object Resource acquires and releases are lost in the
noise of all of the rest of the locks. Introduce a dedicated
subsystem just for Section Objects.
Jeffrey Altman [Wed, 21 Aug 2013 16:27:35 +0000 (12:27 -0400)]
Windows: Call AFSExeceptionFilter for all exceptions
In many cases we capture exceptions record and the Exception Code
as ntStatus and move on with life. This patchset changes that.
All exceptions are passed to AFSExceptionFilter so we do not miss
anything.
Andrew Deason [Wed, 21 Aug 2013 22:07:14 +0000 (17:07 -0500)]
viced: Clarify comment explaining cba sorting
The current comment here is very brief; it may not be immediately
clear to a reader why we are sorting these, and so why we need the
given CBAs in an array. Expand on it a bit.
Note that it seems like it might be possible to refactor multi_Rx to
not require all calls to be created before any packets are sent. If
multi_Rx were changed to send data as we create calls, it may be
possible to eliminate this sorting, and allow for slightly more
efficient callback traversal when breaking callbacks.
Jeffrey Altman [Sat, 17 Aug 2013 14:18:53 +0000 (10:18 -0400)]
Windows: Cap Cache Size on X86
Since we know the cache size cannot be arbitrary size because it
must fit into contiguous process memory and because it is difficult
to compute the actual size limit, cap the size to 716800KB.
Jeffrey Altman [Fri, 16 Aug 2013 19:36:32 +0000 (15:36 -0400)]
Windows: Do not recycle deleted scache on refcnt 0
If the scache object with CM_SCACHEFLAG_DELETED set is recycled
then the deleted state is lost and the cache manager cannot prevent
unnecessary FetchStatus queries to the file server.
Jeffrey Altman [Fri, 16 Aug 2013 16:01:55 +0000 (12:01 -0400)]
Windows: Do not remove scp from hash table on deletion
If the CM_SCACHEFLAG_DELETED flag is going to have any benefit, the
cm_scache object must not be removed from the hash table in response
to a VNOVNODE error. Otherwise, a new cm_scache object is allocated,
the CM_SCACHEFLAG_DELETED is not found, and a new callback request
is issued to the file server which in response returns VNOVNODE.
Do this enough times and the abort threshold is triggered and then
the application becomes very unhappy with performance.
Ben Kaduk [Wed, 17 Jul 2013 00:39:56 +0000 (20:39 -0400)]
Check for over/underflow while allocating PTS ids
The behavior of signed integer over/underflow is implementation-defined,
but even if the compiler is nice and just wraps around, we could get
ourselves into trouble later on.
Ben Kaduk [Wed, 31 Jul 2013 00:17:01 +0000 (20:17 -0400)]
Do not use a non-literal format string
Now that UKERNEL's panic() is a proper varargs function (gerrit 9877),
we can use a literal format string "%s" to print the panic message.
clang warngs about a non-literal format string, and in some build
environments the warning becomes fatal via -Werror.
Andrew Deason [Wed, 31 Jul 2013 20:58:41 +0000 (15:58 -0500)]
budb: Do not use garbage cellinfo
If the -servers option is given, we never initialize cellinfo or the
clones array. So, don't give the cellinfo structure or the clones
array to ubik in that case, or we may crash or do other weird things.
This issue appears to have been introduced in commit fc4ab52e.
Andrew Deason [Thu, 1 Aug 2013 19:06:52 +0000 (14:06 -0500)]
DAFS: Remove AFS_DEMAND_ATTACH_UTIL
Currently we have two DAFS-related preprocessor defines in the
codebase: AFS_DEMAND_ATTACH_FS and AFS_DEMAND_ATTACH_UTIL. DAFS_FS is
the symbol for enabling DAFS code, and turns on demand attachment and
all of the related complicated volume handling; it requires pthreads.
DAFS_UTIL is supposed to be used for utilities interacting with DAFS,
but do not have pthreads and so cannot build the relevant threads for
e.g. the VLRU, so they don't support demand attachment and a lot of
more advanced volume handling techniques.
Having both of these exist is confusing. For example, currently in
partition.c we only initialize dp->volLockFile for DAFS_FS, even
though the structure exists if _either_ DAFS_FS or DAFS_UTIL is
defined. This means when only DAFS_UTIL is defined, volLockFile will
exist in the partition structure, but will be uninitialized!
Amongst other possible issues, this means right now that DAFS_UTIL
users (dasalvager is the only one right now) will try to use an
uninitialized volLockFile whenever they try to use a volume that needs
locking. Since the partition struct is usually initialized to all
zeroes, this means we'll try to issue a lock request for FD 0,
whatever FD 0 is. If FD 0 is not open, we'll fail with EBADF and bail
out. But if FD 0 is open to some random file, the lock will probably
succeed, and we'll proceed without actually locking the volume lock
file. While the fssync volume checkout mechanism still works, the
on-disk locking mechanism protects against race conditions the fssync
volume checkout mechanism cannot protect against, and so handling
volumes in this way is not safe.
This is just one example; there are other issues with the partition
headerLockFile and probably may other things; most instances of
DAFS_FS really should be enabled for DAFS_UTIL as well.
So, instead of trying to account for and fix all of these problems
individually, get rid of AFS_DEMAND_ATTACH_UTIL, and just use
AFS_DEMAND_ATTACH_FS. This means that all relevant code must be
pthreaded, but since the only relevant code is for the dasalvager, we
can just make dasalvager pthreaded. Salvaging does not make use of any
threads or LWPs, so this should not have any side-effects.
Thanks to Ralf Brunckhorst for reporting the issue where we encounter
EBADF when FD 0 is not open, leading to the discovery of this.
Anders Kaseorg [Tue, 23 Jul 2013 18:37:26 +0000 (14:37 -0400)]
volume_inline.h: Down with assert, again
Commit 34767c6a0f914960c9a1efabe69dd9c312a2b400 replaced all assert
calls in this file with osi_Assert (now opr_Assert), but shortly
thereafter, commit db6ee95864a8fc5f33b7e95c19c8ff5058d37e92 added
VTimedWaitStateChange_r with two new assert calls. These are
precarious in a public header; fix them to opr_Assert like the ones in
VWaitStateChange_r.
der-protos.h was generated from Heimdal headers which in turn were
auto-generated. The included a large number of function prototypes
of the form
ret-type func(parm-list, type */* comment */);
where the combination of */* is ambiguous. Does it mean an end comment
followed by a pointer declaration or a pointer declaration followed by
a begin comment. This combination generates warnings on Windows. The
bug was fixed in Heimdal's code generator. Fixing it here by editing
the code.
Michael Laß [Sun, 14 Jul 2013 19:31:27 +0000 (21:31 +0200)]
Use -nofork when starting bosserver via systemd
Systemd does not expect the started process to fork unless
"Type=forking" is given. Use -nofork to run BOS in foreground and allow
systemd to track its state.
Change-Id: I024be12b410d6b8299edd16f309d816a3df469ed
Reviewed-on: http://gerrit.openafs.org/10087 Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Derrick Brashear <shadow@your-file-system.com> Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Mark Vitale <mvitale@sinenomine.net> Tested-by: Michael Laß <lass@mail.uni-paderborn.de> Tested-by: Ken Dreyer <ktdreyer@ktdreyer.com> Reviewed-by: Ken Dreyer <ktdreyer@ktdreyer.com>
Ben Kaduk [Fri, 12 Jul 2013 16:43:57 +0000 (12:43 -0400)]
Update the asetkey man page for rxkad-k5
Also add the usage for the six-argument form while here.
Update some generic text to account for the existence of rxkad-k5,
and mention that the Update Server is not the only thing which can copy
around KeyFiles. Give an example of the seven-argument form's usage for
rxkad-k5.
Derive DES/fcrypt session key from other key types
If a kerberos 5 ticket has a session key with a non-DES enctype,
use the NIST SP800-108 KDF in counter mode with HMAC_MD5 as the PRF to
construct a DES key to be used by rxkad.
To satisfy the requirements of the KDF, DES3 keys are first compressed into a
168 bit form by reversing the RFC3961 random-to-key algorithm
Windows has three additional places to get tokens, who knew?
Krb5 ticket support for server-to-server and localauth
Create a tkt_MakeTicket5 that creates a native krb5 rxkad token with
a service key supported by the rfc3961 library (session keys must be
provided as DES)
Update GenericAuth to search for rxkad_krb5 keys and call tkt_MakeTicket5
if it finds any.
Decrypt tickets with non-des enctypes by calling out to the rfc3961 library.
This requires the security object to be given an enhanced get_key callback
that supports looking up keys by enctype.
Include a wrapper around afsconf_GetKeyByTypes so rxkad doesn't have
to know anything about libauth internals/interfaces
When afs_linux_readdir detects a corrupt directory, the resulting
error message is more useful if it includes the directory's FID
instead of just a kernel inode pointer.
When a corrupt directory is discovered, scanning stops immediately and
readdir returns ENOENT. Currently, the vcache lock is unlocked and the
dcache containing the directory is released, but that's not enough.
It's also necessary to unlock the dcache, on which we hold a read lock,
and to clear the vcache state which records an in-progress readdir.
When reading a callback state dump, check the return values from
read(2) instead of ignoring them. This adds a new static function,
ReadBytes(), which handles reading a requested number of bytes from a
file and bailing if there is an error.
Michael Meffie [Thu, 4 Apr 2013 14:53:00 +0000 (10:53 -0400)]
vlserver: increase the max lwp threads
Increase the maximum LWP threads allowed from 16 to 64. Increasing the number
of LWP threads can reduce the number of calls waiting for threads on
busy vlservers.
Ben Kaduk [Tue, 25 Jun 2013 00:51:53 +0000 (20:51 -0400)]
Make KeyFileExt comment match reality
There is no file format version field as implemented.
Remove it from the format laid out in the comment, and change the
associated text to make more sense.
Ben Kaduk [Wed, 15 May 2013 15:38:53 +0000 (11:38 -0400)]
Document the prdb (ubik) file format
Briefly cover the ubik header and mention that it is not part of the
logical database (since it is just used for the consistency procedure).
Describe the fields of the prheader and how they are used. Mention that
all subsequent entries are blocks of the same size, whose type can be
distinguished by a shared flags field. User and group entries are similar,
and supergroup entries are described as a diff from regular group entries,
as only a handful of fields change. Continuation entries can be used
for user, regular group, or supergroup entries.
Call out what fields are invariant within which classes of entry, so that
these properties can be preserved (or knowingly eliminated) for future
extensions to the format.
Russ Allbery [Sat, 29 Jun 2013 21:29:06 +0000 (14:29 -0700)]
Fix restorevol sanity check on afs_int32
restorevol reads various values of different lengths into an
afs_int32 and does a sanity check to ensure that there is enough
room to store the desired value length. However, the check was
done against the wrong variable, making it ineffective.
This check is unlikely to ever trigger, but fix it just in case.
Russ Allbery [Sat, 29 Jun 2013 21:27:55 +0000 (14:27 -0700)]
Fix restorevol crash on corrupt nDumpTimes value
If the number of dump times claimed in the volume header was greater
than MAXDUMPTIMES, restorevol would happily write over random stack
memory and crash. Sanity-check the loaded value and cap it to
MAXDUMPTIMES with a warning.
Bug found by Mayhem and reported by Alexandre Rebert.
Michael Meffie [Thu, 16 Jul 2009 21:50:53 +0000 (17:50 -0400)]
bosserver dir creation for non-transarc paths
The bosserver attempts to create the server directories with the correct
permissions when bosserver starts. Make the parent directories if needed
as well, using the umask permissions for the parent directories, instead
of failing.
This adds a Perl program, src/afs/findlocks, which grovels through the
kernel module source tree, finds every location where a lock is obtained,
and produces an index of lock site ID numbers. This can be used to find
a lock when debugging, or when picking a new number.
Michael Meffie [Tue, 9 Apr 2013 08:00:16 +0000 (04:00 -0400)]
libafs: initialize hard mount last errors
Initialize the values of the server last errors
introduced in commit 94a8ce970d57498583e249ea61725fce1ee53a50
to avoid logging garbage for the last error codes.
Marc Dionne [Mon, 8 Jul 2013 14:53:00 +0000 (10:53 -0400)]
Linux 3.11: Convert from readdir to iterate file operation
Convert the readdir function so that it can be used as the new
"iterate" file operation. This new operation is passed a context
that contains a pointer to the filldir function and the offset.
The context is passed into the new dir_emit function that will
call the function specified by the context.
The new dir_emit function returns true on success, so we must be
careful about how we check for failure since this is different
behaviour from what filldir currently does.
IBM created a global variable 'afsconf_SawCell' in the kauth package
and manipulated its value from within bos.c as part of the calling
of ka_Init(). Patchset d52398940d58ccdba4114a9975762f48cc24ad15
exported afsconf_SawCell from afsauthent.dll since bos.exe is built
pthreaded. It was previously mixing pthread and not-pthread libraries
to access the variable.
Unfortunately, the export was declared as a function pointer instead
of DATA. Importing a DATA element from a library also requires that
the variable be __declspec(dllimport). The use of afsconf_SawCell
needs to be replaced but in the meantime fix the import so that bos.exe
can start without crashing.
Build the 3961 library in userspace, with support for the common
kerberos enctypes des3-hmac-sha1, arcfour-hmac-md5, and aesXXX-cts-hmac-sha1-96.
Export new symbols as well, and suppress deprecated warnings.
Rename all symbols, both exported and not, with an oafs_h_ prefix
so as to reduce the chance of conflicts.
Rename heim_octet_string to avoid confusion about where things are coming
from.
Ben Kaduk [Tue, 2 Jul 2013 15:13:25 +0000 (11:13 -0400)]
Disable some heimdal bits
For the rfc3961 library, we don't want linker exposure to an
entropy-gathering daemon, and we don't want to be leaving entropy
seed files around in user homedirs.
Ben Kaduk (1):
Be friendly to krb5_generate_random_block consumers
Jeffrey Altman (6):
roken: include limits.h unconditionally
Avoid unused variable warning on Windows
roken: include direct.h if HAVE_DIRECT_H
roken: Define S_IRWXU and friends on Windows
roken: Add rk_mkdir()
libkrb5: Add missing KRB5_LIB_FUNCTION/KRB5_LIB_CALL
Marc Dionne [Fri, 5 Jul 2013 16:50:36 +0000 (12:50 -0400)]
bos: Do encryption if requested
Commit d008089a79 didn't replace the processing of the aencrypt
flag passed to the GetConn() function, causing all bos connections
to be un-encrypted. This causes "addkey" to fail with an error
from the server, and "listkeys" to silently ignore the -showkey
option to display keys.
Set the AFSCONF_SECOPTS_ALWAYSENCRYPT flag, and don't set
AFSCONF_SECOPTS_FALLBACK_NULL since fallback is not acceptable if
the caller requested enrcyption.
Jeffrey Altman [Wed, 26 Jun 2013 15:00:00 +0000 (11:00 -0400)]
Windows: Protect all Mm and Cc calls with try..except
Wrap all Memory Manager and Cache Manager operations in a try..except
block to protect against leaking the SectionObjectResource if an exception
is thrown. Failure to release the SectionObjectResource will result in
subsequent deadlocks.
Don't assume that converting a UUID to a string will always succeed.
Instead, opr_uuid_toString should return a status result to indicate
whether the operation was successful or not.
Jeffrey Altman [Wed, 19 Jun 2013 17:53:51 +0000 (13:53 -0400)]
Windows: Wake waiters on failed cm_SyncOp exit
If cm_SyncOp exits due to failure and there are threads waiting
to use the cm_scache object, wake them before exiting because there
will be no cm_SyncOpDone() operation to wake them later.
projects / packages / o / openafs.git / log