Windows: Redirector interface for afsd_service.exe
Over the last three years the afsd_service sources have been
gradually separated into distinct layers for the SMB server
and the AFS cache. The eventual goal of this work was to
permit the addition of alternative interfaces to the cache
manager in parallel.
This patchset implements the first alternative interface,
a reverse ioctl model that communicates with a native IFS
redirector driver. The driver will be submitted in a
subsequent patchset.
Although it is possible to run afsd_service with both the
SMB and RDR interfaces active at the same time. In practice
it is somewhat impractical because it destroys the uniformity
of the \\AFS name space. The RDR loads at boot time and claims
all of \\AFS. The SMB interface if active at the same time
must use the old \\%HOSTNAME%-AFS. As implemented, if the RDR
interface is functional the SMB interface is not started. Only
if the RDR interface fails will the SMB interface be activated.
The afsd_service.exe maintains all of its primary responsibilities
for communicating with the AFS servers, processing callbacks,
enforcing permissions, handling afs path ioctls, Windows RPC
service simulation, and object management. The biggest change
is in the cm_buf_t management. Data is exchanged with the
RDR by passing control over cm_buf_t->data buffers in the form
of Windows File Extents. This avoids data copies across a
communication channel which significantly improves performance
at a substantial complexity cost.
Credential management is switched from a Windows username binding
to a GUID binding where the GUIDs represent authentication groups
that are managed by the RDR.
This patchset includes additional changes to support integrated
logon in conjunction with the RDR. In particular, adding support
for authentication groups.
Stefan Kueng [Thu, 15 Sep 2011 04:57:23 +0000 (00:57 -0400)]
Windows: Explorer Shell extensions
This patchset implements a broad range of improvements to
the explorer shell. There is still a significant amount of
work to be done.
* Remove the 'cut' and 'delete' options from the
context menu if the selected object is a symlink
or mount point. This is performed in a language
neutral manner.
* Add AFS Property page to the property sheet
* Add AFS Volume Property page
* Add AFS ACL Property page
* force the linker to add the common-controls V6 manifest and
define ISOLATION_AWARE_ENABLED to make property sheets
work for the shell extension dll
* Fix the InfoTip handler. Display symlink and mount point
target strings
Marc Dionne [Wed, 14 Sep 2011 21:16:52 +0000 (17:16 -0400)]
Linux: Remove use of undefined AFS_LINUX_ENV
AFS_LINUX_ENV is used in a few places, mostly from commit dc077b83,
but it is not defined anywhere. As a result the logic was not as
intended.
In the definition of osi_ThreadUnique, this means that on all
Linux we use getpid(), which is OK as it gets defined appropriately
under LINUX and LINUX24 as current->pid or current->tgid.
Ben Kaduk [Tue, 13 Sep 2011 01:54:09 +0000 (21:54 -0400)]
Tidy up event hash table definitions a bit more
The change in http://gerrit.openafs.org/5314 consolidated the
definitions of struct afs_event and some related objects, but
the definitions were not quite in the right place. In
http://gerrit.openafs.org/5392 this content was moved to more
standard headers, but a comment and #define were missed.
This change moves the comment to be in the same place as the content.
Simon Wilkinson [Tue, 13 Sep 2011 08:54:12 +0000 (09:54 +0100)]
Use rra-c-util m4 from src/external
This commit switches us over from using manually maintained copies
of Russ's rra-c-util m4 macros, to using ones that are automatically
imported from his upstream git tree.
The macro versioning in the OpenAFS tree was slightly confused. This
(approximately) updates us from using version 3.3 of the macros to
version 3.8. The signifcant changes are:
*) Use PATH_KRB5_CONFIG rather than KRB5_CONFIG to set the location
of the krb5-config file
*) Use --with-gssapi-{include, lib} if given, rather than krb5-config
*) Use --with-krb5-{include, lib} if given, rather than krb5-config
*) Define HAVE_KERBEROS if we find a Kerberos library
The first, in particular, is likely to cause some confusion.
Simon Wilkinson [Tue, 13 Sep 2011 08:37:04 +0000 (09:37 +0100)]
Add rra-c-util to the src/external regime
We're already using a number of m4 configure tests which are taken
from rra-c-util. Instead of importing these piecemeal, add rra-c-util
to the src/external git import system, so that we can simply keep our
selves in sync
Jeffrey Altman [Tue, 2 Aug 2011 22:33:06 +0000 (18:33 -0400)]
Windows: osi_Debug macros
The osi_Debug macros are like osi_Log macros except when used
the logged values are always printed using OutputDebugString().
This interface is meant only for temporary debugging. No code
that uses osi_Debug should ever be committed to the upstream
repository.
shutdown_rx, unlike rx_Finalize, kills the socket before the conns.
since we call osi_NetSend directly, we lose. just do a simple
check for rxinit_status, and exit immediately before sending if rx
is not up.
ignore KRB5_DEPRECATED_FUNCTIONS for now since it doesnt give any
advice. Some keytypes (but not all) have been renamed. So for
now we need to keep both apparently.
Harald Barth (1):
Move common code to krb5_unsupported_enctype() and make error message contain string instead of error number
Jeffrey Altman (1):
roken: declare IN_LOOPBACKNET if necessary
Love Hornquist Astrand (5):
Warning fixes from Christos Zoulas
Fixes from NetBSD via Thomas Klausner and Roland C. Dowdeswell
Rename subsystem_DEPRECATED to subsystem_DEPRECATED_FUNCTION(X)
remove trailing whitespace
remove warning, remove forward declaration by moving the function up, ident
Love Hörnquist Åstrand (4):
don't set i = 0, its never read
sprinkle doxygen and kode more like the rest of the code base
partly unify enctype/keytype since there is only enctypes
switch to KRB5_ENCTYPE
Roland C. Dowdeswell (1):
Fix a couple of bugs in krb5_c_valid_enctype():
chas williams - CONTRACTOR (1):
hcrypto: var name current conflicts with linux kernel
Change-Id: Id9a7a9ee8258b979b54f4ed2b4175815ab35ea21
Reviewed-on: http://gerrit.openafs.org/5399 Reviewed-by: Simon Wilkinson <sxw@inf.ed.ac.uk> Tested-by: Simon Wilkinson <sxw@inf.ed.ac.uk> Reviewed-by: Jeffrey Altman <jaltman@openafs.org>
Todd Lewis [Sun, 11 Sep 2011 11:42:47 +0000 (12:42 +0100)]
RPM: Fix dkms support on Fedora 15
Newer dkms no longer uses or supplies a $kernelver_array variable;
instead it uses $kernelver. The attached patch uses both, one of
which will be empty, so the test will do the Right Thing regardless
of your dkms version.
Further, the "mv" command at the end of the MAKE[0]= line needs
lots of back-slashes on each of its parms. We need three to make it
all the way to the final dkms.conf file -- so that's six -- plus one
more to escape the '$'; that's seven in all.
In case there's any question (and with all the back-slashes involved,
there should be) about the intent here, the whole point of this
patch is to make the final dkms.conf MAKE[0]= line look like this
(module line breaks:
Simon Wilkinson [Sun, 11 Sep 2011 11:31:55 +0000 (12:31 +0100)]
Unix CM: Fix afs_event fallout on Mac OS X
Commit dc077b83c9b1f107efbc3483743f44117748f23c moved the afs_event
structure definition into afs_osi.h. However, afs_osi.h is included from
lots of places which don't include a complete set of platform headers.
This means that the use of lck_mtx_t in this file breaks the build on
Darwin.
Instead, define the structure in afs.h, along with all of the rest of
the cache manager structures. Also take the opportunity to move the
definition of shutdown_osisleep it afs_prototypes.h, with the rest of
the prototypes.
Simon Wilkinson [Sun, 11 Sep 2011 11:35:18 +0000 (12:35 +0100)]
Darwin: Actually stop the kernel build
At the moment, all of the kernel build lines end with "; true", which
means that the build will always continue, regardless of whether
creating an individual object file succeeds or not. The 'true' is there
to gobble up the name of the source file which the common build
infrastructure adds to the end of the command line.
Instead of using '; true', use '&& true', so that if one of the C
compiler commands, or the lipo, fails, we get told about it, before
we try to link the missing objects.
Simon Wilkinson [Sun, 10 Jul 2011 09:24:26 +0000 (10:24 +0100)]
Move string manipulation functions out of util
Some functions in libafsutil depend upon the RX libraries, which means
that pulling in other functions in this library can create a dependency
upon RX. This is less than ideal for low-level libraries such as cmd and
comerr.
So, create a new low-level library (currently named 'opr') which can
contain low-level functions from util, and elsewhere. This library
should have no dependencies other than on system libraries and libroken.
autoconf: use $XCFLAGS from OPENAFS_OSCONF when making tests
Certain platforms (like ppc64_linux26) compiler options might
affect some autoconf tests. In particular, the ones related to
the size of certain primitive types.
Simon Wilkinson [Wed, 7 Sep 2011 17:20:24 +0000 (18:20 +0100)]
IRIX: Pull NFS translator hooks
We've never had working NFS translator for IRIX, and the system call
codes which are required to install the symbols used by the IBM
translator have long been used for other things by OpenAFS.
Simplify this mess by just removing the translator stubs from the kernel
module, and all of the code in afsd which used to handle pulling
addresses out of the kernel module so that we can hook ourselves in.
Simon Wilkinson [Sun, 10 Jul 2011 12:57:36 +0000 (13:57 +0100)]
volser: Don't initialise all error tables
Initialising the KA and cmd error tables in vsprocs.c makes the
libvolser library dependent upon rxkad and cmd. As we may want to
use that library in programs which use neither rxkad, nor our command
processor, this is less than ideal. So, just drop the initialisations.
Simon Wilkinson [Tue, 6 Sep 2011 13:18:12 +0000 (14:18 +0100)]
afsd: Fix res_init warning
afsd requires res_init to be declared - get a definition for this by
including roken.h (which has the necessary magic to figure out which
set of headers are required to get resolver functions)
Ben Kaduk [Tue, 26 Jul 2011 03:39:27 +0000 (23:39 -0400)]
Free memory from afs_events
DARWIN and LINUX were already doing this, but everybody else had
a memory leak. Consolidate most of the common code to do so,
including afs_event_t definitions.
Simon Wilkinson [Tue, 5 Jul 2011 08:20:15 +0000 (09:20 +0100)]
rx/rxkad: Move rxkad initialisation into rxkad
When the RX pthread conversion was done, the initialisation of rxkad
mutexes was incorporated into the rx library itself. This is a layering
violation (it breaks the relatively clean security object abstraction),
and means that you can't actually use RX without the rxkad library.
So, remove all of this initialisation from RX. As security libraries
don't have an explicit initialisation function, we setup our various
mutexes using a pthread_once function called from the NewSecurityObject
functions.
This has the added bonus that it removes the final error in rx.c for
pthread builds, and so another bit of warning suppression can be
removed.
Simon Wilkinson [Tue, 5 Jul 2011 08:27:05 +0000 (09:27 +0100)]
volser: Don't declare cstruct twice
Both vsprocs.c and vsutils.c declare the global variable 'cstruct'
(which is initialised by vos.c in order to pass a ubik client structure
through to many of the libvolser functions). This double declaration
prevents libvolser from being linked as a shared library on some
platforms. We only need one of these, so just make vsprocs.c declare it,
and vsutils.c refer to it as an extern.
Of course, using a global variable to pass state around is really quite
nasty, but let's fix that in another change.
Rod Widdowson [Tue, 6 Sep 2011 14:23:33 +0000 (15:23 +0100)]
Windows: Replace "%T" as format string to strftime
Checkin I18ed36cc4dce9aa354ad1398710ab7db83c814a2 made strftime
much more widespread. However the %T format is not available on
all platforms and causes (at least) Windows servers to fail on
first attempt to log. %T is just shorthand to %H:%M:%S so this
checkin just replaces the one with the other.
The redirector maintains file object state after the afs
service has shutdown in case it is restarted. It is critical that
Freelance FIDs not be reused. Add cm_data.fakeDirVersion into
the mix when generating unique values.
Instead of dropping the lock for read and reacquiring for write
use lock_ConvertRToW() which will make the change atomicly if
it is possible or place the thread into the wait list if not.
Windows: tailor smb_MapNTError output for redirector
Separate mappings apply for the afs redirector. Add a boolean
parameter to the function signature that is set true when the
call requires the redirector mapping.
The buffer free list least recently used queue has both
head and tail points. Use the proper versions of the queue
mgmt functions and do not handle edge cases as special cases.
Russ Allbery [Tue, 23 Aug 2011 19:50:55 +0000 (12:50 -0700)]
Generate stub header files for h/*.h files included in libuafs
Previously, the libuafs build created a symlink from h to
/usr/include/sys so that files included under h/* by kernel source
files could be found in the normal system header location. However,
this assumption about the system header location is no longer valid.
Debian and Ubuntu systems with multiarch have arch-specific include
paths so that the same host can be used to build 32-bit and 64-bit
binaries with different system headers, and those include paths are
automatically searched by the compiler. This means some standard
headers are no longer found directly in /usr/include/sys but are
instead found in /usr/include/<arch>/sys.
Using a stripped-down version of similar code for building the kernel
module on Linux, create an h directory containing stub header files
that just include the relevant system <sys/*.h> header file instead.
This allows the compiler to implement its normal internal header
search algorithm.
Also remove all the other symlinks, such as sys, netinet, etc., that
just pointed to the same directories under /usr/include. We can assume
the normal compiler search algorithm will find these headers without
requiring this assistance.
Andrew Deason [Mon, 29 Aug 2011 18:07:01 +0000 (13:07 -0500)]
ihandle: OPEN fdPs are not counted in ihP refcount
Just add a comment explaining that an OPEN FdHandle_t does not count
against the ref count for its parent IHandle_t. Recently I've seen
some confusion about this when discussing ihandle internals, and this
should make this abundantly clear.
Simon Wilkinson [Mon, 29 Aug 2011 22:36:41 +0000 (23:36 +0100)]
libafs: Fix directory verification
With the earlier directory verification changes, every directory
was seen as corrupt, because the nde pointer was never being initialised
Rework the way that we check for the failure of afs_dir_GetVerifiedBlob
so that we can more robustly detect problems, whilst still allowing
normal directories to be browsed as before
Marc Dionne [Sat, 3 Sep 2011 16:32:33 +0000 (12:32 -0400)]
Linux: make sure backing_dev_info is zeroed
The afs backing_dev_info structure is allocated dynamically with
kmalloc, which doesn't zero out the contents. In particular
there's no guarantee that congested_fn is NULL, causing spurious
oopses when bdi_congested tries to call it.
Jeffrey Altman [Mon, 28 Feb 2011 01:27:33 +0000 (01:27 +0000)]
Windows: add cm_BPlusDirEnumBulkStatOne
cm_BPlusDirEnumBulkStatOne() is similar to cm_BPlusDirEnumBulkStat()
except that it is used to obtain the status info for one FID in
particular via RXAFS_InlineBulkStat, the parent directory FID, and
up to 48 other FIDs in the same directory which do not currently
have a registered callback.
The parent directory is included to prevent the directory FID
callback from expiring when a directory such as /afs/andrew.cmu.edu/usr/
that requires more status objects then exist in the cache are continuously
recycled.
Up to 48 other FIDs are requested since in most cases on Windows
every entry in a directory is required for an enumeration.
Jeffrey Altman [Fri, 28 Jan 2011 04:09:00 +0000 (23:09 -0500)]
Windows: remove warnings afskfw.c
Sometimes warnings are errors and sometimes they are just warnings.
In this case, the krb5 error message functions were being passed
a krb5_context with the wrong level of indirection which resulted
in an actual error. Other warnings were due to improper typing
and unnecessary casts. Fix them all.
Change the non-persistent cache mode to use a heap allocated
cache in place of a paging file allocated cache. With a heap
cache the memory for the cache can be locked into physical memory
so it won't be swapped out when running in virtual machines.
This patch does not apply such memory locking.
Jeffrey Altman [Fri, 6 Aug 2010 22:24:47 +0000 (18:24 -0400)]
Windows: pioctl path retrying with \\afs\all
When the redirector is in use, \\AFS is identified as being an AFS path.
With the SMB redirector, \\AFS was not considered a valid path. In order
to simulate the notion of \\AFS being the root volume, failover logic
was added to the symlink and fs modules being triggered that
would retry a request for \\afs\foo as \\afs\all\foo.
The VIOC_SYMLINK and VIOC_AFS_CREATE_MT_PT pioctls are broken. The
'path' parameter that is passed in contains the directory entry that
is to be created. Since the AFS redirector maps \\afs to the root.afs
volume the "foo" in \\afs\foo must exist in order for the
_._AFS_IOCTL_._ special file to be opened.
This patch implements an alternative to the failover. If the opcode
is one of the two listed above and the input path is a UNC path,
then the path is re-written as \\AFS\all. This is necessary because
the repeated attempts to open a UNC path through the SMB redirector
with different usernames can result in the smb session becoming
disconnected. If this is done when the SYSTEM account is in use,
the tokens associated with the connection can be lost.
The windows cache manager tracks volumes by volume group.
Up to this point all volume location updates have been performed
by the volume name. What if the volume name was altered? In this
case the volume location information for the in use volume ids will
fail until a mount point to the new name is queried. Before
marking the volume group as non-existent attempt to perform a
lookup using either the volume id for the readwrite or readonly
volume.
Install*Volume is careful to protect against recursing into the volume
lock via ResetVolumeInfo. Unfortunately, GetServer acquires xserver,
and then if it needs to call GetCapabilities, it drops and reacquires
xserver.
turns out the volume locks weren't protecting much. they also aren't
grabbed before xvolume is dropped. fine, so, restructure to do all the
work, then merge the result.
Derrick Brashear [Tue, 30 Aug 2011 05:18:37 +0000 (01:18 -0400)]
redhat: mockbuild updates for repoquery
for starters, mock should be careful not to use parent yum cache
when running repoquery, the host runs it, we want the guests's
yum cache. be careful to not attempt to use the parent's yum cache
(in the default config).
additionally, we need to not try to build 64 bit kernel modules in
32 bit chroots. expand the list of rpms while ejecting any which are
64 bit if we're 32.
additionally, blacklist kernels which are the wrong osver.
Michael Meffie [Fri, 12 Aug 2011 18:29:48 +0000 (14:29 -0400)]
xstat: cm xstat time values are 32 bit
The kernel space cm xstat time structures are implemented as 32
bit values in memory and on the wire. Define the client side
xstat userspace structures as 32 bit time values as well to avoid
size mismatches on systems with native 64 bit time values.
Jeffrey Altman [Sun, 28 Aug 2011 16:03:53 +0000 (12:03 -0400)]
Windows: afslogon network provider debug registry value
create a new TransarcAFSDaemon\NetworkProvider "Debug" value
to be used for activating the network provider debugging.
The overlapping use of TransarcAFSDaemon\Parameters "TraceOption"
is just too confusing.
Jeffrey Altman [Fri, 26 Aug 2011 17:57:15 +0000 (13:57 -0400)]
Windows: afslogon.dll is not a file system interface
Do not return a file system network type that corresponds
to a real file system inter since afslogon is in fact not
associated with a file system interface. We can't return
WNNC_NET_NONE (0) because that prevents NPLogonNotify()
from being executed. However, if we return an in use
file system value that can confuse the system when the
actual file system's network provider is also installed.
Jeffrey Altman [Fri, 26 Aug 2011 13:36:04 +0000 (09:36 -0400)]
Windows: torture error reporting
When LeaveThread() is called and GetLastError() has already
been called, pass the last error value to LeaveThread(). Otherwise,
the GetLastError() call in LeaveThread() may return an inaccurrate
result.
Garrett Wollman [Tue, 9 Aug 2011 03:59:17 +0000 (23:59 -0400)]
libafs: crash in a more useful way if nchunks is zero
In afs_CacheStoreDCaches(), if the parameter nchunks is zero, the
main loop will not execute, leaving the XSTATS pointer unchanged,
which will result in a null dereference in XSTATS_END_TIME. Instead
assert that nchunks is nonzero, which will help the static analyzer
and will also generate a more useful panic message should this
error ever be encountered in operation.
There is presently only one call site, and it may be the case that this
condition can never be triggered.
(While in the neighborhood, also avoid dereferencing tdc immediately
before testing whether it is null.)
Garrett Wollman [Thu, 11 Aug 2011 01:52:44 +0000 (21:52 -0400)]
libafs: FillStoreStats doesn't need to be global; avoid pass-by-pointer
FillStoreStats is only used in afs_fetchstore.c, so make it static.
Parameter xferStartTime is an osi_timeval, which should be small
enough to pass by value, so do so.
Andrew Deason [Wed, 24 Aug 2011 17:48:19 +0000 (12:48 -0500)]
ihandle: Fix IH_REALLYCLOSE for positional I/O
Currently, ih_fdclose (which is called by IH_REALLYCLOSE), goes
through every FD_HANDLE_OPEN FdHandle_t and closes it. If it finds
handles that are FD_HANDLE_INUSE, it skips those and sets a flag on
the parent IHandle_t. For non-positional I/O, any future opens cannot
use these _INUSE handles, since _INUSE handles cannot be reused, and
the handle will be actually closed when it is FDH_CLOSE'd.
For positional I/O, the situation is different. Multiple threads can
use the same _INUSE FdHandle_t, and so there is nothing currently
stopping a thread from IH_OPEN'ing an ihandle that has been
IH_REALLYCLOSE'd, and getting back an FdHandle_t that existed before
the IH_REALLYCLOSE was issued. This is important, since IH_REALLYCLOSE
is used on files that are deleted, and future IH_OPENs for the same
inode must not use the cached file descriptor. Getting this wrong can
cause data loss, since it can cause us to read from or write to a file
descriptor referring to a deleted file, when we instead should open a
new copy of that file.
To fix this, we create a new FdHandle_t state called
FD_HANDLE_CLOSING, which is set in IH_REALLYCLOSE if we encounter an
FD_HANDLE_INUSE FdHandle_t. In IH_OPEN, we always skip
FD_HANDLE_CLOSING handles, so we can never get back a cached file
descriptor from before an IH_REALLYCLOSE call.
Andrew Deason [Wed, 24 Aug 2011 17:30:00 +0000 (12:30 -0500)]
ihandle: Actually assert active fdPs are not AVAIL
FdHandle_t's that are on the linked list for an associated IHandle_t
should not be in the state FD_HANDLE_AVAIL. For the non-PIO case, we
assert that this is the case in ih_open (since we assert that if the
FdHandle_t is not in INUSE state, then it must be in OPEN state).
However, for the PIO case, we were just skipping over any FdHandle_t's
that were in the AVAIL state. These should never exist while on that
linked list, so assert for the PIO case, as well.
In the absence of bugs, there is no functional change here, but it
perhaps makes the ih_open loop easier to understand.
Matt Benjamin [Wed, 24 Aug 2011 20:23:37 +0000 (16:23 -0400)]
LINUX vcache lock ordering in afs_linux_readdir
Normalize shared and exclusive lock operations. Take the lock
exclusive immediately, since the code assumes a write lock if
the vcache state is in flux or the entry is being fetched, releasing
-write- rather than shared, since we do not hold a shared lock.
Jeffrey Altman [Tue, 23 Aug 2011 20:02:28 +0000 (16:02 -0400)]
Windows: change buf_Find*() signature to accept cm_fid_t
The buf_Find*() functions require a cm_fid_t to match with the
cm_buf_t objects not a cm_scache_t. Change the signature so
that the cm_scache_t is not required. It should be possible to
search for a buffer even if the cm_scache_t is not present in
the cache.
projects / packages / o / openafs.git / log