Andrew Deason [Tue, 23 Oct 2012 20:47:06 +0000 (15:47 -0500)]
ptserver: Avoid inet_ntoa
The ptserver uses inet_ntoa in a few places, such as for calculating
host CPS. This isn't safe in pthreaded environments, so use
afs_inet_ntoa_r instead.
Simon Wilkinson [Sun, 21 Oct 2012 19:07:44 +0000 (20:07 +0100)]
Fix mutex assertion
RX mutexes have two mechanisms for asserting ownership of a mutex:
MUTEX_ISMINE, which returns true if the caller is the owner of the
mutex in question, and osirx_AssertMutex which fires an assertion if
the calling thread doesn't own a specified mutex.
Neither of these work with pthread mutexes on all platforms, and the
kernel support for MUTEX_ISMINE is dubious in places. Because in some
implementations, MUTEX_ISMINE tries to lock the mutex in question, a
failing call to MUTEX_ISMINE can lead to the process holding an
additional, unexpected, lock.
This patch reworks all of this, and uses the new opr mutex framework
so that we can do mutex assertions in userspace, too. We remove the
unsafe MUTEX_ISMINE macro, and replace it with MUTEX_ASSERT which
simply asserts if the specified mutex is not held by the current
thread. osirx_AssertMutex is removed as it is now redundant.
MUTEX_ASSERT will always work in kernel code.
For userspace, we provide opr_mutex_assert, which is implemented using
POSIX error checking mutexes. As error checking mutexes have a runtime
overhead, this is only available when configured with
--enable-debug-locks, the rest of the time calls to opr_mutex_assert are
no-ops.
Simon Wilkinson [Sun, 21 Oct 2012 20:19:40 +0000 (21:19 +0100)]
rx: Move kernel assertion macros
Move the kernel assertion macros out of rx_prototypes.h and into
rx_kernel.h. This solves an ordering problem if these macros are to
be used from src/rx/<arch>/*.h
Simon Wilkinson [Sat, 20 Oct 2012 22:14:41 +0000 (23:14 +0100)]
Add opr/lock.h and tidy locking macros
The MUTEX_* and CV_* macros leaked from RX a while ago - they mean
that most of the pthreaded tree has a dependency on RX, as well as
further confusing the difference between userspace and kernel.
Tidy all of this up so that we have opr_mutex_* and opr_cv_* macros
to handle portable locking, and use these throughout the userspace
tree. Only kernel code should now use MUTEX_* and CV_*.
Provide opr/lockstub.h as a header that can be used by non-pthreaded
code to easily stub out these functions.
Simon Wilkinson [Fri, 26 Oct 2012 14:37:52 +0000 (15:37 +0100)]
rx: Move transmit queue clearing
When the client receives a data packet from the server, it means that
the server has completed processing the client's request. This, in turn,
implies that the transmit queue can be cleared. However, we were doing
this with every incoming data packet.
Move the transmit queue clearing to the code which handles the rest of
the data packet, and make the function only run if the transmit queue
is non-empty.
Now that there's no client specific logic in the ReceiveCall section,
clean up this code to reduce duplication.
Simon Wilkinson [Fri, 26 Oct 2012 14:23:48 +0000 (15:23 +0100)]
rx: Refactor code to acknowledge a whole TX queue
We acknowledge a whole transmit queue whenever an ACKALL packet is
received, or whenever the call changes direction. As the same logic
is used in both locations, pull it out into a common helper function.
Simon Wilkinson [Fri, 26 Oct 2012 13:55:02 +0000 (14:55 +0100)]
rx: Remove duplicate out of order ACK check
Once we've moved the congestion window, there's no going back. So
any ACK packets that attempt to move the window backwards by including
a 'firstPacket' value earlier than the current window position must
be ignored.
However, we check (and ignore) these packets twice. Once in
rxi_ReceivePacket, which only checks in the client side case, and again
in rxi_ReceiveAckPacket, which has a more complete check that runs for
both client and server connections.
Remove the identical check from rxi_ReceivePacket in a continuing effort
to clean up this bit of code.
Simon Wilkinson [Fri, 26 Oct 2012 13:52:46 +0000 (14:52 +0100)]
rx: Remove duplicate security layer check
rxi_FindConnection checks that the connection it returns has a
security layer matching that of the incoming packet. Don't duplicate
this check within the rxi_ReceivePacket code.
Simon Wilkinson [Fri, 26 Oct 2012 11:21:41 +0000 (12:21 +0100)]
rx: Don't build a call to immediately abort it
If the server is over the busy threshold, then don't create a new
call structure just to be able to send an abort on that call. Instead,
use rx_SendRawAbort to send an abort packet on the appropriate channel.
Simon Wilkinson [Thu, 25 Oct 2012 12:34:33 +0000 (13:34 +0100)]
rx: Remove unreachable debug statement
ReceivePacket has a dpf which is conditional on the packet having a
zero callnumber. However, just before we reach this debug statement,
we always return if the header doesn't have a call number included.
So, the debug statement can never run. Just remove it, as it's
potentially confusing.
Jeffrey Altman [Mon, 29 Oct 2012 16:59:14 +0000 (12:59 -0400)]
Windows: Set Server Prefs recalc immediately
When processing the set server preferences pioctl call cm_RankServer()
to update the server preference value reported by "getserverprefs"
in addition to cm_ChangeRankVolume() or cm_ChangeRankCellVLServer().
Jeffrey Altman [Mon, 29 Oct 2012 14:33:18 +0000 (10:33 -0400)]
Windows: mark server reference offline for VOFFLINE
cm_Analyze() was not marking the cm_ServerRef_t reference to
a volume instance as srv_offline in response to a VOFFLINE error.
As a result the same volume instance is tried again and again.
Returning STATUS_MEDIA_WRITE_PROTECTED in preference to
STATUS_OBJECT_NAME_COLLISION when the file results in silent
failures by some applications (ie, Firefox.exe) when the
first directory in the path below the share name is the
root of a .readonly volume.
Simon Wilkinson [Thu, 25 Oct 2012 10:49:55 +0000 (11:49 +0100)]
rx: Get rid of AFS_GLOBAL_RXLOCK_KERNEL
Get rid of the AFS_GLOBAL_RXLOCK_KERNEL #define. RX used to have a
single global lock locking mode, but none of our kernel modules use
it any more. In fact, AFS_GLOBAL_RXLOCK_KERNEL is now only defined
when RX_ENABLE_LOCKS is also defined. Simplify the code by renaming
all of the occurrences of AFS_GLOBAL_RXLOCK_KERNEL as RX_ENABLE_LOCKS,
and remove any cases where we're now doing unecessary tests
Simon Wilkinson [Thu, 25 Oct 2012 10:32:03 +0000 (11:32 +0100)]
rx: Don't have 2 different protos for rxi_CheckCall
Use a single prototype for rxi_CheckCall in both the pthread and
lwp cases. Remove the #ifdef maze at the call sites, and take advantage
of the fact that MUTEX_EXIT reduces to an empty string in the lwp
case.
Simon Wilkinson [Thu, 25 Oct 2012 10:27:33 +0000 (11:27 +0100)]
rx: Don't double check conn->call
We currently have
call = conn->call[channel]
if (call) {
...
} else {
call = conn->call[channel]
if (call) {
...
}
}
As we don't drop (or acquire) any locks between the first and the
second check of call, there's no way that the result can be different
from the first time we checked. So just get rid of the uneccessary
code, and reindent the following block to match.
Simon Wilkinson [Tue, 23 Oct 2012 18:21:09 +0000 (19:21 +0100)]
rx: Move bytesSent + bytesRcvd into app only data
The call->bytesSent and call->bytesRcvd counters are only manipulated
by the application thread in running calls. Move them into the app-only
section of the call structure so this is clear.
Simon Wilkinson [Tue, 23 Oct 2012 12:35:43 +0000 (13:35 +0100)]
rx: Don't use app-thread variable in SendXmitList
The value of call->app.mode is changed by the application thread
without taking the call lock. Instead of using this variable in
SendXmitList to determine whether the queue should be flushed, add
a new flag (RX_CALL_FLUSH) to control flushing behaviour.
As call->flags is manipulated under the call lock, its value can
be safely used by SendXmitList.
Simon Wilkinson [Tue, 23 Oct 2012 11:41:07 +0000 (12:41 +0100)]
rx: Make lock-free call data explicit
For speed, the application thread accesses a number of elements of
the call structure without holding the call lock. This is safe, as
long as the application thread is the only place in which these
items of data are accessed.
Make this distinction explicit by creating a new structure to hold
all of these fields, and include this structure within the rx_call.
This turns up one place in the code (SendXmitList) which accesses an
application private piece of data in the listener and event threads.
A forthcoming patch will fix this.
Jeffrey Altman [Mon, 6 Aug 2012 16:19:26 +0000 (12:19 -0400)]
Windows: Send all \\AFS\PIPE to afsd_service
Anytime there is a pipe service request, forward it to the
afsd_service.exe and permit the service to manage the request.
The prior implementation resulted in STATUS_FILE_NOT_FOUND errors
being delivered when an unexpected service was requested.
Jeffrey Altman [Fri, 19 Oct 2012 15:26:21 +0000 (11:26 -0400)]
Windows: ObjectInfo RefCount 0 <-> 1 transitions
When the reference count transitions from 0 <-> 1 ensure that the
ObjectInfoLock is held exclusive to prevent the current thread from
altering the state while another thread is holding the ObjectInfoLock
shared in order to conditionally perform an action based upon
the the reference count being zero.
Patchset eaad522651a81f20eac4966a55a731e0e59e39dd inadvertently
introduced a deadlock with invalidation requests from the service.
It is not safe to hold the ObjectInfoLock resource across calls
to AFSCleanupFcb(). Instead of holding the lock obtain a reference
to the ObjectInformationCB.
Jeffrey Altman [Thu, 25 Oct 2012 18:33:29 +0000 (14:33 -0400)]
Windows: AFSCleanup re-organization
Reorganize the activities of the AFSCleanup() for File FCBs
so that the Fcb Resource can be dropped prior to issuing the
cleanup request to the cache manager. The cache manager can
block for a long period of time while flushing data and holding
the Fcb resource blocks all subsequent CreateFile requests.
Jeffrey Altman [Thu, 25 Oct 2012 18:31:14 +0000 (14:31 -0400)]
Windows: AFSCleanup Flush Data decision
AFSCleanup() should instruct the cache manager to flush dirty
data when the Context Control Block indicates that the handle
being closed was opened for writing and was granted appropriate
permissions. The decision to flush should not be dependent on
the open handle count because the last handle might belong to
an authentication group that does not have write permission.
Jeffrey Altman [Tue, 23 Oct 2012 00:40:21 +0000 (20:40 -0400)]
Windows: AFSMarkDirty() require ExtentsResource held
Instead of dynamically testing if the ExtentsResource is held
and if not acquire it within AFSMarkDirty(), simply require that
it be held. AFSMarkDirty() is only called from one location.
Jeffrey Altman [Tue, 23 Oct 2012 00:34:59 +0000 (20:34 -0400)]
Windows: AFSFlushExtents QueuedFlushCount leak
The FCB QueuedFlushCount was decremented in all code paths
but only incremented if the AuthGroup acquisition succeeded.
Increment the counter before the AuthGroup checks.
Jeffrey Altman [Tue, 23 Oct 2012 00:29:47 +0000 (20:29 -0400)]
Windows: RDRFunction remove DebugBreak
DebugBreak hard coded into the source tree makes debugging
other unrelated issues difficult if the code path being executed
includes them. Remove them.
Jeffrey Altman [Fri, 19 Oct 2012 13:33:18 +0000 (09:33 -0400)]
Windows: Promote DELETED from DirEntry to ObjInfo
On deletion of the DirEntry in AFSDeleteDirEntry() set the
AFS_OBJECT_FLAGS_DELETED flag on the ObjectInformation object
if and only if the AFS_DIR_ENTRY_DELETED flag was set in the
DirEntry. Setting the AFS_OBJECT_FLAGS_DELETED should not
be conditional on the ObjectInformatION ReferenceCount being
zero.
Remove the test and set of AFS_OBJECT_FLAGS_DELETED from
AFSClose() because that operation will already have been
performed in the call to AFSDeleteDirEntry() if necessary.
Jeffrey Altman [Tue, 16 Oct 2012 13:36:56 +0000 (09:36 -0400)]
Windows: Remove 'bAllocatedFcb' from AFSCreate.cpp
All functions now call AFSInitFcb() and the ObjectInfo->Fcb == NULL
test is performed internally. Therefore, it is not possible for
the caller to track whether or not an Fcb was allocated. It is
irrelevant. The Fcb will be cleaned up when the ObjectInfo is
destroyed by the PrimaryVolumeWorker thread.
Jeffrey Altman [Mon, 8 Oct 2012 03:41:32 +0000 (23:41 -0400)]
Windows: PrimaryvolumeWorker do not pause if busy fcb
The AFSPrimaryVolumeWorkerThread should not unnecessarily block
on the FCB Resource because such blockage could be the result of
of waiiting for extents to be delivered from the service. The
AFSPrimaryVolumeWorkerThread is the primary method by which
extents are released back to the service.
AFSCleanupFcb() is modified to return STATUS_RETRY if the Fcb
resource cannot be obtained without blocking.
The AFSPrimaryVolumeWorkerThread() does not call
AFSCleanupFcb() with 'ForceFlush' parameter set to TRUE and
remembers if STATUS_RETRY is returned. If any Fcb was busy,
then the worker does not wait for the 5 second timer to fire.
The FCB ExtentsRequestComplete KEVENT setting, clearing
and testing was racy. Clear the event before issuing the
request to the service and if the request fails, set it in
case two threads issued requests for the same FCB in parallel
and one fails and the other succeeds.
We must ensure that a clear does not mask the event being set
prior to the request thread returning.
Jeffrey Altman [Thu, 18 Oct 2012 13:56:12 +0000 (09:56 -0400)]
Windows: clear pending delete upon deletion
During cleanup processing if the DELETE_PENDING flag is set
the service will be told to delete the file when the handle
count reaches 1. At that point the file will be deleted
and the DELETED flag will be set on the object info object.
The DELETE_PENDING flag was not being cleared which could
lead to confusion. This patchset clears the flag after deletion.
Jeffrey Altman [Sun, 7 Oct 2012 14:23:19 +0000 (10:23 -0400)]
Windows: AFSInitFcb Check ObjectInfo->Fcb for NULL
Now that AFSInitFcb is called under the ObjectInfoLock, it is
once again safe to perform a test for ObjectInfo->Fcb != NULL
and return immediately if an Fcb is already assigned.
Jeffrey Altman [Fri, 5 Oct 2012 15:36:45 +0000 (11:36 -0400)]
Windows: Protect ObjectRefCnts with ObjectInfoLock
The ObjectInfoCB.ObjectReferenceCount is tested to determined
when it is safe to remove an FCB from the ObjectInfoCB. The
value must not be permitted to change while a removal is performed.
Protect AFSRemoveFcb() calls with exclusive holds of the
ObjectInfoCB.NonPagedInfo->ObjectInfoLock. New functions:
AFSObjectInfoIncrement()
AFSObjectInfoDecrement()
perform all increments and decrements while holding the
ObjectInfoLock in a Shared state.
Jeffrey Altman [Wed, 3 Oct 2012 01:07:21 +0000 (21:07 -0400)]
Windows: Always AFSInitFcb and AFSRemoveFcb
Instead of comparing ObjectInfo->Fcb to NULL and conditionally
calling AFSInitFcb() or AFSRemoveFcb(), always call them and use
InterlockedExchangePointer() as the test.
Jeffrey Altman [Sat, 6 Oct 2012 21:36:25 +0000 (17:36 -0400)]
Windows: AFSRequestExtentsAsync and AFSDoExtentsMapRegion
When calling AFSDoExtentsMapRegion() the FCB ExtentsResource
must be held. AFSRequestExtentsAsync() failed to hold the
ExtentsResource across the call.
The LazyWriterThread should not be recorded in the FCB. It is
possible for multiple lazy writes to occur on a file in parallel
in separate threads. The value is not used for anything in any
case. AFSCommonWrite() tests the LazyWriterThread value but only
if 'bMapped' is FALSE. Since 'bMapped' is always TRUE, the
comparison is never performed. Remove the test and the value.
Jeffrey Altman [Sun, 14 Oct 2012 19:46:06 +0000 (15:46 -0400)]
Windows: Add cm_SyncOp to cm_ReadMountPoint()
Add a cm_SyncOp(CM_SCACHESYNC_FETCHDATA) call to cm_ReadMountPoint()
to prevent multiple FetchData RPCs being issued for the same
mount point at the same time.
In AFSInitFcb() assign pFcb->ObjectInformation before the
InterlockedExchangePointer call and not afterwards. Assigning
it afterwards leaves a small race where the ObjectInformation
value will be invalid.
Jeffrey Altman [Wed, 17 Oct 2012 00:26:43 +0000 (20:26 -0400)]
Windows: OpenTargetDirectory AFSInitFcb Reparse Test
In AFSOpenTargetDirectory the test to determine if AFSInitFcb
allocated a FCB or returned an existing one (STATUS_REPARSE)
was reversed. If AFSInitFcb was called and AFSOpenTargetDirectory
eventually failed, an in use FCB would be freed.
Jeffrey Altman [Sat, 6 Oct 2012 05:40:47 +0000 (01:40 -0400)]
Windows: Do not call buf_ClearRDRFlag unlink/rmdir
When processing unlink and remdir operations initiated by the
SMB stack do not call buf_ClearRDRFlag. The redirector upon
receiving the AFS_INVALIDATE_DELETE call will cancel outstanding
extent operations, mark the FCB deleted, and tear down any held
extents.
Simon Wilkinson [Fri, 12 Oct 2012 09:07:22 +0000 (10:07 +0100)]
rx: Use opr queues
Modify RX so that it uses opr queues throughout, rather than the older,
non-type-safe rx_queue structure and macros. Attempt to clarify which
items in a structure are queue headers, and which are linkage pointers.
This has the knock on effect that including an RX header doesn't
automatically give you rx_queue.h in your application's namespace.
Simon Wilkinson [Thu, 11 Oct 2012 11:34:46 +0000 (12:34 +0100)]
rx: Move server queue entry structure out of rx.h
Hide the server queue management structure in its own header file,
rather than exposing it globally in rx.h. This structure has always
been private - applications have no business knowing about it!
Marc Dionne [Fri, 12 Oct 2012 20:31:24 +0000 (16:31 -0400)]
libafs: Fix second pass in ShakeLooseVCaches
Commit 3105c7ff introduced a two phase process for reclaiming
vcache entries. First go through the list and do what's possible
without sleeping (skipping aliased dentries on Linux), then do
a second pass only if necessary, allowing sleeping.
Unfortunately the test for the end of the VLRU scan is incorrect
and can never trigger, so this second pass was effectively disabled
and any code that is conditional on defersleep=1 was never
exercised. The code to start the second scan also has issues.
Fix the end of VLRU test, and also correctly set the variables
needed to restart the scan.
Andrew Deason [Tue, 11 Sep 2012 18:59:21 +0000 (13:59 -0500)]
LINUX: Ignore 'offender' in error queue processing
The 'offender' is who generated the error, possibly who sent us an
icmp packet (the given 'port' will be 0). What we want is the peer
that is actually unavailable, which is already in the 'addr' variable
we received from the recvmsg itself.
Andrew Deason [Tue, 11 Sep 2012 17:48:14 +0000 (12:48 -0500)]
rx: Check for peer deadness in rxi_Resend
If we need to resend something, the peer we're sending to may be dead.
Check if the peer is dead in rxi_Resend, so we don't have to wait
(possibly several seconds) for the next rxi_CheckCall.
Andrew Deason [Fri, 7 Sep 2012 22:04:18 +0000 (17:04 -0500)]
LINUX: Allocate error queue buffer once
We call osi_HandleSocketError in a loop, so make sure we process all
of the errors. We were allocating a buffer to process the errors in
osi_HandleSocketError itself, but we can reuse the same buffer on
subsequent invocations, to reduce allocation/free pressure if we need
to call osi_HandleSocketError more than once. So, do that.
Andrew Deason [Fri, 7 Sep 2012 21:58:05 +0000 (16:58 -0500)]
rx: Process error queue after noticing errors
If errors exist in the socket error queue, we will notice by a sendmsg
or recvmsg returning an error. If we never get an error, we don't need
to check the error queue. So, only call osi_HandleSocketError after
such an error has been returned, so we can avoid unnecessarily
checking the error queue when there are no errors.
The fuse tests are fundamentally broken as they stand:
*) They rely on files that have not been committed to the tree. To
function correctly the file fuse/conf/CellServDB must be present
*) They always run, regardless of whether the fuse helper binaries are
installed on the developers system, or even on whether the tree was
built with fuse support enabled.
*) They pass, even if fuse fails to start up
*) The file fuse.sh is committed, despite being unused. This is
particularly confusing, as it looks like this is where the tests
are performed from (its not, testing is done in dynroot-t)
*) fuse-log should be either cleaned up, or flagged as ignored in
.gitignore
Revert the commit until such time as all these issues can be fixed
Simon Wilkinson [Wed, 10 Oct 2012 13:45:03 +0000 (14:45 +0100)]
tests: Reformat loopback tests
Reformat the loopback tests to match our house style - 4 spaces for
first indent, a tab for the second, and so on, opening brace of a
function on a newline, spaces around assignments, and so on.
Andrew Deason [Thu, 4 Oct 2012 20:49:56 +0000 (15:49 -0500)]
DAFS: VRS_r with VOL_SALVAGE_NO_OFFLINE in attach2
One caller of VRequestSalvage_r in attach2 was not passing the
VOL_SALVAGE_NO_OFFLINE flag. This really should be passed for every
place that manually sets vp->nUsers = 0, since then the VPutVolume_r
handlers will never fire.
Andrew Deason [Wed, 3 Oct 2012 19:44:46 +0000 (14:44 -0500)]
sys: Split up syscall.lo 'echo's
Currently we echo a string to syscall.lo to generate it. However,
'echo' is often a shell builtin, and some shells (such as bash) do not
interpret escape codes like \n unless the -e option is given. So, this
results in syscall.lo containing a single commented line, which
results in .libs/libafsrpc_sys.a not getting created, which later on
causes errors.
Instead, just split the syscall.lo generation into separate echo
invocations, to make sure we work everywhere.
in order that this can potentially be extracted entirely
to a platform-specific file, (and possibly dbus-equivalents
inserted also) consolidate the macos system events handling code
Andrew Deason [Tue, 2 Oct 2012 19:38:20 +0000 (14:38 -0500)]
afs: Avoid tracking file locks for RO volumes
Advisory file locks for RO volumes don't make a lot of sense, since
there are no possible writes to worry about. The fileserver already
does not track these, so don't even bother processing them in the
client.
Troy Benjegerdes [Mon, 27 Feb 2012 04:56:06 +0000 (22:56 -0600)]
Add some basic tests to check out fuse
Update makefiles to have 'make test' and 'make check' use the
_nolibafs build version, since there are no tests that (currently)
require the AFS kernel module to be built.
Clean up fuse test copyright notice, Alphabetize configure.ac
Jeffrey Altman [Mon, 1 Oct 2012 16:03:49 +0000 (12:03 -0400)]
Windows: File Info Query Symlinks
For Symlinks, always set the Reparse Point attribute and
set the Directory attribute if the target is a directory.
Do not return the file attributes of the target.
Jeffrey Altman [Mon, 1 Oct 2012 15:04:23 +0000 (11:04 -0400)]
Windows: Dir Enum behavior for Symlinks / MPs
Comparisons of the behavior of cmd.exe, powershell.exe, and tcc.exe
with regards to directory enumeration show that when Symlink file
information is returned that the "reparse point" data should be
reported along with whether or not the target is a directory.
For mount points, the reparse point file information should always
be returned and the type should always be directory.
The target timestamps, file sizes, etc. should never be returned.
projects / packages / o / openafs.git / log