Andrew Deason [Thu, 2 Dec 2010 23:13:17 +0000 (17:13 -0600)]
RX: Avoid retrying calls on busy channels
When we receive an RX_PACKET_TYPE_BUSY packet, we currently ignore it.
This is a problem when the server has a long-running call on that same
call channel that we don't know about, since we will then keep
retrying the call on the same channel and keep getting
RX_PACKET_TYPE_BUSY responses.
Try to avoid this by returning the RX_CALL_TIMEOUT error when we get a
BUSY packet and there are other free call channels available on the
conn. When the application gets the error and retries the call, we
avoid using the same call channel again where possible. When all of
the call channels appear busy, we revert to effectively the old
behavior of retrying the call on the same channel until we get an
RX_CALL_DEAD (or similar) error.
Jeffrey Altman [Thu, 27 Jan 2011 01:10:57 +0000 (20:10 -0500)]
Windows: Correct cm_volume locking
cm_volume_t flags was used for two categories of flags. The first
protected by the cm_volume_t->rw lock. The second protected by
the global cm_volumeLock. Separate the flags field into two
afs_uint16 fields and break the flag space into FLAG and QFLAG.
Add assertions to the volume LRU functions to ensure that they
are always called with cm_volumeLock write-locked.
Correct two locations where cm_AdjustVolumeLRU() was called
read-locked instead of write-locked.
Andrew Deason [Tue, 25 Jan 2011 16:46:38 +0000 (10:46 -0600)]
vol-salvage: Only delete bad vnodes during !check
In SalvageIndex, if check is true, we're only supposed to check for
consistency and not actually change anything (this is enforced by an
assert later on). The check for vnode magic consistency breaks this
assumption, and just always fixes the vnode.
Change this so we just error out if check is set, and the vnode magic
is wrong.
Rod Widdowson [Sun, 23 Jan 2011 14:29:51 +0000 (14:29 +0000)]
Windows: fix parameters and return value from nt_seek
SetFilePointerEx takes specific values
(FILE_BEGIN/FILE_CURRENT/FILE_END) whilse fseek requires SEEK_SET,
SEK_END, SEEK_CUR. It turns out that these overlap, but we should
not let that pass unchallenged.
SetFilePointerEx returns nonzero for success zero for failure. fseek
returns the other way around.
Neither of these changes currently matter, but we should fix them.
Reviewed-on: http://gerrit.openafs.org/3746 Reviewed-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
(cherry picked from commit 4f838b03bbf9ea0e1ede8a188ea6dde3efb4e231)
Rod Widdowson [Sun, 23 Jan 2011 10:59:28 +0000 (10:59 +0000)]
Windows: read and write take void* buffers, open takes a const char*
nt_read and nt_write were defined to take a char* buffer which was
then cast to a void *. Meantime every call of OS_READ and
OS_WRITE were casting to a char*.
Equally every call of OS_OPEN was passing down a const char*,
causing warnings.
This checkin fixes this:
nt_read : char* to void*
nt_write: char* to const void*
nt_open char* to const char*
OpenBSD: Eliminate complaint about built-in malloc.
With OpenBSD 4.8, OpenBSD now uses gcc 4. With its new
defaults, the OpenAFS compile of the kernel module now
complains incessantly about the conflict between the
built-in malloc versus the kernel version (which has
extra parameters). Therefore, set -fno-builtin-malloc
when compiling the kernel module to remove the noise
since the differences can't be reconciled otherwise.
Rod Widdowson [Sun, 23 Jan 2011 12:04:59 +0000 (12:04 +0000)]
Windows: remove faulty assumptions about device names in vol-salvage
The implementation has an assumption that all disk volumes have an
object name of \Device\HarddiskXXX (where XXX is a number). This is
wrong since the name is purely a convention and since about WXP they
have been called \Device\HarddiskVolumeXXX.
Either way it is spurious to assume the format and then try to compare
the XXX. This change just compares the strings. This is done in a
case insenstive manner which is the safer option. It is quite
feasible, but very unlikely that someone will uses 'case sensitively
different' object names.
Jeffrey Altman [Thu, 20 Jan 2011 07:15:18 +0000 (02:15 -0500)]
vol: fix CreateFile params nt_unlink and nt_open
nt_unlink: when opening a file handle to assign delete on close
status the caller must request DELETE permission.
nt_open: make sure that DELETE permission along with FILE_SHARE_DELETE
is requested if we wish to permit another CreateFile call in the future
to assign delete on close.
Reviewed-on: http://gerrit.openafs.org/3711 Reviewed-by: Andrew Deason <adeason@sinenomine.net> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Rod Widdowson <rdw@steadingsoftware.com> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 5fdc2679dc973fa45bf4b3bc7f21a527492868db)
Jeffrey Altman [Thu, 20 Jan 2011 07:06:12 +0000 (02:06 -0500)]
vol: add comment nt_unlink cannot with fopen handles
On Windows, files opened via the CRT fopen or open
POSIX compatibility functions cannot be marked for delete
on close. Add a comment until such time as the use of
fopen and open on Windows can be replaced.
Reviewed-on: http://gerrit.openafs.org/3707 Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Rod Widdowson <rdw@steadingsoftware.com>
(cherry picked from commit eb6f3a89a3f4da6d3de2d4ae34743014e4eb0fb2)
Andrew Deason [Wed, 10 Nov 2010 15:11:47 +0000 (09:11 -0600)]
Use afs_foff_t for file offsets
Some callers in vol and volser were not using afs_foff_t for file
offsets for calls to STREAM_ASEEK, FDH_TRUNC, FDH_PREAD, or
FDH_PWRITE. Most notably, in code for volume cloning, purging, and
dumping operations. Fix them to use afs_foff_t to prevent errors when
dealing with large special files.
Jeffrey Altman [Thu, 20 Jan 2011 07:11:57 +0000 (02:11 -0500)]
vol: nt_open should not create missing directories
Revert the prior change that added directory creation
to nt_open(). nt_open() should fail and the caller should
create the directories. Failure to do so results in an
incomplete tree which confuses the salvager.
Jeffrey Altman [Thu, 20 Jan 2011 06:39:11 +0000 (01:39 -0500)]
vol: make it clearer that SetOGM is not impl on Windows
Separate the SetOGM function into two implementations to
match the other OGM functions. Implement the Windows version
as returning -1 always.
In the future, the OGM functions could be renamed to express
their purpose: saving and restoring the volume id and tag
associated with a file. Windows currently encodes this
information using the CreationDate and the file name but
in the future this information might be stored in an alternate
data stream.
Jeffrey Altman [Thu, 20 Jan 2011 06:35:08 +0000 (01:35 -0500)]
vol: avoid double dir separators from addtoname
The manner in which addtoname() is used can often result
in double dir separators in path names. Check for it and
avoid doubling the separator in addtoname()
Reviewed-on: http://gerrit.openafs.org/3699 Reviewed-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Rod Widdowson <rdw@steadingsoftware.com> Tested-by: BuildBot <buildbot@rampaginggeek.com>
(cherry picked from commit 959a892afcc1f33871489912a8ea183fc7fca6c2)
Jeffrey Altman [Thu, 20 Jan 2011 06:23:37 +0000 (01:23 -0500)]
vol: clear ih_synced before dropping lock
in ih_reallyclose() clear ihP->ih_synced before dropped
the IH_LOCK. ih_reallyclose() is already committed to
performing the OS_SYNC() call. Failing to clear the ih_synced
flag before dropping the lock permits another thread, the ih_sync_all()
thread for example, to also conclude that it must sync the
file.
Jeffrey Altman [Sun, 16 Jan 2011 20:49:02 +0000 (15:49 -0500)]
Windows: refactor cm_CheckCBExpiration multihomed
cm_CheckCBExpiration() is refactored to make it easier
to read the decision process. cm_CheckCBExpiration()
determines when a callback is no longer usable and as a
result the object status info should be discarded.
The windows cache manager preserves status info past
callback expiration if all of the sources of a volume
became inaccessible prior to the callback expiration
time. The cache manager was improperly preserving the
status info for objects when the callback was issued by
a multi-homed file server when only the interface that
issued the callback is down.
A separate cm_server_t object is used to represent
each file server interface. When one interface goes
down and others are left up, the cache manager will
now replace the down cm_server_t reference for one that
is up. This substitution is performed as a side effect
of computing the effective downTime in cm_CBServersDownTime().
Jeffrey Altman [Fri, 14 Jan 2011 05:22:39 +0000 (00:22 -0500)]
Windows: use cm_ServerEqual() in cm_Analyze()
In cm_Analyze() when processing VNOVOL, VMOVED or VOFFLINE errors
do not use pointer equivalence as the test to determine if a
server is affected by the error. Use cm_ServerEqual() instead.
Fix cm_ServerEqual() to work even when both cm_server_t objects
are not UUID aware.
Jeffrey Altman [Sat, 15 Jan 2011 16:54:40 +0000 (11:54 -0500)]
vol: construct proper VolDir path on Windows
namei_HandleToInodeDir initializes the path name with
the nt_drive value. Therefore calling addtoname(name, name->n_drive)
produces the invalid path "D:\D:\\Vol_.....". Remove the unnecessary
addtoname() call.
Jeffrey Altman [Sat, 15 Jan 2011 16:47:47 +0000 (11:47 -0500)]
vol: use OS_DIRSEP when constructing paths
Do not assume that / is the directory separator when constructing
paths for internal consumption or log messages. Use OS_DIRSEP
or OS_DIRSEPC as appropriate.
Tom Keiser [Thu, 14 Oct 2010 05:24:03 +0000 (01:24 -0400)]
vol: make namei_ListAFSSubDirs deal with multiple/bad linktables
The salvager ends up deadlocking when multiple linktables exist
in the same volume group special directory. The issue is that we
open and flock all discovered linktables, but only close out the
last one found. Consequently, when our child scans the linktables
again, we deadlock against the locked and leaked descriptor(s) our
parent left around before forking.
While we have so far been unable to root-cause the actual creation
of spurious linktables, this patch will at least stop the salvager
from deadlocking against itself when this occurs.
Jeffrey Altman [Sat, 15 Jan 2011 16:39:39 +0000 (11:39 -0500)]
vol: initialize FdHandle_t stack objects
In namei_create() a temporary FdHandle_t is allocated on
the stack but was never initialized. Depending on what
prior data was on the stack the FdHandle_t could result
in subsequent operations failing in unpredictable ways.
Jeffrey Altman [Sat, 15 Jan 2011 16:07:19 +0000 (11:07 -0500)]
vol: Fix ntops to provide expected semantics
nt_open() can be asked to open a file with a path containing
directory components that do not yet exist. Modify nt_open()
to automatically create directories if necessary.
nt_read() and nt_pread() should not treat partial reads due
to reaching EOF as an error. Instead, return the bytes read.
Andrew Deason [Thu, 11 Mar 2010 18:19:47 +0000 (12:19 -0600)]
Parallel I/O extensions to namei backend
This adds the ability for certain namei operations (currently only
ListViceInodes) to occur across multiple different threads in
parallel. Currently this is only enabled when built with the
not-yet-existant AFS_SALSRV_ENV.
Jeffrey Altman [Sat, 15 Jan 2011 16:04:48 +0000 (11:04 -0500)]
vol: Windows requires binary fmode for salvager
In order for salvager to work properly it must read and
write data in binary mode. On Windows, this is accomplished
by calling _set_fmode(_O_BINARY) to set the default mode for
fopen(), etc.
Jeffrey Altman [Sat, 15 Jan 2011 16:01:55 +0000 (11:01 -0500)]
vol: fix OS_LOCKFILE/OS_UNLOCKFILE for Windows
On UNIX the OS_LOCKFILE and OS_UNLOCKFILE macros return zero
on success. Since Windows LockFile() and UnlockFile() return
non-zero (TRUE) on success, negate the return value as part
of the macro definition.
Jeffrey Altman [Fri, 14 Jan 2011 05:13:49 +0000 (00:13 -0500)]
Windows: osilog param size is size_t
The parameter size is size_t which permits pointers to be stored
on 32-bit and 64-bit systems. Make sure we store it as size_t
instead of truncating to long.
Andrew Deason [Sat, 8 Jan 2011 01:48:18 +0000 (20:48 -0500)]
git-version: Do not specify --ignore-submodules
Only new git versions reognize the --ignore-submodules option to
diff-index and diff-files. Do not pass this, to make git-version more
likely to work across different versions, as we don't have any
submodules in the tree anyway.
Jeffrey Altman [Sat, 8 Jan 2011 17:21:23 +0000 (12:21 -0500)]
Windows: refactor buf_Get() to improve readability
Refactor buf_Get() by using a switch() instead of a jumble
of if() conditionals.
Improve comments to make it clear that given the current
use and implementation of cm_BufRead() from cm_dcache.c
that created buffer pages will never be populated with
actual data.
Jeffrey Altman [Tue, 4 Jan 2011 02:07:10 +0000 (21:07 -0500)]
Windows: permit clean when switching platforms
When switching between i386 and amd64 in the same build tree
the "clean" make directive would fail due to NTLang.bat not
having been built in the new platform's destination directory.
Force NTLang.bat to be built as a dependency of cleanup and
delete it at the end along with the version files.
Simon Wilkinson [Sun, 14 Feb 2010 10:01:14 +0000 (10:01 +0000)]
Add "brief" option to rxgen
Add a new -b option to rxgen that turns on "brief" output. This makes a
number of changes to the data definitions produced by rxgen so they can
be more easily used by the calling code.
The changes are:
*) Use the new struct rx_opaque structure for all opaque data
definitions, rather than defining each as a unique structure.
This permits moving opaque data between rxgen structures to be
performed by simple assignment.
*) Use anonymous structures for internal definitions. Currently
rxgen also uses the field name as the structure name, which
prevents the use of a field name more than once within a
source file.
*) Don't embed the structure name within the names of the elements
within the structure. This significantly reduces the length of
assignment code, and makes for more readable callers.
Vincent Archer [Thu, 30 Dec 2010 16:34:28 +0000 (11:34 -0500)]
Windows: MIT license applies to parsemode()
Vincent Archer grants permission to OpenAFS to re-license
his MINIX contributions under the MIT license. A copy of the
e-mail granting permission is located in OpenAFS RT issue 128805.
Jeffrey Altman [Wed, 29 Dec 2010 16:35:17 +0000 (11:35 -0500)]
Windows: buf_CleanAsync scp->fid == bp->fid
If buf_CleanAsync or buf_CleanAsyncLocked are called
with a non-NULL cm_scache_t parameter, that status object's
fid must be the same as the associated cm_buf_t object.
If not, the wrong locks will be held.
If the cm_scache_t parameter is NULL and cm_FindSCache()
returns NULL, it means that the cm_scache_t object associated
with the bp->fid has been flushed from the cache. cm_GetSCache()
must therefore be called to allocate a new status object for the
FID. If the status object cannot be allocated, then any dirty
data stored in the buffer will be discarded.
Jeffrey Altman [Tue, 28 Dec 2010 19:39:47 +0000 (14:39 -0500)]
Windows: fs checkserver skip multi-homed up server
Multi-homed file servers can be detected by comparing the
uuid for the cm_server_t entries. If a server has at least
one up interface, do not list it as being down in the "fs checkserver"
response list.
Jeffrey Altman [Tue, 28 Dec 2010 00:34:14 +0000 (19:34 -0500)]
vos: do not mix memory allocation methods
ListVLDB mixed memory allocated with xdr_alloc() and memory
allocated with malloc(). This is not safe to do since it is
possible on some platforms for xdr_alloc() to allocated memory
using a method other than the malloc() linked to the vos
executable.
Instead of stealing the xdr_alloc()'d buffer, allocate a new
buffer and copy the contents.
Jeffrey Altman [Sat, 18 Dec 2010 23:39:07 +0000 (18:39 -0500)]
Windows: fs chmod and display mode in fs examine
Make use of the new pioctl VIOC_GETUNIXMODE and VIOC_SETUNIXMODE
operations to display the current mode in fs examine and permit
setting the mode with "fs chmod". The mode string parsing makes
use of Vincent Archer's code from Minix. The required copyright
statement and license is attached to parsemode().
afsconfig-windows.h gains definitions for various mode symbols
that are not defined by Visual Studio runtime headers.
Jeffrey Altman [Mon, 20 Dec 2010 14:59:43 +0000 (09:59 -0500)]
Windows: clear mountPointStringp on status change
If the data version of the cm_scache_t changes, we must clear
the mountPointString. Otherwise, if the object is a mountpoint
or symlink, cm_HandleLink() will not attempt to build the new
target string. The Windows CM never updates the contents of
a mountpoint or symlink. It always destroys the old and creates
a new one. However, the Unix CM simply updates the link data.
Apparently, the Windows clients have never noticed.
Andrew Deason [Wed, 8 Dec 2010 20:57:13 +0000 (14:57 -0600)]
DAFS: Avoid logging harmless LEAVE_OFF failures
The DAFS volserver can create temporary destroyMe volumes, which it
then tells the fileserver to keep offline with an FSYNC_VOL_LEAVE_OFF
message. When the fileserver doesn't know that the volume exists,
currently it responds with SYNC_DENIED, which causes the volserver to
log the error with a "negative response on circuit FSSYNC" message.
These are harmless, though, since leaving a volume offline that the
fileserver does not know about is a no-op. So return the SYNC_FAILED
code instead, which does not get logged, and is more consistent with
some other FSYNC handlers like FSYNC_VOL_DONE.
Andrew Deason [Wed, 8 Dec 2010 18:41:44 +0000 (12:41 -0600)]
DAFS: Fix VOL_QUERY_VOP error codes
Volumes in the VOL_STATE_DELETED state effectively do not exist. So,
when receiving an FSYNC_VOL_QUERY_VOP request for a volume, report
FSYNC_UNKNOWN_VOLID for a volume in the VOL_STATE_DELETED state.
Similarly, if the volume exists but on a different partition, we
should return the FSYNC_WRONG_PART reason code.
This allows volumes to be usable by the volume server in some corner
cases. For example, when a volume X exists on partitions A and B, and
the volserver deletes X on B. The fileserver then puts volume X in the
DELETED state, allowing checkouts over fsync, but FSYNC_VOL_QUERY_VOP
reports that no vop exists.
With this change, FSYNC_VOL_QUERY_VOP will instead report that the
volume does not exist, and so FSYNC_VerifyCheckout can succeed for
such volumes.
Reviewed-on: http://gerrit.openafs.org/3493 Reviewed-by: Tom Keiser <tkeiser@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 95fc9ece507d3c5bd7914afadaaae9456eea9b7a)
Andrew Deason [Wed, 8 Dec 2010 17:40:24 +0000 (11:40 -0600)]
Fix AUD_HOST callers
Servers using the AUD_HOST audit message should give the host IP in
network-byte order, as that is how the auditing code interprets it.
Fix the users of AUD_HOST that do not do so.
Stefan Kueng [Fri, 3 Dec 2010 20:24:33 +0000 (21:24 +0100)]
Windows: afs_shl_ext folder bkgrnd context menu
Make the context menu handler also work for folder backgrounds
and on Win7 for library folder backgrounds
For folder backgrounds, the shell passes the PIDL of the folder
instead of a data object. Extract the path from that PIDL. Also
extended the register function of the dll to add the required
registry keys.
Reviewed-on: http://gerrit.openafs.org/3443 Tested-by: BuildBot <buildbot@rampaginggeek.com> Tested-by: Stefan Kueng <tortoisesvn@gmail.com> Reviewed-by: Stefan Kueng <tortoisesvn@gmail.com> Reviewed-by: Jeffrey Altman <jaltman@openafs.org> Tested-by: Jeffrey Altman <jaltman@openafs.org>
(cherry picked from commit f8f8c0a48fbe086870a0cf35a62278c2b4ab743b)
Jeffrey Altman [Wed, 1 Dec 2010 23:47:29 +0000 (18:47 -0500)]
Windows: Remove fallback from GetCaps to GetTime
If the file server responds to RXAFS_GetCapabilities with
RXGEN_OPCODE the server is up and there is no need to fallback to
RXAFS_GetTime. Remove the code that performs the extra work.
Andrew Deason [Thu, 9 Sep 2010 14:41:40 +0000 (09:41 -0500)]
vos: Improve release recovery on timed-out trans
When a portion of 'vos release' takes long enough that a transaction
on one or more RO sites times out, it retries to release to those
sites. However, it does not take into account the situation where the
transaction on the original RO clone has also timed out, which it
usually will have if we took a long time to get to the ForwardMulti
stage for any reason.
Add a check to see if the clone RO transaction has timed out, and if
so, recreate the transaction on it. Check if the volume appears to
have changed at all, and if it has, bail out.
Jeffrey Altman [Fri, 26 Nov 2010 07:55:03 +0000 (02:55 -0500)]
Windows: NSIS installer requires the architecture for CL=1400
The NSIS installer at present is 32-bit only. Production releases
of OpenAFS have always been performed using the CL=1310 compiler
so it wasn't noticed that src/WINNT/install/NSIS/NTMakefile did
not define the ARCH (architecture) for the purpose of identifying
which redistributable runtime library installer should be
integrated into the build. For the CL=1310 compiler there are
not prebuilt installers for OpenAFS to integrate.
Jeffrey Altman [Fri, 26 Nov 2010 07:51:16 +0000 (02:51 -0500)]
Windows: make use of AFSDEV_BIN and set the PATH
ntbuild.bat defines AFSDEV_BIN which is set to the list of
directories from which executables required for the build
process will be executed. However, this value was never used
to modify the PATH environment variable. Make it do so.
Andrew Deason [Fri, 14 Jan 2011 20:00:14 +0000 (14:00 -0600)]
merge-pod: Be more compatible with older perl
merge-pod works fine with perl 5.00503, if we eliminate the 3-argument
invocation of open(). So, replace the open() calls with their
2-argument equivalent, and relax the version requirement a bit.
The build process for OpenBSD builds the no-NFS version of
the kernel module but, at install time, the NFS version is
subseuquently built and installed while the no-NFS version
is ignored. The NFS version does not load and is not needed
in OpenBSD so this patch makes sure that the no-NFS version
is installed and used.
Reviewed-on: http://gerrit.openafs.org/3684 Reviewed-by: Andrew Deason <adeason@sinenomine.net> Tested-by: BuildBot <buildbot@rampaginggeek.com> Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 6d8e3392b77d72f6d05b94fa9c48813ceed5980d)
OpenBSD: Remove duplicate assignment of COMMON_INCLUDE in libafs
When building libafs, the make file variable COMMON_INCLUDE is
assigned a value in two places: Makefile.common.in and
MakefileProto.OBSD.in. The assignment in the latter place is both
wrong and unnecessary. Remove it.
OpenBSD: Don't call non-existent routines in osi_vfsops.c
The OpenBSD version of osi_vfsops.c makes calls to routines
that do not exist in the system and aren't needed. In the
past, these calls were wrappered by an #ifdef for
AFS_DISCON_ENV which was never defined. That #ifdef is now
gone so this patch removes the calls to these routines,
which are remnants of the NBSD origins for this particular
code.