Andrew Deason [Wed, 7 Oct 2009 19:52:02 +0000 (14:52 -0500)]
Correct and use AFS_SIZET_FMT
On *nix, a size_t needs either %lu or %u to be interpolated in a string,
depending on if we're on an ILP32 arch or not. Define AFS_SIZET_FMT
accordingly, and actually use AFS_SIZET_FMT when we try to print a
size_t.
This removes numerous -Wformat warnings, at least when building on
i386_linux26.
Reviewed-on: http://gerrit.openafs.org/592 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Tested-by: Jeffrey Altman <jaltman@openafs.org> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Michael Meffie [Sat, 25 Jul 2009 19:28:07 +0000 (15:28 -0400)]
Allow gnu-style long options
New users are surprised when OpenAFS command options
cannot be prefixed with two dashes, as is common
with most modern tools. Permit an extra dash for
common commands when more than one character is
given. For example vos listvol --server foo.bar.com is
equivalent to vos listvol -server foo.bar.com, but
vos listvol --s foo.bar.com is an error.
Andrew Deason [Wed, 7 Oct 2009 17:14:11 +0000 (12:14 -0500)]
Make namei convertROtoRW'd volumes usable
Right now, if you convertROtoRW a volume on namei, the converted volume
appears to need a salvage before it is usable, and the header of the old
(now empty) RO volume is kept around. Fix this:
-- Set inUse = 0 on the converted volume, so the fileserver will be
able to attach the volume when we give it back
-- Unlink the RO header file, instead of trying to unlink the
VI_VOLINFO file twice
-- Log the actual error code (errno) in the error message for the last
unlink
Simon Wilkinson [Fri, 2 Oct 2009 22:15:26 +0000 (23:15 +0100)]
Use page_offset() on Linux
Since time began, Linux has provided page_offset() to turn a
page index into an offset into the corresponding file. Make us use
this everywhere we calculate a page offset, rather than doing our
own maths.
right now we compare against /etc/services, and a hardcoded array.
but since there's no guarantee /etc/services will be useful, this
can mean inconsistent behavior. hardcode the iana names in the table,
and take the internal structure used private at the same time.
Hartmut Reuter [Tue, 6 Oct 2009 07:10:45 +0000 (09:10 +0200)]
New option '-cmd' for 'fs listacl'
prints 'fs setacl' commands necessary to create the current
ACLs. This is useful for backup scripts using TSM (or others)
to save the ACLs into a file to allow for later restore.
Simon Wilkinson [Fri, 2 Oct 2009 22:43:46 +0000 (23:43 +0100)]
Use standard Linux paths for all headers
As we're being built by the standard Linux module build system, we
can use the normal #include <linux/blah.h> convention from including
kernel headers.
Claudio Bisegni [Mon, 5 Oct 2009 11:42:27 +0000 (13:42 +0200)]
OSX Launchd Startup Manage with Preference Pane
OpenAFS preference pane has been updated to manage the new launchd startup mode. Has been delete the custom startup and shutdown file now is user the /Library/OpenAFS/Tools/root.client/usr/vice/etc/afs.rc
OpenAFS.postinstall has been modified to remove the old existing /Library/StartupItems/OpenAFS directory
OpenAFS.pre_upgrade has been modified to shutdown daemon, if it is runnig, and remove the old preference pane to permit the one to be installed
afsio is a command to pipe data into or out of afs files
afsio bypasses the cache manager to achieve higher throughput.
However, it uses the cache manager to stat or create files over the
AFS path.
Besides 'apropos' and 'help' there are 3 subcommands:
'write' pipes data into a new or empty AFS file
'append' pipes data at the end of an existing AFS file
'read' pipes data out of an AFS file
for 'write' there is an option '-md5' which calculates on the fly
the md5 chscksum and prints it to stdout. (Useful when you create
long time archives to keep it separately for later ...)
for 'write' there is also an option '-synthesize <size>' to do just
performance tests. It creates a file which contains at the
begin of each 4 KB block the offset printed in ascii.
for all subcommands exists a '-verbose' option which writes to
stderr data rate and timing information.
All three subcommands may also be used prefixed with 'fid' accepting
then a Fid instead of an AFS path. With the 'fid' prefix also a '-cell'
option is allowed.
Marc Dionne [Thu, 17 Sep 2009 20:57:52 +0000 (16:57 -0400)]
Linux: 2.6.32 - Adapt to writeback changes
Adapt to the writeback changes in kernel 2.6.32
- Since we define our own backing_dev, it needs to be registered with
the writeback code and attached to the super_block. Otherwise it
might get ignored when writeback is needed.
- Each backing_dev now gets its own kernel thread. The name of the
thread is based on the registered name - the openafs one will appear
as "flush-afs".
Marc Dionne [Sun, 4 Oct 2009 17:40:10 +0000 (13:40 -0400)]
rxfs_storePadd: return 0 on success
On success, storePadd should return 0, not the number of bytes
written by the last rx_Write call. In some cases this return
value would leak all the way out to the close() system call,
causing it to fail.
Simon Wilkinson [Fri, 2 Oct 2009 13:34:50 +0000 (14:34 +0100)]
Refactor linux readpage support
The readpage call on Linux was becoming horribly complex.
This patch refactors it into three main components - a function which
fills a locked page (without unlocking it), a function which triggers
a cache bypass operation on a page, and a function which does
conditional prefetch, based on the properties of the page that was just
read.
In addition, we abstract out the cache bypass code further, and provide
static inline variants which are simply stubbed out when cache bypass is
disabled.
Simon Wilkinson [Fri, 2 Oct 2009 12:43:01 +0000 (13:43 +0100)]
Rationalise some #ifdefs in the LINUX osi layer
Introduce osi_compat.h, a place to hide compatibility #ifdefs. Make
a start on using this by purging #ifdefs from osi_vnodeops, and pushing
changes into inline compatibility functions.
The general idea here is to create a single location where we can do our
multiple version compatibility magic, and by doing so reduce the
complexity of the rest of the code. The approach is that where the
kernel introduces a new interface, we should switch to using it in the
rest of the code, and add a compatibility function (in osi_compat.h)
which implements the new interface in terms of the old one, suitably
protected by #ifdefs.
Where an interface changes signature, but not name, we should implement
an afs_<blah> version of the function which has the signature of the
newest version, and within that call out to the real kernel versions,
mapping between the different parameters as appropriate.
Andrew Deason [Wed, 30 Sep 2009 21:50:07 +0000 (16:50 -0500)]
DAFS: Wait until preattach to service FSSYNC reqs
Make FSYNC_sync wait until VInit == 2 before serving FSSYNC requests.
This follows the existing similar behavior of waiting until after volume
preattachment is completed before serving RPC requests in DAFS. This
prevents FSSYNC clients from being denied requests simply due to a
volume not having been preattached yet, at the small cost of delaying
how soon we can serve FSSYNC requests.
This does not alter the behavior of non-DAFS, which still only waits
until VInit == 1, since volume attachment takes significantly longer
than preattachment.
Andrew Deason [Wed, 30 Sep 2009 21:42:46 +0000 (16:42 -0500)]
DAFS: Avoid volserver segfault in GetVolInfo
It is possible for GetVolObject to return success, but result in a NULL
fs_tv. In particular, when the fileserver responds with FSYNC_WRONG_PART
or FSYNC_UNKNOWN_VOLID. It is possible, though difficult, for the
fileserver to respond with FSYNC_UNKNOWN_VOLID if the requested volume
hasn't been preattached on the fileserver yet.
So, in GetVolInfo, also check fs_tv, so we don't try to dereference it
later.
Tom Keiser [Tue, 29 Sep 2009 14:25:55 +0000 (10:25 -0400)]
make VOL_STATE_VLRU_ADD exclusive
VLRU_Add_r() places a volume into VOL_STATE_VLRU_ADD before
calling VLRU_Wait_r(), which drops VOL_LOCK while waiting
for the required VLRU queue to quiesce. Thus, it is essential
that state VOL_STATE_VLRU_ADD be marked exclusive so that
another thread cannot mutate state until the VLRU generation
add transaction completes.
Simon Wilkinson [Tue, 14 Jul 2009 23:02:31 +0000 (00:02 +0100)]
Add support for background page copies
LICENSE BSD
This patch adds support for backgrounding the page copies which are
caused by a call to readpages() In theory, this should improve the
throughput of the AFS kernel module as it permits processes to start
work whilst data is still being read into the page cache for later
pages.
Simon Wilkinson [Tue, 14 Jul 2009 22:55:18 +0000 (23:55 +0100)]
Add support for blocking readahead
This patchset adds support for the readpages() system call, and enables
readahead on Linux. At the moment each page read causes readpages to
block, so the client won't see much benefit from readahead, beyond the
reduction in call overhead.
Reviewed-on: http://gerrit.openafs.org/536 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Tested-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Derrick Brashear <shadow@dementia.org>
Simon Wilkinson [Tue, 29 Sep 2009 08:25:53 +0000 (10:25 +0200)]
Don't use volatile when it makes no sense
The demand attach code defines volume pointers as volatile in
a number of places, despite the fact that the values in question
aren't being shared between multiple threads. It then also passes
these volatile pointers into functions which don't treat them as
volatile (and even to some which assign them to registers).
This patch simply removes the use of volatile in this situation.
Simon Wilkinson [Tue, 14 Jul 2009 22:39:44 +0000 (23:39 +0100)]
Use readpage, not read for fastpath access
Modify the fast path case so that it uses readpage(), rather than read()
to access data in the cache. This removes a lot of the hidden, uncessary
work that the kernel was doing behind the scenes, and takes advantage of
the fact that we know a page read will always result in a page read
against the backing cache.
Windows: cm_GetBuffer does not need to contact file server when extended a file
If cm_GetBuffer is being called in order to obtain a buffer to store
data beyond the end of the existing file as known to the file server
there is no reason to contact the file server. Instead use the cached
status info in order to allocate a new buffer zero initialized.
This logic avoids triggering the FetchData bug in all file servers
older than 1.4.12 and 1.5.65 in which the file server returns a
large negative number (filesize - requested_offset) when a FetchData
is received where the requested_offset is larger than the filesize.
It also avoids unnecessary work.
Simon Wilkinson [Sat, 26 Sep 2009 16:44:06 +0000 (17:44 +0100)]
Squash more viced warnings
Get rid of more warnings in viced
- Cast fids that we pass into the Dir package to avoid type warnings
- Add ()s to a && b || c, to clarify its meaning - (a && b ) || c
- Don't use %ld to print ints
- Prototype another function from the host package
Simon Wilkinson [Mon, 28 Sep 2009 10:07:58 +0000 (12:07 +0200)]
Add support for warning checked builds
This patch adds a '--enable-checking' configuration option. When this
option is supplied, and gcc is in use, the compiler will treat any
warnings as errors. This will hopefully help stop new warnings from
creeping into the tree.
In order to still be able to build, all of the currently existing
warnings are accepted (these are documented in README.WARNINGS). With
this set of warning inhibitions, the tree is known to build on 32bit
Leopard - other systems may vary. Warning inhibition may be disabled
by supplying --enable-checking=all - in this case the tree will
definitely not build!
If --enabled-checking is not specified, the existing compilation
behaviour is maintained, so there is no user-visible change.
Gcc 4.2, or later, is required to use the pragma sets contained within
this patch. Again, they are not visible unless --enable-checking is
given.
Implementation of new dump tag and length standard
The implementation follows the specification from October 2007.
All old tags are accecpted as before.
New tags 0x06-0x60 (capital letters) are followed by a 1 byte length
field which may contain directly the length (up to 127) or the length
of the then following length field (max 8) ored with 0x80.
New tags 0x61-0x77a are directly followed by 4 bytes building a 32bit
integer.
0x7b-0xf are single-byte tags, 0x7e meaning next tag is critical.
On this basis unknown tags can be skipped unless they were marked
critical. In this case the restore is aborted.
in dump.h tags following the new standard are marked by an asterisk
Last update: added parameter types for HandleUnknownTag.
Simon Wilkinson [Mon, 28 Sep 2009 09:41:52 +0000 (11:41 +0200)]
Avoid redefinition errors for AFS_NONFSTRANS
Some parts of the build provide -DAFS_NONFSTRANS on the command line.
This causes an error when the 'param' file redefines it. Fix this
by #undef ing the token before defining it.
Marc Dionne [Fri, 28 Aug 2009 21:03:44 +0000 (17:03 -0400)]
Use intptr_t and uintptr_t for integer/pointer conversions
Use intptr_t and uintptr_t casts to appease the compiler when
values are converted between 32-bit integers and pointers.
This generates many warnings (currently ~170 on linux amd64)
when pointers are 64-bit.
These types are normally defined in stdint.h, and get defined
by autoconf if that's not the case.
In a few places, NULL is simply replaced with 0 where compared
with an integer.
Simon Wilkinson [Mon, 28 Sep 2009 08:43:24 +0000 (10:43 +0200)]
Make butc logging use va_args
Change the logging functions in butc so that they use va_args. This
means that we can prototype the logging functions, and kill a bunch
more compiler warnings.
Simon Wilkinson [Sat, 26 Sep 2009 17:11:53 +0000 (18:11 +0100)]
Miscellaneous warning cleanup
Assorted warning cleanup, that it didn't seem to make sense having one
patch per file for.
This patch
- Adds some casting for syscall pointers
- ANSIfies SRXAFSCB_GetDE in fsprobe
- Loads sys/ioctl.h on some more platforms in usd
- Includes some missing header files
- Removes unused variables
- Makes it clear that VIsSalvager does handle all enumerated types
- Adds some more prototypes
Simon Wilkinson [Sat, 26 Sep 2009 17:04:06 +0000 (18:04 +0100)]
Remove some more warnings from rx
The mtu variable in InitPeerParams is in the wrong place. It's only
required when the (never used) AFS_USERSPACE_IP_ADDR code is enabled.
Move the variable to the appropriate location, and indent the if and
endif directives in this section to try to make it clearer what the
control flow is (my brain hurt)
Remove the unused rx_pthread_n_event_wakeups variable
ANSIfy rxi_syscall - we can't prototype it yet, sadly.
Simon Wilkinson [Sat, 26 Sep 2009 16:34:48 +0000 (17:34 +0100)]
Fix signed/unsigned warnings in XDR
Our xdr routines use the same native functions to read signed, and
unsigned integers from the wire. This leads to compiler warnings when
the unsigned versions of these functions are called. This patch always
casts to (afs_int32 *) when calling PUT_INT32 and GET_INT32, to resolve
these warnings
Simon Wilkinson [Sat, 26 Sep 2009 16:50:04 +0000 (17:50 +0100)]
Tidy up unlog
Remove warnings from the unlog code
- Prototype internal functions, and make them static
- Define variables appropriately
- Remove uneccesary local function defintions
Simon Wilkinson [Sat, 26 Sep 2009 16:56:48 +0000 (17:56 +0100)]
Squish some warnings in ubik/
Remove some more warnings from the ubik/ directory
- Remove a dangling else case from an if statement
- Prototype ubik_dprint_25
- Don't do (a || b()) - it's horrible, and causes warnings!
Replace with if (!a) b();
Simon Wilkinson [Sat, 26 Sep 2009 19:54:06 +0000 (20:54 +0100)]
Relax compiler specification on Darwin 9 and 10
Commit 1992b702f64f561c75cc74d4645bb0e1afeec6bd made it possible to
relax the compiler choice on a per architecture basis. That commit
allowed the user to specify their compiler of choice on all Linux
systems, but hardcoded 'cc' everywhere else.
This patch permits Darwin 9 and 10 users to also override the default
compiler for user space builds.
Simon Wilkinson [Sat, 26 Sep 2009 09:06:27 +0000 (10:06 +0100)]
Attempt to fix CForeign moredata behaviour
The fetchstore patches changed the behaviour of the multiple chunk handling
that's used in FetchData when the vnode's CForeign bit is set. In the original
code, the data would be read as (length, payload) chunks. The new code does
(length, length, payload), which corrupts the incoming data.
Simon Wilkinson [Thu, 24 Sep 2009 23:27:40 +0000 (00:27 +0100)]
Fix CacheFetchProc in cases where the fileserver hates us
In some situations, the fileserver returns a large negative number
as the length in an FetchData64 call. The old FetchProc code used an int32
to hold this number, and checked length > 0 before attempting to read more
data. The new code uses a uint32, and does while (length), which causes the
cache manage to loop until RX aborts the connection.
This patch restores the old behaviour. length becomes a signed int once more
(and the original 32 bit length from the wire is used, rather than truncating
the 64 bit value), and the conditional checks for > 0.
Andrew Deason [Mon, 21 Sep 2009 17:48:30 +0000 (12:48 -0500)]
Unlink correct files in salvager
Fallout from 0ac956b3384842b3c60e72bde78a4baf58a5877f; a couple of
unlink() calls were not updated to use the absolute path. Update them,
and log errors from unlink(), since otherwise it is difficult to tell
that anything is going wrong.
Andrew Deason [Tue, 22 Sep 2009 21:59:34 +0000 (16:59 -0500)]
Solaris: Default to fs_pathconf in afs_pathconf
Instead of returning EINVAL in afs_pathconf when we don't recognize the
pathconf command, return the result of fs_pathconf. fs_pathconf provides
sensible defaults or correctly represents a filesystem not implementing
a particular pathconf cmd; all in-tree (Open)Solaris filesystems call
fs_pathconf for unimplemented commands. Returning EINVAL allegedly
represents that the fileystem does not support pathconf at all.
Also, account for OpenSolaris adding another parameter to
pathconf-related calls.
Simon Wilkinson [Wed, 23 Sep 2009 17:18:34 +0000 (18:18 +0100)]
Refactor the cache bypass determination code
The code which determines whether the cache can be bypassed occurs in
multiple locations in this file. Make a single static inline function
that can do this, and use it.
Andrew Deason [Mon, 21 Sep 2009 21:57:01 +0000 (16:57 -0500)]
Implement _PC_FILESIZEBITS for solaris pathconf
Using recent NFS clients and servers with the translator under Solaris
causes AFS to be queried for the _PC_FILESIZEBITS pathconf value. Right
now we don't implement it and return EINVAL, causing at least some
modern NFS clients to be unable to mount AFS via the translator on at
least some modern NFS servers.
So, return _PC_FILESIZEBITS as either 32 or 64, depending on whether we
are a 64-bit client or not.
Andrew Deason [Mon, 21 Sep 2009 19:28:29 +0000 (14:28 -0500)]
Avoid salvager vol header read assert
When we read the volume header in order to write it back with a modified
inUse while salvaging, we were aborting if we couldn't read the header.
Since we can fail to read the header data if the volume header file
isn't associated with any data (and will be deleted by the salvager),
don't abort. Do still abort if we can't write the data back, since if
the data can be read but not written, other programs may think that the
volume is not being salvaged.
Andrew Deason [Mon, 21 Sep 2009 16:09:56 +0000 (11:09 -0500)]
DAFS: Put back volumes we get in FSSYNC handlers
fssync-server.c has a few issues with not properly VPutVolume_r'ing
volumes that were VGetVolume_r'd. The most important of these was that
the handler for FSYNC_VOL_MOVE can put back a volume it did not get,
when DAFS is enabled.
Fix that, and another erroneous edge case in the FSYNC_VOL_QUERY_HDR
handler when DAFS is not enabled.
Windows: Protect against null serverp in cm_Analyze
If cm_Analyze is called with both 'serverp' and 'connp' set
to NULL and an RX error is provided, a null pointer dereference
will occur. Test for non-NULL serverp before dereferencing
Windows: Do not mark server down due to Store / Fetch Data protocol error
When performing a StoreData or FetchData operation there are
several data validation checks performed to ensure that the
lengths of data obtained with rx_Write and rx_Read are consistent
with the RXAFS_FetchData and RXAFS_StoreData protocol operations.
When an inconsistency is detected the cache manager terminates the
call and returns an error to the caller which is passed to cm_Analyze().
The cache manager was returning -1 as the error code which is
equivalent to RX_CALL_DEAD which in turn will result in the server
being marked down.
This commit makes the following changes:
. add trace logging to permit monitoring this case
. instead of returning -1 return either RX_PROTOCOL_ERROR or RX_EOF
depending on the situation
. in cm_Analyze do not mark a server as down for rx errors other
than RX_CALL_DEAD. Instead, force a new connection and retry
until the request timeout limit is reached.
Windows: Use "Name not found" instead of "File not found"
When a file cannot be opened because the name of the file or
a directory in the path cannot be found, return "Name not found"
instead of "File not found". "File not found" is the error returned
by the smb redirector to the application when the file cannot be
located in the search path. "Name not found" indicates that the
entry is not present at the location requested.
The use of "File not found" prevented execution of applications
from the \\AFS name space on Windows 7.
Windows: trace logging changes for smb_ReceiveTran2QPathInfo
Add new trace log entries in smb_ReceiveTran2QPathInfo to
help track down the reason for queries in which the path and
tidPath are both the empty string.
Windows: Improve WKSSRC and SRVSVC compatibility with Windows
This commit includes several changes to improve compatibility
with Windows (in particular Windows 2000).
1. Specify a Lan Workgroup name. We use "AFS".
2. Ensure that the server name returned does not include
leading slash or backslash characters. The name provided
might include them but they are not part of the name and
cannot be returned.
3. The NetrWkstaGetInfo and NetrServerGetInfo responses must
be consistent. Otherwise, the Explorer Shell will get confused and
refuse to provide access to the server shares.
This commit also includes some minor comment changes.
Even if the rand_s() function is supported by the compiler
it is not supported on Windows 2000 because the kernel level
functionality it requires does not exist on that platform.
Calling rand_s() on Windows 2000 will throw an exception
and terminate the service.
Marc Dionne [Mon, 14 Sep 2009 21:40:23 +0000 (17:40 -0400)]
Linux: Restore aklog -setpag functionality for kernel 2.6.32+
With kernel 2.6.32 it is now possible for a process to copy its session
keyring to its parent through the use of the KEYCTL_SESSION_TO_PARENT
function of the keyctl syscall.
We can't use this easily from kernel space to cover all calls to
VIOCSETTOK with the setpag flag - we'd need to make a syscall
or have keyctl exported. Instead, a hook is added to ktc_SetToken to
make it honour the AFS_SETTOK_SETPAG flag, which was ineffective with recent
kernels.
This should cover the most common cases (ex: aklog) where this is needed.
The syscall is coded directly to avoid introducing a dependency on the
keyutils library or header files which may not be installed everywhere.
An initial pass at adding indexing to the Windows release notes.
The next pass should refine the index terms and add appropriate
secondary index values.