Jeffrey Altman [Fri, 9 Oct 2009 10:23:10 +0000 (06:23 -0400)]
Windows: Prevent fs fetchdata offset bug error from propagating to caller
When the file server fetchdata offset bug is triggered and detected,
not only do we have to behave as if 'length_found' is zero, but we
also must ignore the RXGEN_CC_UNMARSHAL error that will be returned
from rx_EndCall() due to the parsing error that occurred during the
rx_Read32() call.
Marc Dionne [Fri, 9 Oct 2009 01:53:03 +0000 (21:53 -0400)]
Linux: kmem_cache_create fix and cleanup
Fix the kmem_cache constructor function to match the current
expected prototype, and cleanup related code. This has been wrong
for a while, but since we were just passing extra parameters the
only effect was to generate a warning.
- Add a new configure test to detect the new constructor function
signature
- Define the older versions of the constructor in osi_compat.h,
making them call the current version
- Move a few compatibility #defines to osi_compat.h
- Clean up the call to kmem_cache_create by introducing an inline
afs_kmem_cache_create in osi_compat.h
Andrew Deason [Wed, 7 Oct 2009 20:57:04 +0000 (15:57 -0500)]
Reduce warnings in vos.c
Squash a few miscellaneous that appear when building vos.c:
-- CreateVolume should use util_GetUInt32 to parse vol ids, since they
are now unsigned
-- The m_uuid argument of print_addrs is altered, so do not declare it
const
-- Declare the confDir argument of ugen/vsu_ClientInit to be const,
since it does not change, and vos.c passes a const confDir in
MyBeforeProc
-- Ignore implicit function declaration warnings and remove
CFLAGS_NOERROR from the vos.c build, since the only remaining
warning is an implicit function declaration
Andrew Deason [Wed, 7 Oct 2009 22:33:02 +0000 (17:33 -0500)]
Fix warnings in rxkad
Some miscellaneous warnings throughout rxkad:
rxkad/ticket5.c:
- Explicitly cast ticket blob to unsigned char*, as some functions take
a ticket as a char*, and some take it as an unsigned char*
- in tkt_DecodeTicket5, make 'start' and 'end' be unsigned, since that
is what their callers think they are
- Ignore implicit function declaration warnings (for CRC functons)
- Remove general warning inhibition
rxkad/ticket.c:
- in tkt_DecodeTicket, make 'start' and 'end' be unsigned, since that
is what their callers think they are
- Remove general warning inhibition
rxkad/v5der.c:
- In decode_enumerated, use der_get_unsigned instead of der_get_int,
since we're decoding an unsigned integer.
rxkad/rxkad_server.c:
- The last two arguments to rxkad_DeriveXORInfo are char*s, so
explicitly cast them to be so
- Remove general warning inhibition
rxkad/rxkad_common.c:
- rxkad_stats_t has had a couple of spares transform into
clientObjects/serverObjects; account for them and don't overrun the
spares array
Marc Dionne [Thu, 8 Oct 2009 12:12:28 +0000 (08:12 -0400)]
authclient.c: fix 64-bit specific warnings
Commit 36be36e034141ded8fff5ff2edab3223b23c9fcf introduced some casts
from an integer type to void *. This generates warnings on a 64-bit
platform.
Use uintptr_t casts to make the compiler happy.
Jeffrey Altman [Tue, 6 Oct 2009 18:46:09 +0000 (14:46 -0400)]
Permit DNS SRV record lookups to be used by the Windows afsconf_GetAfsdbInfo
Permit DNS SRV record lookups to be used by the Windows afsconf_GetAfsdbInfo
As per the published DNS SRV internet draft for the AFS3 protocol,
DNS SRV records are to replace AFSDB records as the primary method
of looking up AFS3 service location information.
In order to make that happen for Windows:
1. afsconf_GetAfsdbInfo must be able to translate from internal service
names to IANA assigned service names.
2. The Windows getAFSServer() API must accept IANA service and protocol
parameters as well as the port number to use if AFSDB record fallback is
required.
3. The Windows cache manager must pay attention to the port number
stored within the sockaddr_in structure instead of using hard coded
values based upon the server type.
For afsconf_GetAfsdbInfo, in addition to searching for the requested
IANA service, if that service is prserver or kaserver fallback to
the vlserver record (if any).
Andrew Deason [Wed, 7 Oct 2009 22:14:08 +0000 (17:14 -0500)]
xdrproc_t functions take a caddr_t, not caddr_t*
Callers of xdrproc_t functions give the function a foo* as the second
argument, not a foo**, and the xdrproc_t functions themselves expect
this. Make the xdrproc_t typedef accurately reflect that, thereby
ridding us of some warnings.
Andrew Deason [Wed, 7 Oct 2009 22:19:22 +0000 (17:19 -0500)]
Prototype strcasestr in afsmonitor.c
If we HAVE_STRCASESTR, we lack the prototype for strcasestr in
afsmonitor.c unless we def _GNU_SOURCE before including string.h on GNU
systems. To avoid side-effects or pulling in extra stuff, just prototype
strcasestr ourselves for the time being (if we have it) instead of
def'ing _GNU_SOURCE.
Andrew Deason [Wed, 7 Oct 2009 19:52:02 +0000 (14:52 -0500)]
Correct and use AFS_SIZET_FMT
On *nix, a size_t needs either %lu or %u to be interpolated in a string,
depending on if we're on an ILP32 arch or not. Define AFS_SIZET_FMT
accordingly, and actually use AFS_SIZET_FMT when we try to print a
size_t.
This removes numerous -Wformat warnings, at least when building on
i386_linux26.
Reviewed-on: http://gerrit.openafs.org/592 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Tested-by: Jeffrey Altman <jaltman@openafs.org> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Michael Meffie [Sat, 25 Jul 2009 19:28:07 +0000 (15:28 -0400)]
Allow gnu-style long options
New users are surprised when OpenAFS command options
cannot be prefixed with two dashes, as is common
with most modern tools. Permit an extra dash for
common commands when more than one character is
given. For example vos listvol --server foo.bar.com is
equivalent to vos listvol -server foo.bar.com, but
vos listvol --s foo.bar.com is an error.
Andrew Deason [Wed, 7 Oct 2009 17:14:11 +0000 (12:14 -0500)]
Make namei convertROtoRW'd volumes usable
Right now, if you convertROtoRW a volume on namei, the converted volume
appears to need a salvage before it is usable, and the header of the old
(now empty) RO volume is kept around. Fix this:
-- Set inUse = 0 on the converted volume, so the fileserver will be
able to attach the volume when we give it back
-- Unlink the RO header file, instead of trying to unlink the
VI_VOLINFO file twice
-- Log the actual error code (errno) in the error message for the last
unlink
Simon Wilkinson [Fri, 2 Oct 2009 22:15:26 +0000 (23:15 +0100)]
Use page_offset() on Linux
Since time began, Linux has provided page_offset() to turn a
page index into an offset into the corresponding file. Make us use
this everywhere we calculate a page offset, rather than doing our
own maths.
right now we compare against /etc/services, and a hardcoded array.
but since there's no guarantee /etc/services will be useful, this
can mean inconsistent behavior. hardcode the iana names in the table,
and take the internal structure used private at the same time.
Hartmut Reuter [Tue, 6 Oct 2009 07:10:45 +0000 (09:10 +0200)]
New option '-cmd' for 'fs listacl'
prints 'fs setacl' commands necessary to create the current
ACLs. This is useful for backup scripts using TSM (or others)
to save the ACLs into a file to allow for later restore.
Simon Wilkinson [Fri, 2 Oct 2009 22:43:46 +0000 (23:43 +0100)]
Use standard Linux paths for all headers
As we're being built by the standard Linux module build system, we
can use the normal #include <linux/blah.h> convention from including
kernel headers.
Claudio Bisegni [Mon, 5 Oct 2009 11:42:27 +0000 (13:42 +0200)]
OSX Launchd Startup Manage with Preference Pane
OpenAFS preference pane has been updated to manage the new launchd startup mode. Has been delete the custom startup and shutdown file now is user the /Library/OpenAFS/Tools/root.client/usr/vice/etc/afs.rc
OpenAFS.postinstall has been modified to remove the old existing /Library/StartupItems/OpenAFS directory
OpenAFS.pre_upgrade has been modified to shutdown daemon, if it is runnig, and remove the old preference pane to permit the one to be installed
afsio is a command to pipe data into or out of afs files
afsio bypasses the cache manager to achieve higher throughput.
However, it uses the cache manager to stat or create files over the
AFS path.
Besides 'apropos' and 'help' there are 3 subcommands:
'write' pipes data into a new or empty AFS file
'append' pipes data at the end of an existing AFS file
'read' pipes data out of an AFS file
for 'write' there is an option '-md5' which calculates on the fly
the md5 chscksum and prints it to stdout. (Useful when you create
long time archives to keep it separately for later ...)
for 'write' there is also an option '-synthesize <size>' to do just
performance tests. It creates a file which contains at the
begin of each 4 KB block the offset printed in ascii.
for all subcommands exists a '-verbose' option which writes to
stderr data rate and timing information.
All three subcommands may also be used prefixed with 'fid' accepting
then a Fid instead of an AFS path. With the 'fid' prefix also a '-cell'
option is allowed.
Marc Dionne [Thu, 17 Sep 2009 20:57:52 +0000 (16:57 -0400)]
Linux: 2.6.32 - Adapt to writeback changes
Adapt to the writeback changes in kernel 2.6.32
- Since we define our own backing_dev, it needs to be registered with
the writeback code and attached to the super_block. Otherwise it
might get ignored when writeback is needed.
- Each backing_dev now gets its own kernel thread. The name of the
thread is based on the registered name - the openafs one will appear
as "flush-afs".
Marc Dionne [Sun, 4 Oct 2009 17:40:10 +0000 (13:40 -0400)]
rxfs_storePadd: return 0 on success
On success, storePadd should return 0, not the number of bytes
written by the last rx_Write call. In some cases this return
value would leak all the way out to the close() system call,
causing it to fail.
Simon Wilkinson [Fri, 2 Oct 2009 13:34:50 +0000 (14:34 +0100)]
Refactor linux readpage support
The readpage call on Linux was becoming horribly complex.
This patch refactors it into three main components - a function which
fills a locked page (without unlocking it), a function which triggers
a cache bypass operation on a page, and a function which does
conditional prefetch, based on the properties of the page that was just
read.
In addition, we abstract out the cache bypass code further, and provide
static inline variants which are simply stubbed out when cache bypass is
disabled.
Simon Wilkinson [Fri, 2 Oct 2009 12:43:01 +0000 (13:43 +0100)]
Rationalise some #ifdefs in the LINUX osi layer
Introduce osi_compat.h, a place to hide compatibility #ifdefs. Make
a start on using this by purging #ifdefs from osi_vnodeops, and pushing
changes into inline compatibility functions.
The general idea here is to create a single location where we can do our
multiple version compatibility magic, and by doing so reduce the
complexity of the rest of the code. The approach is that where the
kernel introduces a new interface, we should switch to using it in the
rest of the code, and add a compatibility function (in osi_compat.h)
which implements the new interface in terms of the old one, suitably
protected by #ifdefs.
Where an interface changes signature, but not name, we should implement
an afs_<blah> version of the function which has the signature of the
newest version, and within that call out to the real kernel versions,
mapping between the different parameters as appropriate.
Andrew Deason [Wed, 30 Sep 2009 21:50:07 +0000 (16:50 -0500)]
DAFS: Wait until preattach to service FSSYNC reqs
Make FSYNC_sync wait until VInit == 2 before serving FSSYNC requests.
This follows the existing similar behavior of waiting until after volume
preattachment is completed before serving RPC requests in DAFS. This
prevents FSSYNC clients from being denied requests simply due to a
volume not having been preattached yet, at the small cost of delaying
how soon we can serve FSSYNC requests.
This does not alter the behavior of non-DAFS, which still only waits
until VInit == 1, since volume attachment takes significantly longer
than preattachment.
Andrew Deason [Wed, 30 Sep 2009 21:42:46 +0000 (16:42 -0500)]
DAFS: Avoid volserver segfault in GetVolInfo
It is possible for GetVolObject to return success, but result in a NULL
fs_tv. In particular, when the fileserver responds with FSYNC_WRONG_PART
or FSYNC_UNKNOWN_VOLID. It is possible, though difficult, for the
fileserver to respond with FSYNC_UNKNOWN_VOLID if the requested volume
hasn't been preattached on the fileserver yet.
So, in GetVolInfo, also check fs_tv, so we don't try to dereference it
later.
Tom Keiser [Tue, 29 Sep 2009 14:25:55 +0000 (10:25 -0400)]
make VOL_STATE_VLRU_ADD exclusive
VLRU_Add_r() places a volume into VOL_STATE_VLRU_ADD before
calling VLRU_Wait_r(), which drops VOL_LOCK while waiting
for the required VLRU queue to quiesce. Thus, it is essential
that state VOL_STATE_VLRU_ADD be marked exclusive so that
another thread cannot mutate state until the VLRU generation
add transaction completes.
Simon Wilkinson [Tue, 14 Jul 2009 23:02:31 +0000 (00:02 +0100)]
Add support for background page copies
LICENSE BSD
This patch adds support for backgrounding the page copies which are
caused by a call to readpages() In theory, this should improve the
throughput of the AFS kernel module as it permits processes to start
work whilst data is still being read into the page cache for later
pages.
Simon Wilkinson [Tue, 14 Jul 2009 22:55:18 +0000 (23:55 +0100)]
Add support for blocking readahead
This patchset adds support for the readpages() system call, and enables
readahead on Linux. At the moment each page read causes readpages to
block, so the client won't see much benefit from readahead, beyond the
reduction in call overhead.
Reviewed-on: http://gerrit.openafs.org/536 Reviewed-by: Matt Benjamin <matt@linuxbox.com> Tested-by: Derrick Brashear <shadow@dementia.org> Reviewed-by: Derrick Brashear <shadow@dementia.org>
Simon Wilkinson [Tue, 29 Sep 2009 08:25:53 +0000 (10:25 +0200)]
Don't use volatile when it makes no sense
The demand attach code defines volume pointers as volatile in
a number of places, despite the fact that the values in question
aren't being shared between multiple threads. It then also passes
these volatile pointers into functions which don't treat them as
volatile (and even to some which assign them to registers).
This patch simply removes the use of volatile in this situation.
Simon Wilkinson [Tue, 14 Jul 2009 22:39:44 +0000 (23:39 +0100)]
Use readpage, not read for fastpath access
Modify the fast path case so that it uses readpage(), rather than read()
to access data in the cache. This removes a lot of the hidden, uncessary
work that the kernel was doing behind the scenes, and takes advantage of
the fact that we know a page read will always result in a page read
against the backing cache.
Windows: cm_GetBuffer does not need to contact file server when extended a file
If cm_GetBuffer is being called in order to obtain a buffer to store
data beyond the end of the existing file as known to the file server
there is no reason to contact the file server. Instead use the cached
status info in order to allocate a new buffer zero initialized.
This logic avoids triggering the FetchData bug in all file servers
older than 1.4.12 and 1.5.65 in which the file server returns a
large negative number (filesize - requested_offset) when a FetchData
is received where the requested_offset is larger than the filesize.
It also avoids unnecessary work.
Simon Wilkinson [Sat, 26 Sep 2009 16:44:06 +0000 (17:44 +0100)]
Squash more viced warnings
Get rid of more warnings in viced
- Cast fids that we pass into the Dir package to avoid type warnings
- Add ()s to a && b || c, to clarify its meaning - (a && b ) || c
- Don't use %ld to print ints
- Prototype another function from the host package
Simon Wilkinson [Mon, 28 Sep 2009 10:07:58 +0000 (12:07 +0200)]
Add support for warning checked builds
This patch adds a '--enable-checking' configuration option. When this
option is supplied, and gcc is in use, the compiler will treat any
warnings as errors. This will hopefully help stop new warnings from
creeping into the tree.
In order to still be able to build, all of the currently existing
warnings are accepted (these are documented in README.WARNINGS). With
this set of warning inhibitions, the tree is known to build on 32bit
Leopard - other systems may vary. Warning inhibition may be disabled
by supplying --enable-checking=all - in this case the tree will
definitely not build!
If --enabled-checking is not specified, the existing compilation
behaviour is maintained, so there is no user-visible change.
Gcc 4.2, or later, is required to use the pragma sets contained within
this patch. Again, they are not visible unless --enable-checking is
given.
Implementation of new dump tag and length standard
The implementation follows the specification from October 2007.
All old tags are accecpted as before.
New tags 0x06-0x60 (capital letters) are followed by a 1 byte length
field which may contain directly the length (up to 127) or the length
of the then following length field (max 8) ored with 0x80.
New tags 0x61-0x77a are directly followed by 4 bytes building a 32bit
integer.
0x7b-0xf are single-byte tags, 0x7e meaning next tag is critical.
On this basis unknown tags can be skipped unless they were marked
critical. In this case the restore is aborted.
in dump.h tags following the new standard are marked by an asterisk
Last update: added parameter types for HandleUnknownTag.
Simon Wilkinson [Mon, 28 Sep 2009 09:41:52 +0000 (11:41 +0200)]
Avoid redefinition errors for AFS_NONFSTRANS
Some parts of the build provide -DAFS_NONFSTRANS on the command line.
This causes an error when the 'param' file redefines it. Fix this
by #undef ing the token before defining it.
Marc Dionne [Fri, 28 Aug 2009 21:03:44 +0000 (17:03 -0400)]
Use intptr_t and uintptr_t for integer/pointer conversions
Use intptr_t and uintptr_t casts to appease the compiler when
values are converted between 32-bit integers and pointers.
This generates many warnings (currently ~170 on linux amd64)
when pointers are 64-bit.
These types are normally defined in stdint.h, and get defined
by autoconf if that's not the case.
In a few places, NULL is simply replaced with 0 where compared
with an integer.
Simon Wilkinson [Mon, 28 Sep 2009 08:43:24 +0000 (10:43 +0200)]
Make butc logging use va_args
Change the logging functions in butc so that they use va_args. This
means that we can prototype the logging functions, and kill a bunch
more compiler warnings.
Simon Wilkinson [Sat, 26 Sep 2009 17:11:53 +0000 (18:11 +0100)]
Miscellaneous warning cleanup
Assorted warning cleanup, that it didn't seem to make sense having one
patch per file for.
This patch
- Adds some casting for syscall pointers
- ANSIfies SRXAFSCB_GetDE in fsprobe
- Loads sys/ioctl.h on some more platforms in usd
- Includes some missing header files
- Removes unused variables
- Makes it clear that VIsSalvager does handle all enumerated types
- Adds some more prototypes
Simon Wilkinson [Sat, 26 Sep 2009 17:04:06 +0000 (18:04 +0100)]
Remove some more warnings from rx
The mtu variable in InitPeerParams is in the wrong place. It's only
required when the (never used) AFS_USERSPACE_IP_ADDR code is enabled.
Move the variable to the appropriate location, and indent the if and
endif directives in this section to try to make it clearer what the
control flow is (my brain hurt)
Remove the unused rx_pthread_n_event_wakeups variable
ANSIfy rxi_syscall - we can't prototype it yet, sadly.
Simon Wilkinson [Sat, 26 Sep 2009 16:34:48 +0000 (17:34 +0100)]
Fix signed/unsigned warnings in XDR
Our xdr routines use the same native functions to read signed, and
unsigned integers from the wire. This leads to compiler warnings when
the unsigned versions of these functions are called. This patch always
casts to (afs_int32 *) when calling PUT_INT32 and GET_INT32, to resolve
these warnings
Simon Wilkinson [Sat, 26 Sep 2009 16:50:04 +0000 (17:50 +0100)]
Tidy up unlog
Remove warnings from the unlog code
- Prototype internal functions, and make them static
- Define variables appropriately
- Remove uneccesary local function defintions
Simon Wilkinson [Sat, 26 Sep 2009 16:56:48 +0000 (17:56 +0100)]
Squish some warnings in ubik/
Remove some more warnings from the ubik/ directory
- Remove a dangling else case from an if statement
- Prototype ubik_dprint_25
- Don't do (a || b()) - it's horrible, and causes warnings!
Replace with if (!a) b();
Simon Wilkinson [Sat, 26 Sep 2009 19:54:06 +0000 (20:54 +0100)]
Relax compiler specification on Darwin 9 and 10
Commit 1992b702f64f561c75cc74d4645bb0e1afeec6bd made it possible to
relax the compiler choice on a per architecture basis. That commit
allowed the user to specify their compiler of choice on all Linux
systems, but hardcoded 'cc' everywhere else.
This patch permits Darwin 9 and 10 users to also override the default
compiler for user space builds.
Simon Wilkinson [Sat, 26 Sep 2009 09:06:27 +0000 (10:06 +0100)]
Attempt to fix CForeign moredata behaviour
The fetchstore patches changed the behaviour of the multiple chunk handling
that's used in FetchData when the vnode's CForeign bit is set. In the original
code, the data would be read as (length, payload) chunks. The new code does
(length, length, payload), which corrupts the incoming data.
Simon Wilkinson [Thu, 24 Sep 2009 23:27:40 +0000 (00:27 +0100)]
Fix CacheFetchProc in cases where the fileserver hates us
In some situations, the fileserver returns a large negative number
as the length in an FetchData64 call. The old FetchProc code used an int32
to hold this number, and checked length > 0 before attempting to read more
data. The new code uses a uint32, and does while (length), which causes the
cache manage to loop until RX aborts the connection.
This patch restores the old behaviour. length becomes a signed int once more
(and the original 32 bit length from the wire is used, rather than truncating
the 64 bit value), and the conditional checks for > 0.
Andrew Deason [Mon, 21 Sep 2009 17:48:30 +0000 (12:48 -0500)]
Unlink correct files in salvager
Fallout from 0ac956b3384842b3c60e72bde78a4baf58a5877f; a couple of
unlink() calls were not updated to use the absolute path. Update them,
and log errors from unlink(), since otherwise it is difficult to tell
that anything is going wrong.