Windows: Preserve volume location info in case of comm fail
The cache manager refreshes volume location information every
two hours. If during a refresh the communication with the
vldb server fails, the previously known volume location information
should continue to be used.
The previous behavior in which the volume location information
is discarded first and then the update is performed can result
in unnecessary client failures when a temporary disruption in
communication with the vldb server occurs. Instead, wait until
we have a successful response from the vldb server before the
previous server list is discarded.
The idle error value (if any) is stored in the cm_req_t object.
Since the value is never cleared, the same value can be returned
for all requests that make use of the same cm_req_t object.
Change the behavior to only return an idle error once and then
clear it.
Rx: make conn_call_lock and conn_data_lock usage consistent
The rx_connection.flags field is protected by the conn_data_lock
but the conn_data_lock is not held everywhere the conn flags
field is altered. This produces a race that can result in a
deadlock when waiter flags are inadvertently prevented from being
cleared.
The conn_call_lock usage in rx_EndCall which was removed in
Change e169708681eb1bbbb31951b95f68e861a4b01c7e must be restored.
If rx_EndCall never obtains the conn_call_lock it can't ensure
that the thread in rx_NewCall actively checking the calls will
not end up blocking when there is now a call channel that can
be reused. This usage of conn_call_lock can be removed only
if a true producer/consumer model is implemented.
Windows: cm_UpdateCell must hold cell lock across server random
cm_UpdateCell fails to hold the cell lock across the server
randomization. As a result the vlserver list can be destroyed
while randomization is taking place.
Windows: CM_SCACHESYNC_STOREDATA for non-files have no buffers
Do not add QData objects with null cm_buf_t pointers to the
cm_scache_t bufWritep queue when synchronizing directory changes.
If a callback is required while the directory change is being
pushed it can result in a deadlock.
Windows: define new event log messages for cm_Analyze VBUSY, VRESTARTING, etc.
Add MSG_SERVER_REPORTS_VBUSY, MSG_SERVER_REPORTS_VRESTARTING,
MSG_ALL_SERVERS_BUSY, MSG_ALL_SERVERS_OFFLINE,
and MSG_ALL_SERVERS_DOWN.
Add event message throttling. Only permit one copy of a message
to be generated every five seconds if the message will duplicate
the prior message. This often occurs when a server or volume becomes
inaccessible and there were a large number of requests queued on it.
Integrate these new messages into cm_Analyze processing for VBUSY,
VRESTARTING, ALLDOWN, ALLOFFLINE, and ALLBUSY errors.
Windows: wait for I/O on buffers to complete in cm_SetupStoreBIOD
cm_SetupStoreBIOD constructs a list of dirty buffers for a file
that are to be written to the file server. When constructing
the list, if when determining the first dirty buffer we come across
a buffer that is already actively involved in an I/O operation,
call buf_WaitIO() to wait until the buffer is no longer busy before
continuing. This reduces lock contention and synchronization
conflicts.
Windows: split cm_buf_t.flags field to ensure proper locking
It turns out that for all these years the locks protecting
the cm_buf_t flags field have been racy. Some of the flags
were protected by the cm_buf_t mutex and others by the
buf_globalLock. This patchset splits the flags field so that
the appropriate lock will be used exclusively to protect a
common set of flags.
we could mask the mode setting on symlinks, however, it would be nice
to change the fileserver to allow mode setting on symlinks in some
(safe) cases. preserve our ability to do so.
Harald Barth [Fri, 16 Apr 2010 05:45:35 +0000 (01:45 -0400)]
Add vos setaddrs command and man page
The vos setaddrs command sets the IP addresses for a server entry
in the Volume Location Database (VLDB). Specify one or serveral hosts.
All existing hosts in the VLDB entry are replaced with the new entries
on the command line.
The src/config/param.rs_aix61.h source file was stored
in the repo with CR-LF end of line. This is causing
problems for Windows Git which converts CR-LF to LF
for storage in the repo.
Simon Wilkinson [Thu, 15 Apr 2010 19:52:11 +0000 (20:52 +0100)]
Tidy up UKERNEL includes
UKERNEL is just another userspace build - there's no need to
maintain completely separate header file lists in each object file
for "userspace" and "ukernel". Tidy this up to improve the
readability of these sections of code.
Change-Id: I69f476a0b8aae1204cd4207c7c656ec7e07184df
Reviewed-on: http://gerrit.openafs.org/1758 Tested-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Thread safety in rx_NewCall requires that only one thread be
actively allocating or recycling a call at a time. Since we are
no longer holding the conn_call_lock across the entire transaction
we need to have another synchronization mechanism. Add a new
rx_connection flag, RX_CONN_MAKECALL_ACTIVE, which when set indicates
that a thread is actively obtaining a call. If any other threads
see this flag set, they will wait until being signalled that the
thread has completed its activity.
In addition, because the call->lock may be dropped when processing
rxi_ResetCall(), we must hold a reference to the call once we
begin using it. Otherwise, the call may be garbage collected
behind our back.
new contact to a fileserver can trigger an InitCallBackStateN RPC
to us, which our agent will need afs_xserver to handle. don't hold it;
we only need it to fill in capabilities. racing here is ok.
Change-Id: Ie0aaea3ab462e421bd31ba3b703d8cd0cb0d61df
Reviewed-on: http://gerrit.openafs.org/1754 Reviewed-by: Marc Dionne <marc.c.dionne@gmail.com> Tested-by: Marc Dionne <marc.c.dionne@gmail.com> Reviewed-by: Andrew Deason <adeason@sinenomine.net> Tested-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Autogenerate a Debian changelog for correct package versioning
The Debian packaging infrastructure takes the package version from the
most recent entry in the changelog file. Change the changelog file to
a template and add an entry to the top that will be set to the current
version of OpenAFS, with a Debian revision that will sort before any
official package.
Andrew Deason [Tue, 6 Apr 2010 22:07:33 +0000 (17:07 -0500)]
Use afsd code in libuafs
Share the same CM code for the kernel client as in libuafs, so we
don't duplicate code for initializing the cache and other things. In
order to do this:
- Remove some libuafs global variables that share name and
functionality with those in afsd, and declare some static
- Remove uafs_Init(), and move the ukernel-specific code in it to
osi_Init(); replace with uafs_Setup(), uafs_ParseArgs(), and
uafs_Run(), which just call into afsd functions
- Remove libuafs' cache initialization code (CreateCacheFile,
SweepAFSCache, etc); instead just use afsd's
- Add uafs_mount(), to perform the 'mount'ing step that takes place
in the normal kernel CM
- Add afsd_uafs.c for the glue between afsd and libuafs
Marc Dionne [Tue, 13 Apr 2010 22:58:11 +0000 (18:58 -0400)]
Fix new UKERNEL warnings on 64-bit
Commit 830cb48c enabled new warnings when building UKERNEL, which
causes builds with --enable-checking to fail. These are 64-bit
specific warnings from int to pointer conversions and one printf
warning.
Changes:
- cast printf argument to (int) in afs_usrops.c
- use (iparmtype)(uintptrsz) to convert 32-bit integers to
pointers
- move the definition of uintptrsz to src/afs/afs.h so its
available to other source files, and remove the original definition
in afs_syscall.c
Michael Meffie [Tue, 12 Jan 2010 02:16:06 +0000 (21:16 -0500)]
DAFS: avoid volume lock contention during initialization
Avoid the excessive volume lock contention during startup to
improve the time to pre-attach a very large number of volumes.
The parallel attach worker threads avoid the volume lock
while scanning the partitions for volumes and send batches of
volume ids to the main thread to be preattached under the
volume lock.
Felix Frank [Thu, 4 Mar 2010 03:41:15 +0000 (22:41 -0500)]
Fileserver capabilities support for the UNIX client
The attached patch has the client perform a GetCapabilities RPC
on fileservers it encounters.
It uses an additional server flag bit to keep track of the servers that
have been queried already.
In the case of afs_CeckServers(), GetTime RPCs are largely replaced by
GetCapabilities. GetTime is performed on a server if and only if
afs_setTime is nonzero and either
(a) no setTimeHost has yet been determined or
(b) the server in question has been designated as setTimeHost
The GetServers() function could thus be simplified even further wrt. the
setTime mechanism, but doing so would imply more rewriting (violating
the KISS principle; a followup patch should deal with that).
When a client is asked to reset callback states, it also resets the
"capabilites known" bit.
Thanks go to Simon Wilkinson, Jeffrey Altman and Jeffrey Hutzelman for
input regarding logic and implementation details.
Rx: avoid out of order lock acquisition in rx_NewCall
Sha-1 33010ef25e716f2ec2df17cc113f4ef8f67e3a74 broke the lock order
conventions between the conn->conn_call_lock and the call-lock.
This patchset corrects the ordering and handles the synchronization
issues that might occur when the call->lock is dropped within
rx_NewCall.
Andrew Deason [Mon, 12 Apr 2010 17:39:00 +0000 (12:39 -0500)]
Do not turn off AFS_HAVE_STATVFS for UKERNEL
Many param files turn off AFS_HAVE_STATVFS for UKERNEL. We obviously
still have statvfs() available whether we are running with UKERNEL or
not, so modify param files to enable it for UKERNEL if it was enabled
for non-UKERNEL.
The only places using this define are afsd and vol/partition.c, the
latter of which will not be affected.
Matt Smith [Sat, 10 Apr 2010 06:36:59 +0000 (01:36 -0500)]
Fix problems from afs_osi_gcpags reorganization
Corrections to mistakes made during the reorganization
of afs_osi_gcpags.c to per-OS directories. Includes fixes to
LINUX24 and whitespace corrections.
Michael Meffie [Sat, 10 Apr 2010 01:03:09 +0000 (21:03 -0400)]
afsmonitor: show busy counts
Update afsmonitor to display rx_nBusies, fs_nBusies,
sysname_ID, and fs_GetCapabilities, which where claimed from
spare fields long ago. Add a new group name called
Busies_group to show just the busy fields.
Marc Dionne [Sat, 10 Apr 2010 00:19:19 +0000 (20:19 -0400)]
Fix UKERNEL build error - include afs/afs_osi.h
Commit d0f2ffca breaks the build on my system because
afs/afs_osi.h no longer gets included for the UKERNEL + !KERNEL
case. This causes many errors as a result of missing definitions
such as afs_kmutex_t.
Pull out this include into a separate ifdef based only on "KERNEL".
Matt Smith [Fri, 9 Apr 2010 18:32:46 +0000 (13:32 -0500)]
Move contents of afs_osi_gcpags to per-OS files
Moved the relevant chunks from src/afs/afs_osi_gcpags.c
to osi_gcpags.c located in each of the OS subdirectories.
Also updated libafs and libuafs to reflect these changes.
Andrew Deason [Thu, 1 Apr 2010 18:55:51 +0000 (13:55 -0500)]
Correct incorrect type-punning fixes
Commit 549002c906795f978eebf81c706995116a04a8ff attempted to resolve a
few type-punning-related warnings, but did so using unions. Unions are
not guaranteed to work correctly with type-punning on non-gcc, so
partially revert and reimplement.
Michael Meffie [Thu, 8 Apr 2010 17:27:36 +0000 (13:27 -0400)]
afsmonitor: add fs callback xstats collection
Adds the capability to afsmonitor to display and process
thresholds for fileserver callback xstats collections. The
callback counters are placed to the right of the full perf
data. When afsmonitor is started without a configuration
file, the callback xstats data is retrieved and displayed,
since by default afsmonitor will show every field. When a
configuration file is given, poll the fileserver only for the
collections needed to display information requested by the
'show fs' commands.
Michael Meffie [Fri, 9 Apr 2010 15:46:10 +0000 (11:46 -0400)]
afsmonitor: avoid showing full perf stats garbage
Unfortunately, the full perf stats contain timeval structures which
do not have the same size everywhere. Avoid displaying gargbage,
but at least try to show the overall stats.
Andrew Deason [Thu, 8 Apr 2010 20:35:12 +0000 (15:35 -0500)]
UKERNEL: allow creation of non-detached threads
Make usr_thread_create create a non-detached thread by default, and
just have callers call usr_thread_detach if they want it detached. The
only current caller of usr_thread_create already calls
usr_thread_detach.
Also add usr_thread_join to make it possible to join a created thread.
Andrew Deason [Fri, 14 Aug 2009 21:17:11 +0000 (16:17 -0500)]
Make osi_GetTime work on 64-bit libuafs
libuafs was previously using an implementation of osi_GetTime (and
thus clock_GetTime) that required 32-bit time_t's to work properly.
Instead, just use the non-kernel osi_GetTime for UKERNEL, since we're
just in userspace either way.
The Debian packaging files in the tree were ancient. Update to the
packaging files from the just-uploaded 1.5.73.3-1 version, which should
build and correctly install the current development source.
The current packaging files no longer use a hard-coded version in
debian/rules, so remove the configure machinery for creating it. The
correct way of handling the version number is to generate a new
changelog entry. That work will come in a subsequent patch.
Simon Wilkinson [Tue, 6 Apr 2010 23:29:44 +0000 (00:29 +0100)]
Linux: kmap() not page_address()
Using page_address() will return NULL if the page is in highmem. To
avoid this, we must kmap() the page we're getting the address of,
and kunmap() when we are done. If the page isn't in high memory, then
kmap() is equivalent to page_address().
aklog was previously writing the magic AFS ID string into previously
alloated memory with sprintf, but the variable in question was only
as long as the username, so this code could overwrite memory and lead
to heap corruption. Free previously allocated memory and use
afs_asprintf to format the AFS ID string instead.
Add a caution to the fileserver man page explaining that traditional
and demand-attach require different configurations, and also mention
that there are two different server implementations. Add an example
of a bos create command for creating a demand-attach File Server to
the fileserver man page.
Add a caution to the bos create man page that a traditional fs node
won't work with demand-attach and vice versa. Document the necessary
arguments for the dafs type. Clarify in EXAMPLES which bos create
commands are traditional and which are demand-attach. Add an example
of changing from a traditional to a demand-attach configuration.
Change kaserver to ptserver in the example of a simple process.
Michael Meffie [Wed, 10 Mar 2010 14:48:38 +0000 (08:48 -0600)]
pts mem -expandgroups option
Improve support for supergroups in the pts membership command
with a new option called -expandgroups. This option will
recursively show the complete membership of users and groups.
The expanded members of a group are all the users which are
members of all of the group's sub-groups. The expanded groups
of a user are all the groups which are supergroups of the
users's groups.
Change-Id: I811a4e5e73632e5e205fe10f3f3a36a98464d49e
Reviewed-on: http://gerrit.openafs.org/1601 Reviewed-by: Andrew Deason <adeason@sinenomine.net> Tested-by: Andrew Deason <adeason@sinenomine.net> Reviewed-by: Derrick Brashear <shadow@dementia.org> Tested-by: Derrick Brashear <shadow@dementia.org>
Simon Wilkinson [Tue, 6 Apr 2010 23:29:44 +0000 (00:29 +0100)]
Linux: kmap() not page_address()
Using page_address() will return NULL if the page is in highmem. To
avoid this, we must kmap() the page we're getting the address of,
and kunmap() when we are done. If the page isn't in high memory, then
kmap() is equivalent to page_address().
Explain in CellServDB man page that server lines can be omitted
It's possible to use AFSDB records only to locate the VLDB servers but
still list the cell in the client CellServDB so that the client is
aware of it and populates it into dynroot. Describe doing this in the
man page.
aklog was previously writing the magic AFS ID string into previously
alloated memory with sprintf, but the variable in question was only
as long as the username, so this code could overwrite memory and lead
to heap corruption. Free previously allocated memory and use
afs_asprintf to format the AFS ID string instead.
Add a caution to the fileserver man page explaining that traditional
and demand-attach require different configurations, and also mention
that there are two different server implementations. Add an example
of a bos create command for creating a demand-attach File Server to
the fileserver man page.
Add a caution to the bos create man page that a traditional fs node
won't work with demand-attach and vice versa. Document the necessary
arguments for the dafs type. Clarify in EXAMPLES which bos create
commands are traditional and which are demand-attach. Add an example
of changing from a traditional to a demand-attach configuration.
Change kaserver to ptserver in the example of a simple process.
* Upstream no longer includes files covered by the Apple Public Source
License, so we no longer have to repackage the upstream source to
remove those files. Drop Debian patches to the build system that were
compensating for the removal of those files.
* Switch to generating upstream tarballs from the upstream Git tag
rather than from tarball releases. There's no reason to include all
the generated files when we're going to regenerate them anyway, and
this saves effort combining the doc and src tarballs.
Russ Allbery [Fri, 26 Mar 2010 19:25:01 +0000 (12:25 -0700)]
Don't add CellServDB records if AFSDB is in use
* If the user configures openafs-client to use AFSDB records for VLDB
server location, don't prompt the user for VLDB servers for the local
cell even if they're not present in CellServDB and don't try to add an
entry for the local cell to CellServDB. (Closes: #575299)
Windows: Support new Cygwin docbook stylesheet location
Make the computation of the docbook stylesheet location
based upon testing for directory existence now that versions
of cygwin 1.7.2 and place the stylesheets in a new location.
projects / packages / o / openafs.git / log