rx: Reverse the consumption order of idle queue

Currently, the rx server thread idle queue is used in an LRU manner.
This means that we round robin requests between all of the threads
configured on a given system, which means that we end up thrashing
CPU caches on machines whose workload doesn't require that all of
the configured threads be used.

Change this so that we always use the most recently idle thread. This
isn't as "fair" to all of our waiting threads, but should mean that we
scale better on SMP machines, as a thread that is recently idle is
likely to have been recently scheduled.

Performance numbers to follow ...

Reviewed-on: http://gerrit.openafs.org/4871
Reviewed-by: Derrick Brashear <shadow@dementia.org>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@openafs.org>
Tested-by: Jeffrey Altman <jaltman@openafs.org>
(cherry picked from commit 5614305853f0f87a2418a6fb7ca472a8ef966084)

Change-Id: I6b961a470adfe053ecd85af05821e42898fd7847
Reviewed-on: http://gerrit.openafs.org/4912
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementia.org>

diff --git a/src/rx/rx.c b/src/rx/rx.c
index 9b5266a003dfbc83ffbda2bb91fe2979e57f84f6..3b62030c89100628d2dfe0b1cfc2d401b4b0f80a 100644 (file)
--- a/src/rx/rx.c
+++ b/src/rx/rx.c
@@ -4741,7 +4741,7 @@ rxi_AttachServerProc(struct rx_call *call,
            queue_Append(&rx_incomingCallQueue, call);
        }
     } else {
-       sq = queue_First(&rx_idleServerQueue, rx_serverQueueEntry);
+       sq = queue_Last(&rx_idleServerQueue, rx_serverQueueEntry);
 
        /* If hot threads are enabled, and both newcallp and sq->socketp
         * are non-null, then this thread will process the call, and the