Linux: Restrict # of cbrs we allocate at once

With commit a309e274632993c5aeec04c6e090f5ac95837a40, we changed the
number of CBRs that we allocate in a chunk from 300 to 1024. However,
this change takes the amount of memory requried to allocate a chunk
of CBRs above PAGE_SIZE on Linux. This changes the allocator that we
use from kmalloc to vmalloc. Whilst we can, and do, prevent kmalloc
from flushing filesystem pages when we invoke it, we don't have a
similar level of control over vmalloc. 1.6 only: limit it back to 300
elsewhere.

In one reported case, clients deadlock whilst attempting to allocate
this memory, in a call stack that looks something like:

  afs_Daemon -> afs_ShakeLooseVCaches -> osi_TryEvictVCache
      -> afs_FlushVCache -> afs_QueueVCB -> afs_AllocCBR
      -> osi_linux_alloc -> ... -> __vmalloc_node -> ...
      -> try_to_free_pages

This is probably because we end up deadlocking in the writeback
invoked by try_to_free_pages, likely due to locks which are held
by ShakeLooseVCaches.

As a quick fix for the problem, make sure that the memory we
allocate always fits within a page, so we always use kmalloc for
it.

FIXES 129751

Reviewed-on: http://gerrit.openafs.org/4510
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Jeffrey Altman <jaltman@openafs.org>
Reviewed-by: Derrick Brashear <shadow@dementia.org>
(cherry picked from commit 3cb97cdaa867b0d9fef84c683251a0a8f59a1519)

Change-Id: I28a480788271eac1b69fb9704bed5c286be2c4c5
Reviewed-on: http://gerrit.openafs.org/4555
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Derrick Brashear <shadow@dementia.org>

diff --git a/src/afs/afs.h b/src/afs/afs.h
index f83c8d2d3cb469d7ecaa88038fbf4e409bee1a3f..6e6881b5acd722020fe4d9498aaa9540bb92bf7a 100644 (file)
--- a/src/afs/afs.h
+++ b/src/afs/afs.h
@@ -85,7 +85,9 @@ extern int afs_shuttingdown;
 #define MAXNUMSYSNAMES 32      /* max that current constants allow */
 #define        NOTOKTIMEOUT    (2*3600)        /* time after which to timeout conns sans tokens */
 #define        NOPAG           0xffffffff
-#define AFS_NCBRS      1024    /* max # of call back return entries */
+
+
+
 #define AFS_MAXCBRSCALL        32      /* max to return in a given call (must be <= AFSCBMAX) */
 #define        AFS_SALLOC_LOW_WATER    250     /* Min free blocks before allocating more */
 #define        AFS_LRALLOCSIZ  4096    /* "Large" allocated size */
@@ -261,6 +263,16 @@ struct afs_cbr {
     unsigned int dynalloc:1;
 };
 
+#ifdef AFS_LINUX22_ENV
+/* On Linux, we have to be able to allocate the storage for this using
+ * kmalloc, as otherwise we may deadlock. So, it needs to be able to fit
+ * in a single page
+ */
+# define AFS_NCBRS     PAGE_SIZE/sizeof(struct afs_cbr)
+#else
+# define AFS_NCBRS     300     /* max # of call back return entries */
+#endif
+
 /* cellinfo file magic number */
 #define AFS_CELLINFO_MAGIC     0xf32817cd