SOLARIS: prevent BAD TRAP panic with Studio 12.5

Starting with Solaris Studio 12.3, it is documented that Solaris kernel
modules (such as libafs) must not use any floating point, vector, or
SIMD/SSE instructions on x86 hardware.  However, each new Studio
compiler release (12.4 and especially 12.5) is more likely to use these
types of instructions by default.

If the libafs kernel module includes any forbidden kernel instructions,
Solaris will panic the system with:
  BAD TRAP:  type=7 (#nm Device not available)

Provide a new autoconfig test to specify the required compiler options
(-xvector=%none -xregs=no%float) when building the OpenAFS kernel module
for Solaris, so that no invalid x86 instructions are used.

In addition, reinstate default kernel module optimization for Solaris.
It had been disabled in commit 80592c53cbb0bce782eb39a5e64860786654be9f
to address this same issue in Studio 12.3 and 12.4.  However, Studio
12.5 started using some SSE instructions even with no optimization.

This commit has been tested with OpenAFS master and Studio 12.5 at all
optimization levels (none, -xO1 through -xO5) and verified to contain no
XMM register instructions via the following command:
  $ gobjdump -dlr libafs64.o | grep xmm | wc -l

[wiesand: limit change to solaris 5.11 for stable branch]

Reviewed-on: https://gerrit.openafs.org/12558
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
(cherry picked from commit 22d841a45fff7026318b529a41dd957ce8bb0ddf)

Change-Id: I2e87f26dbac47289694346639b396dfc556368f4
Reviewed-on: https://gerrit.openafs.org/12567
Reviewed-by: Mark Vitale <mvitale@sinenomine.net>
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
Reviewed-by: Stephan Wiesand <stephan.wiesand@desy.de>

diff --git a/acinclude.m4 b/acinclude.m4
index 4b521c434a2a33a20c5231411b71cbb95530b521..53ffcc567d6a362d2f28b38fffae1870b1451695 100644 (file)
--- a/acinclude.m4
+++ b/acinclude.m4
@@ -387,6 +387,8 @@ case $system in
                MKAFS_OSTYPE=SOLARIS
                 AC_MSG_RESULT(sun4)
                SOLARIS_PATH_CC
+               SOLARIS_CC_TAKES_XVECTOR_NONE
+               AC_SUBST(SOLARIS_CC_KOPTS)
                SOLARIS_UFSVFS_HAS_DQRWLOCK
                SOLARIS_FS_HAS_FS_ROLLED
                SOLARIS_SOLOOKUP_TAKES_SOCKPARAMS

diff --git a/src/cf/osconf.m4 b/src/cf/osconf.m4
index 964e4742f4bcc5406b6be39ac99ab2ae365cd024..2fee044fd2c723e6bac0b8e7f33fb3990ff08be6 100644 (file)
--- a/src/cf/osconf.m4
+++ b/src/cf/osconf.m4
@@ -716,6 +716,7 @@ case $AFS_SYSNAME in
                MT_CC=$SOLARISCC
                MT_CFLAGS='-mt -DAFS_PTHREAD_ENV ${XCFLAGS}'
                MT_LIBS="-lpthread -lsocket"
+               KERN_OPTMZ="-xO3"
                PAM_CFLAGS="-KPIC"
                PAM_LIBS="-lc -lpam -lsocket -lnsl -lm"
                SHLIB_CFLAGS="-KPIC"
@@ -736,10 +737,12 @@ if test "x$enable_optimize_kernel" = "x" ; then
   AS_CASE([$AFS_SYSNAME],
     [sunx86_510|sunx86_511],
       dnl Somewhere around Solaris Studio 12.*, the compiler started adding SSE
-      dnl instructions to optimized code, without any ability to turn it off.
-      dnl So just default to not optimizing kernel code for the relevant
-      dnl platforms, until we get a better autoconf test for this.
-      [enable_optimize_kernel=no],
+      dnl instructions to optimized code, without any known way to turn it off.
+      dnl To cope, this condition was added to change the default to
+      dnl 'no'.
+      dnl Now that we have an autoconf test to allow disabling the SSE
+      dnl optimizations, it's safe to once more default to 'yes' here.
+      [enable_optimize_kernel=yes],
     [enable_optimize_kernel=yes])
 fi
 

diff --git a/src/cf/solaris-test1.m4 b/src/cf/solaris-test1.m4
new file mode 100644 (file)
index 0000000..a0b572e
--- /dev/null
+++ b/src/cf/solaris-test1.m4
@@ -0,0 +1,38 @@
+dnl This test is for the Solaris x86 kernel module
+dnl build.  They prevent newer Solaris compilers (12.3
+dnl and up) from using XMM registers (SIMD instructions)
+dnl and floating point registers, which are invalid in
+dnl Solaris kernel code.
+dnl Without this, Solaris may panic in libafs with:
+dnl   BAD TRAP: type=7 (#nm Device not available)
+dnl
+dnl
+AC_DEFUN([SOLARIS_CC_TAKES_XVECTOR_NONE], [
+  AC_CACHE_CHECK([if $CC accepts -xvector=%none],
+    [ac_cv_solaris_cc_takes_xvector_none],
+    [save_CFLAGS="$CFLAGS"
+     CFLAGS="$CFLAGS -xvector=%none"
+     AC_TRY_COMPILE([],
+                   [int x;],
+                   [ac_cv_solaris_cc_takes_xvector_none=yes],
+                   [ac_cv_solaris_cc_takes_xvector_none=no])
+     CFLAGS="$save_CFLAGS"
+    ])
+
+dnl -xvector=%none first appeared in Studio 11, but has only been
+dnl documented as required for Solaris x86 kernel code since Studio
+dnl 12.3.  Studio 12.3 is when the compiler started making more
+dnl aggressive optimizations by using SSE/SIMD instructions with XMM
+dnl (floating point/ SIMD) registers.  Although -xvector=%none is
+dnl required to prevent these optimizations, it is not sufficient.
+dnl Experiments have shown that -xregs=no%float is also needed to
+dnl 1) eliminate a few optimizations not squelched by -xvector=%none,
+dnl and 2) prevent actual use of floating point types in the kernel
+dnl module.  -xregs=no%float has been present since before Studio 8, so
+dnl it is safe to assume its presence when -xvector=%none is present.
+dnl
+
+  AS_IF([test "$ac_cv_solaris_cc_takes_xvector_none" = "yes"],
+        [SOLARIS_CC_KOPTS="-xvector=%none -xregs=no%float "])
+])
+

diff --git a/src/libafs/MakefileProto.SOLARIS.in b/src/libafs/MakefileProto.SOLARIS.in
index 121f77d9d32d0ff2fa01be4eb7f992886f8b9ac9..7432285d4559b21f10632a2ebd849d9be8d697e0 100644 (file)
--- a/src/libafs/MakefileProto.SOLARIS.in
+++ b/src/libafs/MakefileProto.SOLARIS.in
@@ -50,7 +50,7 @@ KDEFS_32 =
 KDEFS_64 = -xarch=amd64 -xmodel=kernel
 
 <all>
-CFLAGS=-I. -I.. -I${TOP_OBJDIR}/src/config ${FSINCLUDES} $(DEFINES) $(KDEFS) $(KOPTS) ${DBUG}
+CFLAGS=-I. -I.. -I${TOP_OBJDIR}/src/config ${FSINCLUDES} $(DEFINES) $(KDEFS) @SOLARIS_CC_KOPTS@ ${DBUG}
 
 <sun4x_58 sunx86_58 sun4x_59 sunx86_59>
 LDFLAGS=-r -dy -N drv/ip -N drv/udp -N strmod/rpcmod