hcrypto: Avoid 'double' param in arm64 kernel code

Currently, the RAND_add function in hcrypto uses a floating point
argument (specifically, a 'double'), as well as any implementations of
RAND_add. On Linux arm64, we cannot use floating point code in the
kernel, since the kernel module is compiled with -mgeneral-regs-only,
which prevents the use of floating point registers. No code in the
tree actually makes use of this argument, but its mere presence is
enough to cause an error with at least some versions of gcc with
certain arguments.

To get around this, simply change all instances of 'double' in hcrypto
to be a void pointer instead. This allows the code to compile as long
as nobody actually uses that argument in the kernel. If the code is
changed such that we do actually use that argument, the argument will
be a void* and so will probably (hopefully) cause a compiler error,
and the code will need to be examined to make sure this workaround
doesn't break anything.

We already do this on Solaris, which has similar issues for different
compiler versions and compiler flags. Add arm64 Linux to the cases
where we do this, but restrict this to kernel code only, to try to
avoid doing this more often than necessary.

Reviewed-on: https://gerrit.openafs.org/11939
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
(cherry picked from commit b792dea0f1f83673b0b045adf608412901b3024c)

Change-Id: I95a016ba4a3dc368680b512110eb0058fcfdb468
Reviewed-on: https://gerrit.openafs.org/13094
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

diff --git a/src/crypto/hcrypto/kernel/config.h b/src/crypto/hcrypto/kernel/config.h
index 81c2944d501fdfb81eec74fc2a1487be19937539..b6104348b3cfaadee7a42762c4f165a19623f731 100644 (file)
--- a/src/crypto/hcrypto/kernel/config.h
+++ b/src/crypto/hcrypto/kernel/config.h
@@ -98,7 +98,16 @@ static_inline int close(int d) {return -1;}
 static_inline int gettimeofday(struct timeval *tp, void *tzp)
     {if (tp == NULL) return -1; tp->tv_sec = osi_Time(); tp->tv_usec = 0; return 0;}
 
-#ifdef AFS_SUN5_ENV
-/* workaround to allow --disable-optimize-kernel on Solaris */
-#define double int
+#if defined(KERNEL) && (defined(AFS_SUN5_ENV) || defined(AFS_ARM64_LINUX26_ENV))
+/*
+ * Some functions such as RAND_add take a 'double' as an argument, but floating
+ * point code generally cannot be used in kernelspace. We never actually use
+ * that argument in kernel code, but just that it exists as an argument is
+ * enough to break the kernel code on Linux (on arm64) and Solaris (depending
+ * on the compiler version and flags). Change all instances of double to void*
+ * to avoid this; if someone does try to use that argument, hopefully the fact
+ * that it is now a void* will flag an error at compile time before it causes
+ * any further problems.
+ */
+# define double void*
 #endif