Avoid libtool 'nm' errors

Starting around Solaris 11.3, '/usr/bin/nm -p' starts reporting some
symbols with the 'C' code. libtool cannot handle this (libtool bug
 #22373), which causes global_symbol_pipe in the generated libtool
script to be empty. This causes a rather confusing error when we go to
actually use libtool to link something ("syntax error near unexpected
token '|'"; see libtool bug #20947), and prevents the build from
continuing.

Address this in two ways:

For all Solaris 11 builds, default to /usr/sfw/bin/gnm over
/usr/bin/nm. This avoids any interop issues with libtool and nm, since
libtool of course works very well with GNU tooling.

In addition, try to catch any nm-related errors with libtool at
configure time, to provide a more helpful error message.

To implement these changes, create a wrapper around LT_INIT, called
AFS_LT_INIT.

Reviewed-on: https://gerrit.openafs.org/12945
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>
(cherry picked from commit 3e9ea6107973ccc4fa3d405f5b5d76666bfd624f)

Change-Id: I4a5a358857ec5bfbc31cd99fcca59f3390ad4d16
Reviewed-on: https://gerrit.openafs.org/13066
Tested-by: BuildBot <buildbot@rampaginggeek.com>
Reviewed-by: Andrew Deason <adeason@sinenomine.net>
Reviewed-by: Benjamin Kaduk <kaduk@mit.edu>

diff --git a/configure.ac b/configure.ac
index 32092971f7cd4ba271cf6e1b7c7111406b315fc7..aabe8eef988f0874d8009a050017a46846e932ab 100644 (file)
--- a/configure.ac
+++ b/configure.ac
@@ -19,7 +19,7 @@ AS_IF([test -z "$CFLAGS"], [CFLAGS=" "])
 
 AC_USE_SYSTEM_EXTENSIONS
 
-LT_INIT
+AFS_LT_INIT
 
 AC_PROG_CC
 AC_PROG_LIBTOOL

diff --git a/src/cf/afs-libtool.m4 b/src/cf/afs-libtool.m4
new file mode 100644 (file)
index 0000000..63ec5c1
--- /dev/null
+++ b/src/cf/afs-libtool.m4
@@ -0,0 +1,30 @@
+dnl Change the default 'nm' in some situations
+AC_DEFUN([AFS_LT_PATH_NM],
+  [AC_REQUIRE([AC_CANONICAL_HOST])
+   AS_CASE([$host_os],
+
+dnl Starting around Solaris 11.3, the default 'nm' tool starts reporting
+dnl symbols with the 'C' code when given '-p'. Libtool currently cannot deal
+dnl with this, and it fails to figure out how to extract symbols from object
+dnl files (see libtool bug #22373). To work around this, try to use the GNU nm
+dnl instead by default, which is either always or almost always available.
+dnl libtool, of course, works with that just fine.
+     [solaris2.11*],
+     [AS_IF([test x"$NM" = x && test -x /usr/sfw/bin/gnm],
+       [NM=/usr/sfw/bin/gnm])])])
+
+dnl Our local wrapper around LT_INIT, to work around some bugs/limitations in
+dnl libtool.
+AC_DEFUN([AFS_LT_INIT],
+  [AC_REQUIRE([AFS_LT_PATH_NM])
+
+   LT_INIT
+
+dnl If libtool cannot figure out how to extract symbol names from 'nm', then it
+dnl will log a failure and lt_cv_sys_global_symbol_pipe will be unset, but it
+dnl will not cause configure to bail out. Later on, when we try to link with
+dnl libtool, it will cause a very confusing error message (see libtool bug
+dnl #20947).  To try and avoid that bad error message, try to catch that
+dnl situation here, closer to when the error actually occurs.
+   AS_IF([test x"$lt_cv_sys_global_symbol_pipe" = x],
+     [AC_MSG_ERROR([libtool cannot figure out how to extract symbol names; look above for failures involving 'nm'])])])