[PATCH v3 0/4] limit number of cat and sed forks

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[PATCH v3 0/4] limit number of cat and sed forks

Eric Blake-3
Work in progress, because I still have to review patches 2 and 3
from Paolo's v2. (Patch 3 can probably be used as-is; patch 2
will probably necessitate the same changes to CHECK_DECLS_ONCE for
honoring AC_LANG and reworking the shell list variable to hold
tuples as I do here for CHECK_FUNCS_ONCE).

The end result is the same number of processes shaved as in Paolo's
v2 patch 1/3, but now the code is no longer subject to subtle
semantic differences (the ONCE functions are still checked in the
same sequence in relation to the rest of configure as before), and
the configure script does not grow quite as large (on coreutils,
Paolo's cost about 17k, my approach cost about 2k).

Eric Blake (3):
  AC_CHECK_FUNCS_ONCE: honor current AC_LANG
  AC_CHECK_FUNCS_ONCE: hoist CPP name computation to m4 time
  AC_CHECK_FUNCS_ONCE: hoist cache name computation to m4 time

Paolo Bonzini (1):
  autoconf: prefer an unrolled loop for trivial AC_CHECK_FUNCS

 NEWS                      |  4 ++++
 lib/autoconf/functions.m4 | 60 +++++++++++++++++++++++++++++++++--------------
 2 files changed, 47 insertions(+), 17 deletions(-)

--
2.7.4


Reply | Threaded
Open this post in threaded view
|

[PATCH v3 1/4] AC_CHECK_FUNCS_ONCE: honor current AC_LANG

Eric Blake-3
Previously, AC_CHECK_FUNCS_ONCE collected a list of function names
to check, but ran the checks using the AC_LANG that was active
during the first encounter of the macro.  In practice, this is
usually the C language, and we haven't had actual reports of projects
attempting to use AC_CHECK_FUNCS_ONCE across multiple languages,
rather this was discovered by code inspection.

With this patch, the code now tracks a separate per-language list of
names to check.  Note, however, that it is only possible to check for
a given function name in one language; attempting to add a name again
under AC_CHECK_FUNCS_ONCE while a different language is active is a
no-op (this still makes sense because the side-effect of defining
the CPP macro HAVE_FUNC does not include a language prefix).

* lib/autoconf/functions.m4 (_AC_CHECK_FUNC_ONCE)
(_AC_FUNCS_EXPANSION):
* NEWS: Mention it.

Signed-off-by: Eric Blake <[hidden email]>
---
 NEWS                      |  4 ++++
 lib/autoconf/functions.m4 | 17 +++++++++--------
 2 files changed, 13 insertions(+), 8 deletions(-)

diff --git a/NEWS b/NEWS
index a4c53dd..532e15e 100644
--- a/NEWS
+++ b/NEWS
@@ -111,6 +111,10 @@ GNU Autoconf NEWS - User visible changes.
   useful effect is to trigger those checks, with this macro.  It is
   unlikely to be useful otherwise.

+- The AC_CHECK_FUNCS_ONCE macro now supports use with multiple
+  languages, rather than forcing all checks in the language used by
+  the first encounter of the macro.
+
 ** Man pages for config.guess and config.sub are no longer provided.
 They were moved to the master source tree for config.guess and config.sub.

diff --git a/lib/autoconf/functions.m4 b/lib/autoconf/functions.m4
index 66abe29..7b98a06 100644
--- a/lib/autoconf/functions.m4
+++ b/lib/autoconf/functions.m4
@@ -96,9 +96,9 @@ AC_DEFUN([AC_CHECK_FUNCS],
 # -----------------------------
 # Check for a single FUNCTION once.
 m4_define([_AC_CHECK_FUNC_ONCE],
-[_AH_CHECK_FUNC([$1])AC_DEFUN([_AC_Func_$1],
-  [m4_divert_text([INIT_PREPARE], [AS_VAR_APPEND([ac_func_list], [" $1"])])
-_AC_FUNCS_EXPANSION])AC_REQUIRE([_AC_Func_$1])])
+[_AH_CHECK_FUNC([$1])AC_DEFUN([_AC_Func_$1], [m4_divert_text([INIT_PREPARE],
+  [AS_VAR_APPEND([ac_func_]]_AC_LANG_ABBREV[[_list], [" $1"])])
+_AC_FUNCS_EXPANSION(_AC_LANG_ABBREV)])AC_REQUIRE([_AC_Func_$1])])

 # AC_CHECK_FUNCS_ONCE(FUNCTION...)
 # --------------------------------
@@ -107,12 +107,13 @@ _AC_FUNCS_EXPANSION])AC_REQUIRE([_AC_Func_$1])])
 AC_DEFUN([AC_CHECK_FUNCS_ONCE],
 [m4_map_args_w([$1], [_AC_CHECK_FUNC_ONCE(], [)])])

+# _AC_FUNCS_EXPANSION(LANG)
+# -------------------------
+# One-shot code per language LANG for checking all functions registered by
+# AC_CHECK_FUNCS_ONCE while that language was active.
 m4_define([_AC_FUNCS_EXPANSION],
-[
-  m4_divert_text([DEFAULTS], [ac_func_list=])
-  AC_CHECK_FUNCS([$ac_func_list])
-  m4_define([_AC_FUNCS_EXPANSION], [])
-])
+[m4_ifndef([$0($1)], [m4_define([$0($1)])m4_divert_text([DEFAULTS],
+[ac_func_$1_list=])AC_CHECK_FUNCS([$ac_func_$1_list])])])


 # _AC_REPLACE_FUNC(FUNCTION)
--
2.7.4


Reply | Threaded
Open this post in threaded view
|

[PATCH v3 2/4] AC_CHECK_FUNCS_ONCE: hoist CPP name computation to m4 time

Eric Blake-3
In reply to this post by Eric Blake-3
Rather than perform a sed script on each element of the
$ac_func_c_list to compute the corresponding CPP name, we can
make the list store a series of pairs of function names and
CPP names all computed at m4 time.

The resulting configure script is slightly larger based on
how many function names are checked once, but also performs
slightly faster.

There is still a sed call in AC_CHECK_FUNC for computing the
cache variable name; that will be dealt with next.

* lib/autoconf/functions.m4 (_AC_CHECK_FUNC_ONCE): Track the CPP
name in the list...
(_AC_FUNCS_EXPANSION): ...and rewrite the list walk to parse off
pairs of arguments, for fewer sed calls.

Signed-off-by: Eric Blake <[hidden email]>
---
 lib/autoconf/functions.m4 | 16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/lib/autoconf/functions.m4 b/lib/autoconf/functions.m4
index 7b98a06..b937b2c 100644
--- a/lib/autoconf/functions.m4
+++ b/lib/autoconf/functions.m4
@@ -97,8 +97,9 @@ AC_DEFUN([AC_CHECK_FUNCS],
 # Check for a single FUNCTION once.
 m4_define([_AC_CHECK_FUNC_ONCE],
 [_AH_CHECK_FUNC([$1])AC_DEFUN([_AC_Func_$1], [m4_divert_text([INIT_PREPARE],
-  [AS_VAR_APPEND([ac_func_]]_AC_LANG_ABBREV[[_list], [" $1"])])
-_AC_FUNCS_EXPANSION(_AC_LANG_ABBREV)])AC_REQUIRE([_AC_Func_$1])])
+  [AS_VAR_APPEND([ac_func_]]_AC_LANG_ABBREV[[_list],
+  [" $1 ]AS_TR_CPP([HAVE_$1])["])])]dnl
+[_AC_FUNCS_EXPANSION(_AC_LANG_ABBREV)])AC_REQUIRE([_AC_Func_$1])])

 # AC_CHECK_FUNCS_ONCE(FUNCTION...)
 # --------------------------------
@@ -113,7 +114,16 @@ AC_DEFUN([AC_CHECK_FUNCS_ONCE],
 # AC_CHECK_FUNCS_ONCE while that language was active.
 m4_define([_AC_FUNCS_EXPANSION],
 [m4_ifndef([$0($1)], [m4_define([$0($1)])m4_divert_text([DEFAULTS],
-[ac_func_$1_list=])AC_CHECK_FUNCS([$ac_func_$1_list])])])
+[ac_func_$1_list=])ac_func=
+for ac_item in $ac_func_$1_list
+do
+  if test $ac_func; then
+    AC_CHECK_FUNC([$ac_func], [AC_DEFINE_UNQUOTED([$ac_item])])
+    ac_func=
+  else
+    ac_func=$ac_item
+  fi
+done])])


 # _AC_REPLACE_FUNC(FUNCTION)
--
2.7.4


Reply | Threaded
Open this post in threaded view
|

[PATCH v3 3/4] AC_CHECK_FUNCS_ONCE: hoist cache name computation to m4 time

Eric Blake-3
In reply to this post by Eric Blake-3
Rather than perform a sed script on each element of the
$ac_func_c_list to compute the corresponding cache name, we
can inline enough of AC_CHECK_FUNC to bypass the normal
polymorphic code, and instead directly use the literal
function name that we are consuming from the list.

While at it, we can use echo instead of cat to append to
confdefs.h, for another process shaved.

The resulting configure script is roughly unchanged in size,
but performs slightly faster.

* lib/autoconf/functions.m4 (AC_CHECK_FUNC): Split out shell
function registration...
(_AC_CHECK_FUNC_FN): ...to here.
(_AC_FUNCS_EXPANSION): Use it to inline enough of AC_CHECK_FUNC to
operate on a literal rather than a shell variable, for fewer sed
calls.

Signed-off-by: Eric Blake <[hidden email]>
---
 lib/autoconf/functions.m4 | 24 +++++++++++++++++-------
 1 file changed, 17 insertions(+), 7 deletions(-)

diff --git a/lib/autoconf/functions.m4 b/lib/autoconf/functions.m4
index b937b2c..76824a2 100644
--- a/lib/autoconf/functions.m4
+++ b/lib/autoconf/functions.m4
@@ -51,17 +51,20 @@ m4_define([_AC_CHECK_FUNC_BODY],
 ])# _AC_CHECK_FUNC_BODY


+m4_define([_AC_CHECK_FUNC_FN],
+[AC_REQUIRE_SHELL_FN([ac_fn_]_AC_LANG_ABBREV[_check_func],
+  [AS_FUNCTION_DESCRIBE([ac_fn_]_AC_LANG_ABBREV[_check_func],
+    [LINENO FUNC VAR],
+    [Tests whether FUNC exists, setting the cache variable VAR accordingly])],
+  [_AC_CHECK_FUNC_BODY])])
+
 # AC_CHECK_FUNC(FUNCTION, [ACTION-IF-FOUND], [ACTION-IF-NOT-FOUND])
 # -----------------------------------------------------------------
 # Check whether FUNCTION links in the current language.  Set the cache
 # variable ac_cv_func_FUNCTION accordingly, then execute
 # ACTION-IF-FOUND or ACTION-IF-NOT-FOUND.
 AC_DEFUN([AC_CHECK_FUNC],
-[AC_REQUIRE_SHELL_FN([ac_fn_]_AC_LANG_ABBREV[_check_func],
-  [AS_FUNCTION_DESCRIBE([ac_fn_]_AC_LANG_ABBREV[_check_func],
-    [LINENO FUNC VAR],
-    [Tests whether FUNC exists, setting the cache variable VAR accordingly])],
-  [_$0_BODY])]dnl
+[_AC_CHECK_FUNC_FN()]dnl
 [AS_VAR_PUSHDEF([ac_var], [ac_cv_func_$1])]dnl
 [ac_fn_[]_AC_LANG_ABBREV[]_check_func "$LINENO" "$1" "ac_var"
 AS_VAR_IF([ac_var], [yes], [$2], [$3])
@@ -111,14 +114,21 @@ AC_DEFUN([AC_CHECK_FUNCS_ONCE],
 # _AC_FUNCS_EXPANSION(LANG)
 # -------------------------
 # One-shot code per language LANG for checking all functions registered by
-# AC_CHECK_FUNCS_ONCE while that language was active.
+# AC_CHECK_FUNCS_ONCE while that language was active.  We have to inline
+# portions of AC_CHECK_FUNC, because although we operate on shell
+# variables, we know they represent literals at that point in time,
+# where we don't want to trigger normal AS_VAR_PUSHDEF shell code.
 m4_define([_AC_FUNCS_EXPANSION],
 [m4_ifndef([$0($1)], [m4_define([$0($1)])m4_divert_text([DEFAULTS],
 [ac_func_$1_list=])ac_func=
 for ac_item in $ac_func_$1_list
 do
   if test $ac_func; then
-    AC_CHECK_FUNC([$ac_func], [AC_DEFINE_UNQUOTED([$ac_item])])
+    _AC_CHECK_FUNC_FN()ac_fn_$1_check_func "$LINENO" ]dnl
+[$ac_func ac_cv_func_$ac_func
+    if eval test \"x\$ac_cv_func_$ac_func\" = xyes; then
+      echo "[#]define $ac_item 1" >> confdefs.h
+    fi
     ac_func=
   else
     ac_func=$ac_item
--
2.7.4


Reply | Threaded
Open this post in threaded view
|

[PATCH v3 4/4] autoconf: prefer an unrolled loop for trivial AC_CHECK_FUNCS

Eric Blake-3
In reply to this post by Eric Blake-3
From: Paolo Bonzini <[hidden email]>

An unrolled loop avoids the cost of spawning sed in AS_TR_SH and
AS_TR_CPP.  Prefer it if there is nothing in the second and third
argument of AC_CHECK_FUNCS and the first argument is a literal.

* lib/autoconf/functions.m4 (AC_CHECK_FUNCS): Unroll loop if safe.
(_AC_CHECK_FUNCS): Move basic implementation here.

Signed-off-by: Paolo Bonzini <[hidden email]>
Message-Id: <[hidden email]>
[eblake: perform AC_CHECK_FUNCS_ONCE changes separately, use
dnl to reduce generated blank lines]
Signed-off-by: Eric Blake <[hidden email]>
---
 lib/autoconf/functions.m4 | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/lib/autoconf/functions.m4 b/lib/autoconf/functions.m4
index 76824a2..a227cd4 100644
--- a/lib/autoconf/functions.m4
+++ b/lib/autoconf/functions.m4
@@ -88,12 +88,17 @@ m4_define([_AH_CHECK_FUNC],
 # `break' to stop the search.
 AC_DEFUN([AC_CHECK_FUNCS],
 [m4_map_args_w([$1], [_AH_CHECK_FUNC(], [)])]dnl
-[AS_FOR([AC_func], [ac_func], [$1],
-[AC_CHECK_FUNC(AC_func,
-       [AC_DEFINE_UNQUOTED(AS_TR_CPP([HAVE_]AC_func)) $2],
-       [$3])dnl])
+[m4_if([$2$3]AS_LITERAL_IF([$1], [[yes]], [[no]]), [yes],
+       [m4_map_args_w([$1], [_$0(], [)])],
+       [AS_FOR([AC_func], [ac_func], [$1], [_$0(AC_func, [$2], [$3])])])dnl
 ])# AC_CHECK_FUNCS

+m4_define([_AC_CHECK_FUNCS],
+[AC_CHECK_FUNC([$1],
+       [AC_DEFINE_UNQUOTED(AS_TR_CPP([HAVE_]$1)) $2],
+       [$3])dnl
+])
+

 # _AC_CHECK_FUNC_ONCE(FUNCTION)
 # -----------------------------
--
2.7.4


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v3 0/4] limit number of cat and sed forks

Paolo Bonzini-2
In reply to this post by Eric Blake-3


On 04/11/2016 03:37, Eric Blake wrote:

> Work in progress, because I still have to review patches 2 and 3
> from Paolo's v2. (Patch 3 can probably be used as-is; patch 2
> will probably necessitate the same changes to CHECK_DECLS_ONCE for
> honoring AC_LANG and reworking the shell list variable to hold
> tuples as I do here for CHECK_FUNCS_ONCE).
>
> The end result is the same number of processes shaved as in Paolo's
> v2 patch 1/3, but now the code is no longer subject to subtle
> semantic differences (the ONCE functions are still checked in the
> same sequence in relation to the rest of configure as before), and
> the configure script does not grow quite as large (on coreutils,
> Paolo's cost about 17k, my approach cost about 2k).
>
> Eric Blake (3):
>   AC_CHECK_FUNCS_ONCE: honor current AC_LANG
>   AC_CHECK_FUNCS_ONCE: hoist CPP name computation to m4 time
>   AC_CHECK_FUNCS_ONCE: hoist cache name computation to m4 time
>
> Paolo Bonzini (1):
>   autoconf: prefer an unrolled loop for trivial AC_CHECK_FUNCS
>
>  NEWS                      |  4 ++++
>  lib/autoconf/functions.m4 | 60 +++++++++++++++++++++++++++++++++--------------
>  2 files changed, 47 insertions(+), 17 deletions(-)
>


Looks good, thanks!

Paolo

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v3 0/4] limit number of cat and sed forks

Eric Blake-3
On 11/05/2016 04:16 AM, Paolo Bonzini wrote:

>
>
> On 04/11/2016 03:37, Eric Blake wrote:
>> Work in progress, because I still have to review patches 2 and 3
>> from Paolo's v2. (Patch 3 can probably be used as-is; patch 2
>> will probably necessitate the same changes to CHECK_DECLS_ONCE for
>> honoring AC_LANG and reworking the shell list variable to hold
>> tuples as I do here for CHECK_FUNCS_ONCE).
>>
>> The end result is the same number of processes shaved as in Paolo's
>> v2 patch 1/3, but now the code is no longer subject to subtle
>> semantic differences (the ONCE functions are still checked in the
>> same sequence in relation to the rest of configure as before), and
>> the configure script does not grow quite as large (on coreutils,
>> Paolo's cost about 17k, my approach cost about 2k).
>>
>> Eric Blake (3):
>>   AC_CHECK_FUNCS_ONCE: honor current AC_LANG
>>   AC_CHECK_FUNCS_ONCE: hoist CPP name computation to m4 time
>>   AC_CHECK_FUNCS_ONCE: hoist cache name computation to m4 time
>>
>> Paolo Bonzini (1):
>>   autoconf: prefer an unrolled loop for trivial AC_CHECK_FUNCS
>>
>>  NEWS                      |  4 ++++
>>  lib/autoconf/functions.m4 | 60 +++++++++++++++++++++++++++++++++--------------
>>  2 files changed, 47 insertions(+), 17 deletions(-)
>>
>
>
> Looks good, thanks!
Apologies for sitting on this for over a month, but I finally had time
to finish the series, and basically duplicated the AC_CHECK_FUNCS*
patches into corresponding AC_CHECK_HEADERS* versions, and pushed the
result.

Thanks again for your initial investigation on how to reduce some of the
fork expense of configure files.

--
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


signature.asc (617 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v3 0/4] limit number of cat and sed forks

Paolo Bonzini-2
In reply to this post by Paolo Bonzini-2
On 05/11/2016 10:16, Paolo Bonzini wrote:
>> Work in progress, because I still have to review patches 2 and 3
>> from Paolo's v2. (Patch 3 can probably be used as-is; patch 2
>> will probably necessitate the same changes to CHECK_DECLS_ONCE for
>> honoring AC_LANG and reworking the shell list variable to hold
>> tuples as I do here for CHECK_FUNCS_ONCE).

Any news about patch 3 at least? :)

Paolo

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH v3 0/4] limit number of cat and sed forks

Paolo Bonzini-2
On 15/08/2017 02:22, Paolo Bonzini wrote:
> On 05/11/2016 10:16, Paolo Bonzini wrote:
>>> Work in progress, because I still have to review patches 2 and 3
>>> from Paolo's v2. (Patch 3 can probably be used as-is; patch 2
>>> will probably necessitate the same changes to CHECK_DECLS_ONCE for
>>> honoring AC_LANG and reworking the shell list variable to hold
>>> tuples as I do here for CHECK_FUNCS_ONCE).
>
> Any news about patch 3 at least? :)

Doh, you had committed that one too. :)  Thanks

Paolo