meaning autotest exit code 13

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

meaning autotest exit code 13

djien

I generate test.at to a testsuite script using autom4te (v.2.68) and execute this test. It fails with a return code 13.
Please inform me whether there is a list of return-codes which explains this error code.

Test is simple: it compares output Hello, world

Thank you for your effort


t.at
#AT_PACKAGE_STRING
#AT_PACKAGE_BUGREPORT
m4_define([AT_PACKAGE_STRING],    [hello])
m4_define([AT_PACKAGE_BUGREPORT], [gnuprog2-devel@sourceforge.org])
AT_INIT([hello])
AT_SETUP([hello])
AT_TESTED(hello)
AT_CHECK([hello],['Hello, world'])
AT_CLEANUP

t.log
## ---------------------- ##
## Detailed failed tests. ##
## ---------------------- ##

#                             -*- compilation -*-
1. testsuite.at:6: testing hello ...
++ set +x
/testsuite.at:8: hello
++ hello
--- /dev/null 2013-03-05 22:33:26.850202646 +0100
+++ /media/wd40/TODOs/work/work_20130306/4a_autotest/t.dir/at-groups/1/stdout 2013-03-06 09:36:49.099438123 +0100
@@ -0,0 +1 @@
+Hello, world
/testsuite.at:8: exit code was 13, expected Hello, world
1. testsuite.at:6: 1. hello (testsuite.at:6): FAILED (testsuite.at:8)
Reply | Threaded
Open this post in threaded view
|

Re: meaning autotest exit code 13

Eric Blake-3
On 03/06/2013 02:29 AM, djien wrote:

>
> I generate test.at to a testsuite script using autom4te (v.2.68) and execute
> this test. It fails with a return code 13.
> Please inform me whether there is a list of return-codes which explains this
> error code.
>
> Test is simple: it compares output Hello, world
>
> Thank you for your effort
>
>
> t.at
> #AT_PACKAGE_STRING
> #AT_PACKAGE_BUGREPORT
> m4_define([AT_PACKAGE_STRING],    [hello])
> m4_define([AT_PACKAGE_BUGREPORT], [[hidden email]])
> AT_INIT([hello])
> AT_SETUP([hello])
> AT_TESTED(hello)
> AT_CHECK([hello],['Hello, world'])
This is a usage error.  This line says to execute 'hello', and expect $?
(which will always be numeric) to contain the string 'Hello, world'.
Perhaps we can enhance the AT_CHECK macro to sanity check the second
argument (omitted, literal 'ignore', a numeric value, or a shell
substitution); but remember that it is impossible to sanity check that
all shell substitutions would eventually resolve into a numeric value.

> @@ -0,0 +1 @@
> +Hello, world
> /testsuite.at:8: exit code was 13, expected Hello, world
> 1. testsuite.at:6: 1. hello (testsuite.at:6): FAILED (testsuite.at:8)

Meanwhile, this says that when the testsuite executed 'hello', the exit
status was 13.  But autotest did not write the 'hello' program, so
that's something your package presumably provided.  The testsuite itself
exited with status 1 (a test failed), not status 13.  I have no idea
what hello's status 13 means.

I suspect you meant to test that 'hello' prints 'Hello, world\n' to
stdout, and exits with status 0; if so, write:

AT_CHECK([hello], [], [[Hello, world
]])

Or, if it is normal for 'hello' to have non-zero status, change the
second argument to [13] to match an exact status, or to [ignore] to pass
the test in spite of the exit status.

--
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf

signature.asc (634 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: meaning autotest exit code 13

djien
Thank you Eric for the explanation. Let me explain what i have done:

step 1:  the hello.c code
include <stdio.h>

main( int argc, char *argv[] )
{
    printf( "Hello, world\n" );
}

step2: compile gcc hello.c -o hello

step3: t17.at
# Process with autom4te to create an -*- Autotest -*- test suite.

m4_define([AT_PACKAGE_STRING],    [GNU Programming])
m4_define([AT_PACKAGE_BUGREPORT], [gnuprog2-devel@sourceforge.org])

# AT_PACKAGE_STRING([Hello],[v.1.0])
# AT_PACKAGE_BUGREPORT([autotest@gnu.org])
AT_INIT([hello])
AT_SETUP([hello])
AT_TESTED([hello])
AT_CHECK([hello],[],[[Hello, world
]])
AT_CLEANUP()

step 4: execute hello
$ ./hello
Hello, world
$

step 5:
 autom4te -l autotest t17.at -o  t17_hello

step 6: t17_hello script
 update PATH with the current working dir

## ------------- ##
## Actual tests. ##
## ------------- ##
#AT_START_1
..
{ set +x
$as_echo "$at_srcdir/t17.at:11: hello"
at_fn_check_prepare_trace "t17.at:11"
( $at_check_trace; hello
) >>"$at_stdout" 2>>"$at_stderr"
at_status=$? at_failed=false
$at_check_filter
at_fn_diff_devnull "$at_stderr" || at_failed=:
echo >>"$at_stdout"; $as_echo "Hello, world
" | \
  $at_diff - "$at_stdout" || at_failed=:
at_fn_check_status 0 $at_status "$at_srcdir/t17.at:11"
$at_failed && at_fn_log_failure
$at_traceon; }

..
#AT_STOP_1

step 7: $ ./t17_hello -x -v

step 8: t17_hello.log

#                             -*- compilation -*-
1. t17.at:9: testing hello ...
++ set +x
/t17.at:11: hello
++ hello
/t17.at:11: exit code was 13, expected 0
1. t17.at:9: 1. hello (t17.at:9): FAILED (t17.at:11)

step 9: modift t17.at
AT_CHECK([hello],[],[["Hello, world
"]])

step 10: and do the same steps as above:
t17_hello script:

echo >>"$at_stdout"; $as_echo "\"Hello, world
\"" | \

step 10: t17_hello.log
1. t18.at:9: testing hello ...
++ set +x
/t18.at:11: hello
++ hello
--- - 2013-03-07 14:01:44.798854464 +0100
+++ /media/wd40/TODOs/work/work_20130306/autotest/trial2b/t18_hello.dir/at-groups/1/stdout2013-03-07 14:01:44.791793357 +0100
@@ -1,2 +1,2 @@
-"Hello, world
-"
+Hello, world
+
/t18.at:11: exit code was 13, expected 0
1. t18.at:9:  FAILED (t18.at:11)

Please inform me where the error is.
Thank you

Reply | Threaded
Open this post in threaded view
|

Re: meaning autotest exit code 13

Eric Blake-3
On 03/07/2013 06:04 AM, djien wrote:
> Thank you Eric for the explanation. Let me explain what i have done:
>
> step 1:  the hello.c code
> include <stdio.h>
>
> main( int argc, char *argv[] )
> {
>     printf( "Hello, world\n" );
> }

Where's your return statement?  You forgot one, so your C compiler
assumed that you wanted the return value of printf() to become your
return value of your program.  You printed 13 bytes, so printf()
returned 13, hence, your 'hello' exits with status 13, with your choice
of compilers (not all compilers would do the same, though, because your
program uses unspecified behavior).  Fix your C program to avoid
unspecified behavior if you want a reliable exit status from 'hello'.


> /t17.at:11: hello
> ++ hello
> /t17.at:11: exit code was 13, expected 0
> 1. t17.at:9: 1. hello (t17.at:9): FAILED (t17.at:11)

Your testsuite is telling you that 'hello' failed to meet the
expectations you had in the testsuite of it exiting with status 0.

>
> step 9: modift t17.at
> AT_CHECK([hello],[],[["Hello, world
> "]])

This modification says that you want your 'hello' program to output
double quotes as part of its literal output - but that's not what your C
program did.

> @@ -1,2 +1,2 @@
> -"Hello, world
> -"
> +Hello, world
> +

So now the testsuite is telling you that your 'hello' program produced
different output than your expectation,

> /t18.at:11: exit code was 13, expected 0
> 1. t18.at:9:  FAILED (t18.at:11)

in addition to still having the wrong exit status for your expectation.

>
> Please inform me where the error is.

In your .c file, or in your testsuite expectations.

--
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org


_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf

signature.asc (634 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: meaning autotest exit code 13

djien
Thank you Eric. Return code in the sourcecode: main() is mandatory for this example.