How to deal with test data in ouf of source tree builds?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to deal with test data in ouf of source tree builds?

Paul Menzel
Dear Autoconf folks,


When doing an out of source build of Unbound, the test suite fails with
the error below.

```
test signature verify functions
testpkts error: could not open file testdata/test_signatures.1: No such
file or directory
Makefile:304: recipe for target 'test' failed
make: *** [test] Error 1
```

In the source directory there is a directory `testdata`, and that gets
referenced incorrectly in `testcode/unitverify.c`.

```
void
verify_test(void)
{
         unit_show_feature("signature verify");
#ifdef USE_SHA1
         verifytest_file("testdata/test_signatures.1", "20070818005004");
#endif
#if defined(USE_DSA) && defined(USE_SHA1)
         verifytest_file("testdata/test_signatures.2", "20080414005004");
         verifytest_file("testdata/test_signatures.3", "20080416005004");
[…]
```

Unfortunately, I couldn’t find any documentation how to deal with test
data when building out of the source tree. Should the test data be
copied to the build directory, despite it being not changed, should it
be linked, or does the code need to reference only absolute paths, and
therefore has to be preprocessed(?)?


Kind regards,

Paul

_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to deal with test data in ouf of source tree builds?

Nick Bowler-2
Hello Paul,

On 2017-06-15, Paul Menzel <[hidden email]> wrote:
> When doing an out of source build of Unbound, the test suite fails
[...]
> In the source directory there is a directory `testdata`, and that gets
> referenced incorrectly in `testcode/unitverify.c`.
[...]
>          verifytest_file("testdata/test_signatures.1", "20070818005004");
[...]
> Unfortunately, I couldn’t find any documentation how to deal with test
> data when building out of the source tree. Should the test data be
> copied to the build directory, despite it being not changed, should it
> be linked, or does the code need to reference only absolute paths, and
> therefore has to be preprocessed(?)?

There are many possible solutions.  The simplest is probably to cd into
the source directory before running the test programs.  But this may have
other undesired effects.

I suggest eliminating the dependency on the working directory completely.

For example, you can pass the location of necessary data files to the
test programs via environment variable(s) or command-line arguments.  If
all your data files are in one place this might be something simple like:

  UNBOUND_TEST_DATA=${srcdir}/testdata ./the_test_program

and the test program can construct filenames based on that environment
variable.

Cheers,
  Nick

_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to deal with test data in ouf of source tree builds?

Bob Friesenhahn
On Thu, 15 Jun 2017, Nick Bowler wrote:

>
> I suggest eliminating the dependency on the working directory completely.
>
> For example, you can pass the location of necessary data files to the
> test programs via environment variable(s) or command-line arguments.  If
> all your data files are in one place this might be something simple like:
>
>  UNBOUND_TEST_DATA=${srcdir}/testdata ./the_test_program
>
> and the test program can construct filenames based on that environment
> variable.

Environment variables are convenient.  Command line arguments to the
test program are also convenient.  If the test program is also the
installed program, take care that security or reliability issues are
not introduced due to responding to environment variables.

Microsoft Windows has very limited command line plus environment space
(come out of same allocation) so take care not to use so much
resources that it is not possible to run the tests under Windows (if
Windows is supported).

Bob
--
Bob Friesenhahn
[hidden email], http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to deal with test data in ouf of source tree builds?

Peter Johansson-6
In reply to this post by Paul Menzel
Hi Paul,


On 6/15/2017 10:06 PM, Paul Menzel wrote:

> Dear Autoconf folks,
>
>
> When doing an out of source build of Unbound, the test suite fails
> with the error below.
>
> ```
> test signature verify functions
> testpkts error: could not open file testdata/test_signatures.1: No
> such file or directory
> Makefile:304: recipe for target 'test' failed
> make: *** [test] Error 1
> ```
>
> In the source directory there is a directory `testdata`, and that gets
> referenced incorrectly in `testcode/unitverify.c`.
>

One solution I've used is to export ${abs_top_srcdir} to config.h and
then use that value as prefix in filenames.

Another solution is to create a link to directory testdata when $srcdir
!= $builddir. There is a macro AC_CONFIG_LNKS that might be useful.

Cheers,
Peter

_______________________________________________
Autoconf mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/autoconf
Loading...