From patchwork Mon Dec 12 13:50:08 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Weimer X-Patchwork-Id: 87690 Delivered-To: patch@linaro.org Received: by 10.140.20.101 with SMTP id 92csp1648941qgi; Mon, 12 Dec 2016 05:50:39 -0800 (PST) X-Received: by 10.98.10.198 with SMTP id 67mr98983158pfk.157.1481550639672; Mon, 12 Dec 2016 05:50:39 -0800 (PST) Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id 30si43473563plb.330.2016.12.12.05.50.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Dec 2016 05:50:39 -0800 (PST) Received-SPF: pass (google.com: domain of libc-alpha-return-75769-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@sourceware.org; spf=pass (google.com: domain of libc-alpha-return-75769-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=libc-alpha-return-75769-patch=linaro.org@sourceware.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type; q=dns; s=default; b=vHTGj5w2hqfXKWe7uU6kbFY/K6CIl IEYuc6L0SoMxLhViiq20IuO4tAMpgl6y4dzkIt9esZvDuk1H7aaf/q4fC15zs+P3 x5EmdvKEZZxJZ9CIaHlkCECAT5VvXrJm+rNY8Emj/wVK8yQM8C+AH+kTHr/W5w6q hNR5OVkopgbUpc= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:to:from:subject:message-id:date:mime-version :content-type; s=default; bh=B+7owYXL5t6jlcfXGViS9n2Roaw=; b=gAY 1m/2dqRjrLTaHaLSAFNW4yv7ReEqSnYf0kflAVZ3Ywq5GhtuWROY/pZ1FJ4dSxOI s+FSyYJ6VcAwWkLs+2VTG6Y7BXCDD7FOMCecYssQzoLfw9xmBVEvJ1pXndVulwJr 2kJc5WAYWB2NGnqT6x+ID0jkaz+Qn0iTI28SVoms= Received: (qmail 87046 invoked by alias); 12 Dec 2016 13:50:23 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 86993 invoked by uid 89); 12 Dec 2016 13:50:23 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-5.0 required=5.0 tests=BAYES_00, RP_MATCHES_RCVD, SPF_HELO_PASS autolearn=ham version=3.3.2 spammy=Functionality, recording, _exit, MAP_ANONYMOUS X-HELO: mx1.redhat.com To: GNU C Library , Torvald Riegel From: Florian Weimer Subject: [PATCH] support: Add support for delayed test failure reporting Message-ID: <372cbbc7-2396-12af-49e8-8bca20cae4ae@redhat.com> Date: Mon, 12 Dec 2016 14:50:08 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 I need this for the container and resolver tests, which may end up detecting errors in code which does not run on the main thread/process. A MAP_SHARED mapping is used to share the test status across the entire process group spawned from the test driver. Torvald, do the concurrency bits in support/support_record_failure.c look okay to you? Thanks, Florian support: Add support for delayed test failure reporting 2016-12-12 Florian Weimer * support/Makefile (libsupport-routines): Add support_record_failure, xfork, xwaitpid. (tests): Add tst-support_record_failure. (tests-special): tst-support_record_failure-2. (tst-support_record_failure-2.out): Depend on tst-support_record_failure-2.sh and tst-support_record_failure. * support/check.h (support_record_failure, support_report_failure) (support_report_failure_reset): Declare. * support/support_test_main.c (adjust_exit_status): New function. (support_test_main): Call it to incorporate record test failures. * support/support_record_failure.c: New file. * support/tst-support_record_failure.c: Likewise. * support/tst-support_record_failure-2.sh: Likewise. * support/xunistd.h: Likewise. * support/xfork.c: Likewise. * support/xwaitpid.c: Likewise. diff --git a/support/Makefile b/support/Makefile index bd425af..4821147 100644 --- a/support/Makefile +++ b/support/Makefile @@ -30,11 +30,13 @@ libsupport-routines = \ ignore_stderr \ oom_error \ set_fortify_handler \ + support_record_failure \ support_test_main \ temp_file \ write_message \ xasprintf \ xcalloc \ + xfork \ xmalloc \ xpthread_barrier_destroy \ xpthread_barrier_init \ @@ -51,6 +53,7 @@ libsupport-routines = \ xpthread_spin_lock \ xpthread_spin_unlock \ xrealloc \ + xwaitpid \ libsupport-static-only-routines := $(libsupport-routines) # Only build one variant of the library. @@ -59,6 +62,18 @@ ifeq ($(build-shared),yes) libsupport-inhibit-o += .o endif -tests = README-testing +tests = \ + README-testing \ + tst-support_record_failure \ + +tests-special = \ + $(objpfx)tst-support_record_failure-2.out + +$(objpfx)tst-support_record_failure-2.out: tst-support_record_failure-2.sh \ + $(objpfx)tst-support_record_failure + $(SHELL) $< $(common-objpfx) '$(test-program-prefix-before-env)' \ + '$(run-program-env)' '$(test-program-prefix-after-env)' \ + > $@; \ + $(evaluate-test) include ../Rules diff --git a/support/check.h b/support/check.h index ff2652c..3a01d2b 100644 --- a/support/check.h +++ b/support/check.h @@ -1,4 +1,4 @@ -/* Macros for reporting test results. +/* Functionality for reporting test results. Copyright (C) 2016 Free Software Foundation, Inc. This file is part of the GNU C Library. @@ -43,6 +43,20 @@ void support_exit_failure_impl (int exit_status, const char *format, ...) __attribute__ ((noreturn, nonnull (2), format (printf, 4, 5))); +/* Record a test failure. This function returns and does not + terminate the process. The failure counter is stored in a shared + memory mapping, so that failures reported in child processes are + visible to the parent process and test driver. This function + depends on initialization by an ELF constructor, so it can only be + invoked after the test driver has run. */ +void support_record_failure (void); + +/* Internal function called by the test driver. */ +int support_report_failure (int status) + __attribute__ ((weak, warn_unused_result)); + +/* Internal function used to test the failure recording framework. */ +void support_record_failure_reset (void); __END_DECLS diff --git a/support/support_record_failure.c b/support/support_record_failure.c new file mode 100644 index 0000000..dc8303f --- /dev/null +++ b/support/support_record_failure.c @@ -0,0 +1,104 @@ +/* Global test failure counter. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include + +#include +#include +#include +#include +#include + +/* This structure keeps track of test failures. The counter is + incremented on each failure. The failed member is set to true if a + failure is detected, so that even if the counter wraps around to + zero, the failure of a test can be detected. + + The init constructor function below puts *state on a shared + annonymous mapping, so that failure reports from subprocesses + propagate to the parent process. */ +struct test_failures +{ + unsigned counter; + unsigned failed; +}; +static struct test_failures *state; + +static __attribute__ ((constructor)) void +init (void) +{ + void *ptr = mmap (NULL, sizeof (*state), PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_SHARED, -1, 0); + if (ptr == MAP_FAILED) + { + printf ("error: could not map %zu bytes: %m\n", sizeof (*state)); + exit (1); + } + /* Zero-initialization of the struct is sufficient. */ + state = ptr; +} + +void +support_record_failure (void) +{ + if (state == NULL) + { + write_message + ("error: support_record_failure called without initialization\n"); + _exit (1); + } + /* The release MO store synchronize with the acquire MO load in + support_report_failure. */ + __atomic_store_n (&state->failed, 1, __ATOMIC_RELEASE); + __atomic_add_fetch (&state->counter, 1, __ATOMIC_RELEASE); +} + +int +support_report_failure (int status) +{ + if (state == NULL) + { + write_message + ("error: support_report_failure called without initialization\n"); + return 1; + } + + /* The acquire MO load synchronize with the release MO store in + support_record_failure. */ + bool failed = __atomic_load_n (&state->failed, __ATOMIC_ACQUIRE); + if (failed) + printf ("error: %u test failures\n", + __atomic_load_n (&state->counter, __ATOMIC_ACQUIRE)); + + if ((status == 0 || status == EXIT_UNSUPPORTED) && failed) + /* If we have a recorded failure, it overrides a non-failure + report from the test function. */ + status = 1; + return status; +} + +void +support_record_failure_reset (void) +{ + /* Only used for testing the test framework, with external + synchronization, but use release MO for consistency. */ + __atomic_store_n (&state->failed, 0, __ATOMIC_RELEASE); + __atomic_add_fetch (&state->counter, 0, __ATOMIC_RELEASE); +} diff --git a/support/support_test_main.c b/support/support_test_main.c index e185281..96828a0 100644 --- a/support/support_test_main.c +++ b/support/support_test_main.c @@ -17,6 +17,7 @@ . */ #include +#include #include #include @@ -164,6 +165,17 @@ static bool test_main_called; const char *test_dir = NULL; + +/* If test failure reporting has been linked in, it may contribute + additional test failures. */ +static int +adjust_exit_status (int status) +{ + if (support_report_failure != NULL) + return support_report_failure (status); + return status; +} + int support_test_main (int argc, char **argv, const struct test_config *config) { @@ -300,7 +312,7 @@ support_test_main (int argc, char **argv, const struct test_config *config) /* If we are not expected to fork run the function immediately. */ if (direct) - return run_test_function (argc, argv, config); + return adjust_exit_status (run_test_function (argc, argv, config)); /* Set up the test environment: - prevent core dumps @@ -363,8 +375,8 @@ support_test_main (int argc, char **argv, const struct test_config *config) if (config->expected_status == 0) { if (config->expected_signal == 0) - /* Simply exit with the return value of the test. */ - return WEXITSTATUS (status); + /* Exit with the return value of the test. */ + return adjust_exit_status (WEXITSTATUS (status)); else { printf ("Expected signal '%s' from child, got none\n", @@ -382,7 +394,7 @@ support_test_main (int argc, char **argv, const struct test_config *config) exit (1); } } - return 0; + return adjust_exit_status (0); } /* Process was killed by timer or other signal. */ else @@ -401,6 +413,6 @@ support_test_main (int argc, char **argv, const struct test_config *config) exit (1); } - return 0; + return adjust_exit_status (0); } } diff --git a/support/tst-support_record_failure-2.sh b/support/tst-support_record_failure-2.sh new file mode 100644 index 0000000..eef6f01 --- /dev/null +++ b/support/tst-support_record_failure-2.sh @@ -0,0 +1,57 @@ +#!/bin/sh +# Test failure recording (with and without --direct). +# Copyright (C) 2016 Free Software Foundation, Inc. +# This file is part of the GNU C Library. + +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . */ + +set -e + +common_objpfx=$1; shift +test_program_prefix_before_env=$1; shift +run_program_env=$1; shift +test_program_prefix_after_env=$1; shift + +run_test () { + expected="$1" + shift + args="${common_objpfx}support/tst-support_record_failure $*" + echo "running: $args" + set +e + output="$(${test_program_prefix_before_env} \ + ${run_program} ${test_program_prefix_after_env} $args)" + actual=$? + set -e + echo " exit status: $actual" + if test $actual -ne $expected ; then + echo "error: exit status $expected expected" + exit 1 + fi + if test "$output" != "error: 1 test failures"; then + echo "error: unexpected ouput: $output" + exit 1 + fi +} + +different_status () { + direct="$1" + run_test 1 $direct --status=0 + run_test 1 $direct --status=1 + run_test 2 $direct --status=2 + run_test 1 $direct --status=77 +} + +different_status +different_status --direct diff --git a/support/tst-support_record_failure.c b/support/tst-support_record_failure.c new file mode 100644 index 0000000..10e601d --- /dev/null +++ b/support/tst-support_record_failure.c @@ -0,0 +1,122 @@ +/* Test support_record_failure state sharing. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include + +#include +#include +#include + +static int exit_status_with_failure = -1; +enum { OPT_STATUS = 10001 }; +#define CMDLINE_OPTIONS \ + { "status", required_argument, NULL, OPT_STATUS }, +static void +cmdline_process (int c) +{ + switch (c) + { + case OPT_STATUS: + exit_status_with_failure = atoi (optarg); + break; + } +} +#define CMDLINE_PROCESS cmdline_process + +static void +check_failure_reporting (int phase, int zero, int unsupported) +{ + int status = support_report_failure (0); + if (status != zero) + { + printf ("real-error (phase %d): support_report_failure (0) == %d\n", + phase, status); + exit (1); + } + status = support_report_failure (1); + if (status != 1) + { + printf ("real-error (phase %d): support_report_failure (1) == %d\n", + phase, status); + exit (1); + } + status = support_report_failure (2); + if (status != 2) + { + printf ("real-error (phase %d): support_report_failure (2) == %d\n", + phase, status); + exit (1); + } + status = support_report_failure (EXIT_UNSUPPORTED); + if (status != unsupported) + { + printf ("real-error (phase %d): " + "support_report_failure (EXIT_UNSUPPORTED) == %d\n", + phase, status); + exit (1); + } +} + +static int +do_test (void) +{ + if (exit_status_with_failure >= 0) + { + /* External invocation with requested error status. Used by + tst-support_report_failure-2.sh. */ + support_record_failure (); + return exit_status_with_failure; + } + + printf ("info: This test tests the test framework.\n" + "info: It reports some expected errors on stdout.\n"); + + /* Check that the status is passed through unchanged. */ + check_failure_reporting (1, 0, EXIT_UNSUPPORTED); + + /* Check state propagation from a subprocess. */ + pid_t pid = xfork (); + if (pid == 0) + { + support_record_failure (); + _exit (0); + } + int status; + xwaitpid (pid, &status, 0); + if (status != 0) + { + printf ("real-error: incorrect status from subprocess: %d\n", status); + return 1; + } + check_failure_reporting (2, 1, 1); + + /* Also test directly in the parent process. */ + support_record_failure_reset (); + check_failure_reporting (3, 0, EXIT_UNSUPPORTED); + support_record_failure (); + check_failure_reporting (4, 1, 1); + + /* We need to mask the failure above. */ + support_record_failure_reset (); + return 0; +} + +#include diff --git a/support/xfork.c b/support/xfork.c new file mode 100644 index 0000000..4b2ce91 --- /dev/null +++ b/support/xfork.c @@ -0,0 +1,34 @@ +/* fork with error checking. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#include +#include + +pid_t +xfork (void) +{ + pid_t result = fork (); + if (result < 0) + { + printf ("error: fork: %m\n"); + exit (1); + } + return result; +} diff --git a/support/xunistd.h b/support/xunistd.h new file mode 100644 index 0000000..f0c7419 --- /dev/null +++ b/support/xunistd.h @@ -0,0 +1,35 @@ +/* POSIX-specific extra functions. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +/* These wrapper functions use POSIX types and therefore cannot be + declared in . */ + +#ifndef SUPPORT_XUNISTD_H +#define SUPPORT_XUNISTD_H + +#include +#include + +__BEGIN_DECLS + +pid_t xfork (void); +pid_t xwaitpid (pid_t, int *status, int flags); + +__END_DECLS + +#endif /* SUPPORT_XUNISTD_H */ diff --git a/support/xwaitpid.c b/support/xwaitpid.c new file mode 100644 index 0000000..5a6e540 --- /dev/null +++ b/support/xwaitpid.c @@ -0,0 +1,35 @@ +/* waitpid with error checking. + Copyright (C) 2016 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include + +#include +#include +#include + +int +xwaitpid (int pid, int *status, int flags) +{ + pid_t result = waitpid (pid, status, flags); + if (result < 0) + { + printf ("error: waitpid: %m\n"); + exit (1); + } + return result; +}