From patchwork Wed May 17 17:51:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shuah Khan X-Patchwork-Id: 684173 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA32FC77B7D for ; Wed, 17 May 2023 17:51:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229624AbjEQRv3 (ORCPT ); Wed, 17 May 2023 13:51:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60064 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229452AbjEQRv2 (ORCPT ); Wed, 17 May 2023 13:51:28 -0400 Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A8099023 for ; Wed, 17 May 2023 10:51:15 -0700 (PDT) Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-76c6e795650so7100739f.1 for ; Wed, 17 May 2023 10:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; t=1684345874; x=1686937874; h=subject:from:cc:to:content-language:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=ZeGga+wRA3DdbN6qGermomi0t82TyTTjoICSjLR9Oak=; b=ZOWQ3xq8cR30z4CmAL/UBjRPsScKhCwFj+r8333u4PiRDsGDCMRjEyZtYqLGebnzpU q5oLs3cbabusvm5yTKq6qdQCFw6hLBYsIz1N+iaC58wPfCvybwGcSaSM3pZxw6djOnJn kx521tNH9qgu+ZeeWDlEo/3IqU+JpS4gD7z0U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684345874; x=1686937874; h=subject:from:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZeGga+wRA3DdbN6qGermomi0t82TyTTjoICSjLR9Oak=; b=fdGmtwQTSYkf/2NK676jrwnJvUIzmog4V+ads1za+tqMmZHrrEJo6wvHc2HucuAK2e Oev39kK0z5ORbdgHDy93dNyknfPGMwCDRoKjkb+5ww75Zeet5EOShzq6WNEc7aenX5ql nYM11Ef25Dr+gOiecuvTDS3ruFuvnHDBsmgc+ATJ7Zt+NGWrIYJOR1ZAqNKRbbMngkEb ekctZ6IRcY+NTLPr7tiDhoVr8BKu8hYAprQInLP51RMHL1lCbUWb8JQH/Oq9g7GPMtKY AeYa9d2Wya1DBcaY2eq1JwwCOzICuptWEoAuOqfwf2sDP/k6rVYmAHVL4KKv1AQ4wRwA zj1w== X-Gm-Message-State: AC+VfDxIVtY455hSQHjt9tT2Cuu1N3baEgECxFPapa+YD56FZ7opsA8Y 7Lhy1dPl0edNqeZKx0mK2P97BQ== X-Google-Smtp-Source: ACHHUZ7i6LAUJx7suPIRn5udDpxPFW5/BSsgRqwXtOB28GbGFRFVKLyVGxOIh0VXUQg4hV/aHq8YHA== X-Received: by 2002:a05:6602:371b:b0:76c:58d8:ff14 with SMTP id bh27-20020a056602371b00b0076c58d8ff14mr2537996iob.2.1684345874596; Wed, 17 May 2023 10:51:14 -0700 (PDT) Received: from [192.168.1.128] ([38.15.45.1]) by smtp.gmail.com with ESMTPSA id l14-20020a6b750e000000b0076c872823b7sm4442428ioh.22.2023.05.17.10.51.13 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 17 May 2023 10:51:14 -0700 (PDT) Message-ID: Date: Wed, 17 May 2023 11:51:13 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US To: torvalds@linux-foundation.org Cc: skhan@linuxfoundation.org, shuah , linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org From: Shuah Khan Subject: [GIT PULL] Kselftest fixes update for Linux 6.4-rc3 Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org Hi Linus, Please pull the following Kselftest fixes update for Linux 6.4-rc3. This Kselftest fixes update for Linux 6.4-rc3 consists of: - sgx test fix for false negatives. - ftrace output is hard to parse and it masks inappropriate skips etc. This fix addresses the problems by integrating with kselftest runner. diff is attached. thanks, -- Shuah ---------------------------------------------------------------- The following changes since commit ac9a78681b921877518763ba0e89202254349d1b: Linux 6.4-rc1 (2023-05-07 13:34:35 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest tags/linux-kselftest-fixes-6.4-rc3 for you to fetch changes up to dbcf76390eb9a65d5d0c37b0cd57335218564e37: selftests/ftrace: Improve integration with kselftest runner (2023-05-08 11:10:13 -0600) ---------------------------------------------------------------- linux-kselftest-fixes-6.4-rc3 This Kselftest fixes update for Linux 6.4-rc3 consists of: - sgx test fix for false negatives. - ftrace output is hard to parse and it masks inappropriate skips etc. This fix addresses the problems by integrating with kselftest runner. ---------------------------------------------------------------- Mark Brown (1): selftests/ftrace: Improve integration with kselftest runner Yi Lai (1): selftests/sgx: Add "test_encl.elf" to TEST_FILES tools/testing/selftests/ftrace/Makefile | 3 +- tools/testing/selftests/ftrace/ftracetest | 63 ++++++++++++++++++++++++-- tools/testing/selftests/ftrace/ftracetest-ktap | 8 ++++ tools/testing/selftests/sgx/Makefile | 1 + 4 files changed, 71 insertions(+), 4 deletions(-) create mode 100755 tools/testing/selftests/ftrace/ftracetest-ktap ---------------------------------------------------------------- diff --git a/tools/testing/selftests/ftrace/Makefile b/tools/testing/selftests/ftrace/Makefile index d6e106fbce11..a1e955d2de4c 100644 --- a/tools/testing/selftests/ftrace/Makefile +++ b/tools/testing/selftests/ftrace/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 all: -TEST_PROGS := ftracetest +TEST_PROGS_EXTENDED := ftracetest +TEST_PROGS := ftracetest-ktap TEST_FILES := test.d settings EXTRA_CLEAN := $(OUTPUT)/logs/* diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest index c3311c8c4089..2506621e75df 100755 --- a/tools/testing/selftests/ftrace/ftracetest +++ b/tools/testing/selftests/ftrace/ftracetest @@ -13,6 +13,7 @@ echo "Usage: ftracetest [options] [testcase(s)] [testcase-directory(s)]" echo " Options:" echo " -h|--help Show help message" echo " -k|--keep Keep passed test logs" +echo " -K|--ktap Output in KTAP format" echo " -v|--verbose Increase verbosity of test messages" echo " -vv Alias of -v -v (Show all results in stdout)" echo " -vvv Alias of -v -v -v (Show all commands immediately)" @@ -85,6 +86,10 @@ parse_opts() { # opts KEEP_LOG=1 shift 1 ;; + --ktap|-K) + KTAP=1 + shift 1 + ;; --verbose|-v|-vv|-vvv) if [ $VERBOSE -eq -1 ]; then usage "--console can not use with --verbose" @@ -178,6 +183,7 @@ TEST_DIR=$TOP_DIR/test.d TEST_CASES=`find_testcases $TEST_DIR` LOG_DIR=$TOP_DIR/logs/`date +%Y%m%d-%H%M%S`/ KEEP_LOG=0 +KTAP=0 DEBUG=0 VERBOSE=0 UNSUPPORTED_RESULT=0 @@ -229,7 +235,7 @@ prlog() { # messages newline= shift fi - printf "$*$newline" + [ "$KTAP" != "1" ] && printf "$*$newline" [ "$LOG_FILE" ] && printf "$*$newline" | strip_esc >> $LOG_FILE } catlog() { #file @@ -260,11 +266,11 @@ TOTAL_RESULT=0 INSTANCE= CASENO=0 +CASENAME= testcase() { # testfile CASENO=$((CASENO+1)) - desc=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` - prlog -n "[$CASENO]$INSTANCE$desc" + CASENAME=`grep "^#[ \t]*description:" $1 | cut -f2- -d:` } checkreq() { # testfile @@ -277,40 +283,68 @@ test_on_instance() { # testfile grep -q "^#[ \t]*flags:.*instance" $1 } +ktaptest() { # result comment + if [ "$KTAP" != "1" ]; then + return + fi + + local result= + if [ "$1" = "1" ]; then + result="ok" + else + result="not ok" + fi + shift + + local comment=$* + if [ "$comment" != "" ]; then + comment="# $comment" + fi + + echo $CASENO $result $INSTANCE$CASENAME $comment +} + eval_result() { # sigval case $1 in $PASS) prlog " [${color_green}PASS${color_reset}]" + ktaptest 1 PASSED_CASES="$PASSED_CASES $CASENO" return 0 ;; $FAIL) prlog " [${color_red}FAIL${color_reset}]" + ktaptest 0 FAILED_CASES="$FAILED_CASES $CASENO" return 1 # this is a bug. ;; $UNRESOLVED) prlog " [${color_blue}UNRESOLVED${color_reset}]" + ktaptest 0 UNRESOLVED UNRESOLVED_CASES="$UNRESOLVED_CASES $CASENO" return $UNRESOLVED_RESULT # depends on use case ;; $UNTESTED) prlog " [${color_blue}UNTESTED${color_reset}]" + ktaptest 1 SKIP UNTESTED_CASES="$UNTESTED_CASES $CASENO" return 0 ;; $UNSUPPORTED) prlog " [${color_blue}UNSUPPORTED${color_reset}]" + ktaptest 1 SKIP UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO" return $UNSUPPORTED_RESULT # depends on use case ;; $XFAIL) prlog " [${color_green}XFAIL${color_reset}]" + ktaptest 1 XFAIL XFAILED_CASES="$XFAILED_CASES $CASENO" return 0 ;; *) prlog " [${color_blue}UNDEFINED${color_reset}]" + ktaptest 0 error UNDEFINED_CASES="$UNDEFINED_CASES $CASENO" return 1 # this must be a test bug ;; @@ -371,6 +405,7 @@ __run_test() { # testfile run_test() { # testfile local testname=`basename $1` testcase $1 + prlog -n "[$CASENO]$INSTANCE$CASENAME" if [ ! -z "$LOG_FILE" ] ; then local testlog=`mktemp $LOG_DIR/${CASENO}-${testname}-log.XXXXXX` else @@ -405,6 +440,17 @@ run_test() { # testfile # load in the helper functions . $TEST_DIR/functions +if [ "$KTAP" = "1" ]; then + echo "TAP version 13" + + casecount=`echo $TEST_CASES | wc -w` + for t in $TEST_CASES; do + test_on_instance $t || continue + casecount=$((casecount+1)) + done + echo "1..${casecount}" +fi + # Main loop for t in $TEST_CASES; do run_test $t @@ -439,6 +485,17 @@ prlog "# of unsupported: " `echo $UNSUPPORTED_CASES | wc -w` prlog "# of xfailed: " `echo $XFAILED_CASES | wc -w` prlog "# of undefined(test bug): " `echo $UNDEFINED_CASES | wc -w` +if [ "$KTAP" = "1" ]; then + echo -n "# Totals:" + echo -n " pass:"`echo $PASSED_CASES | wc -w` + echo -n " faii:"`echo $FAILED_CASES | wc -w` + echo -n " xfail:"`echo $XFAILED_CASES | wc -w` + echo -n " xpass:0" + echo -n " skip:"`echo $UNTESTED_CASES $UNSUPPORTED_CASES | wc -w` + echo -n " error:"`echo $UNRESOLVED_CASES $UNDEFINED_CASES | wc -w` + echo +fi + cleanup # if no error, return 0 diff --git a/tools/testing/selftests/ftrace/ftracetest-ktap b/tools/testing/selftests/ftrace/ftracetest-ktap new file mode 100755 index 000000000000..b3284679ef3a --- /dev/null +++ b/tools/testing/selftests/ftrace/ftracetest-ktap @@ -0,0 +1,8 @@ +#!/bin/sh -e +# SPDX-License-Identifier: GPL-2.0-only +# +# ftracetest-ktap: Wrapper to integrate ftracetest with the kselftest runner +# +# Copyright (C) Arm Ltd., 2023 + +./ftracetest -K diff --git a/tools/testing/selftests/sgx/Makefile b/tools/testing/selftests/sgx/Makefile index 75af864e07b6..50aab6b57da3 100644 --- a/tools/testing/selftests/sgx/Makefile +++ b/tools/testing/selftests/sgx/Makefile @@ -17,6 +17,7 @@ ENCL_CFLAGS := -Wall -Werror -static -nostdlib -nostartfiles -fPIC \ -fno-stack-protector -mrdrnd $(INCLUDES) TEST_CUSTOM_PROGS := $(OUTPUT)/test_sgx +TEST_FILES := $(OUTPUT)/test_encl.elf ifeq ($(CAN_BUILD_X86_64), 1) all: $(TEST_CUSTOM_PROGS) $(OUTPUT)/test_encl.elf