From patchwork Fri Jan 20 20:32:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Holmes X-Patchwork-Id: 92115 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp196374qgi; Fri, 20 Jan 2017 12:36:02 -0800 (PST) X-Received: by 10.233.239.17 with SMTP id d17mr13819343qkg.13.1484944562232; Fri, 20 Jan 2017 12:36:02 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id n205si5617707qke.139.2017.01.20.12.36.01; Fri, 20 Jan 2017 12:36:02 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D528C60C02; Fri, 20 Jan 2017 20:36:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id BB12660C20; Fri, 20 Jan 2017 20:33:31 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 285EE60C1A; Fri, 20 Jan 2017 20:33:13 +0000 (UTC) Received: from mail-yb0-f182.google.com (mail-yb0-f182.google.com [209.85.213.182]) by lists.linaro.org (Postfix) with ESMTPS id DD83F60C17 for ; Fri, 20 Jan 2017 20:32:13 +0000 (UTC) Received: by mail-yb0-f182.google.com with SMTP id l23so70084655ybj.2 for ; Fri, 20 Jan 2017 12:32:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=kncTlw/IAT6FxTFadvv/riOacW1k+MXtiG4K7oYGYWc=; b=DMrN63TzKda0/7ivLxYr6qS8a5EyKhTTGALi9lVoNhlaFcDCH41/J+RAHOMX0xsxZU I5iVTp4cBS4ob4CISDDzYs7cvZCndxIZKRj1zm6jArKQyRmxpXoAFE++ASqheE/okT7c sggwfm0rL1oQkrHWVn0n4S+PjsmR2ZmmsC61OunqO2AWIYUTSsiFBr8fjxMR9Zfk8kdC FkIrqxAuk8TzYTP/Fh5bWziIEtAo357JXMpMUlqPt9tJM1xj9W/Ib6tzi4IwpnlM8brx C2r+XQiyCa5L9O+TaBgpymriC4Pi4K9v3WKxv47TnR4OysF4x5Y56PWLo3+hHDLK3dff okiQ== X-Gm-Message-State: AIkVDXLfNzpq3HfiEyPqmofUHd+MbiijOjsCdf7QEM2DHX5lB4Z2EOIFdGdYCNyT8vxfCiYYH1w= X-Received: by 10.237.53.124 with SMTP id b57mr14263683qte.7.1484944332752; Fri, 20 Jan 2017 12:32:12 -0800 (PST) Received: from localhost (c-98-221-136-245.hsd1.nj.comcast.net. [98.221.136.245]) by smtp.gmail.com with ESMTPSA id x62sm6762461qkg.31.2017.01.20.12.32.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 20 Jan 2017 12:32:11 -0800 (PST) From: Mike Holmes To: lng-odp@lists.linaro.org Date: Fri, 20 Jan 2017 15:32:02 -0500 Message-Id: <20170120203203.2454-4-mike.holmes@linaro.org> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20170120203203.2454-1-mike.holmes@linaro.org> References: <20170120203203.2454-1-mike.holmes@linaro.org> Subject: [lng-odp] [PATCH v4 3/4] helper: move thread implementation under platform X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Only the portable api is built by default, use --enable-helper-extn to enable non portable APIs for a helper platform Signed-off-by: Mike Holmes --- configure.ac | 17 +- example/Makefile.inc | 2 +- helper/Makefile.am | 16 +- .../helper/platform/linux-generic/threads_extn.h | 112 ++++++++ helper/include/odp/helper/threads.h | 76 ----- helper/m4/configure.m4 | 11 + helper/platform/linux-generic/thread.c | 313 +++++++++++++++++++++ helper/test/Makefile.am | 19 +- helper/test/linux-generic/Makefile.am | 5 + helper/test/{ => linux-generic}/process.c | 2 +- helper/test/{ => linux-generic}/thread.c | 2 +- helper/threads.c | 238 +--------------- ...inux.pc.in => libodphelper-linux-generic.pc.in} | 4 +- test/Makefile.inc | 2 +- test/common_plat/validation/api/Makefile.inc | 2 +- test/linux-generic/Makefile.inc | 2 +- 16 files changed, 498 insertions(+), 325 deletions(-) create mode 100644 helper/include/odp/helper/platform/linux-generic/threads_extn.h create mode 100644 helper/platform/linux-generic/thread.c create mode 100644 helper/test/linux-generic/Makefile.am rename helper/test/{ => linux-generic}/process.c (97%) rename helper/test/{ => linux-generic}/thread.c (97%) rename pkgconfig/{libodphelper-linux.pc.in => libodphelper-linux-generic.pc.in} (72%) -- 2.9.3 diff --git a/configure.ac b/configure.ac index 1384130..2cf6536 100644 --- a/configure.ac +++ b/configure.ac @@ -138,6 +138,18 @@ AC_SUBST([with_platform]) AC_SUBST([platform_with_platform], ["platform/${with_platform}"]) ########################################################################## +# Determine which helper platform to build for +########################################################################## +AC_ARG_WITH([helper_platform], + [AS_HELP_STRING([--with-helper_platform=platform], + [select helper platform to be used, default linux-generic])], + [], + [with_helper_platform=${with_platform} + ]) + +AC_SUBST([with_helper_platform]) + +########################################################################## # Run platform specific checks and settings ########################################################################## IMPLEMENTATION_NAME="" @@ -202,6 +214,7 @@ AM_CONDITIONAL([test_example], [test x$test_example = xyes ]) AM_CONDITIONAL([HAVE_DOXYGEN], [test "x${DOXYGEN}" = "xdoxygen"]) AM_CONDITIONAL([user_guide], [test "x${user_guides}" = "xyes" ]) AM_CONDITIONAL([HAVE_MSCGEN], [test "x${MSCGEN}" = "xmscgen"]) +AM_CONDITIONAL([helper_extn], [test x$helper_extn = xyes ]) ########################################################################## # Setup doxygen documentation @@ -295,7 +308,7 @@ AM_CXXFLAGS="-std=c++11" AC_CONFIG_FILES([Makefile pkgconfig/libodp-linux.pc - pkgconfig/libodphelper-linux.pc + pkgconfig/libodphelper-linux-generic.pc ]) AC_SEARCH_LIBS([timer_create],[rt posix4]) @@ -322,6 +335,8 @@ AC_MSG_RESULT([ implementation_name: ${IMPLEMENTATION_NAME} ARCH_DIR ${ARCH_DIR} with_platform: ${with_platform} + with_helper_platform: ${with_helper_platform} + helper_extn: ${helper_extn} prefix: ${prefix} sysconfdir: ${sysconfdir} libdir: ${libdir} diff --git a/example/Makefile.inc b/example/Makefile.inc index 19d3994..ea596d5 100644 --- a/example/Makefile.inc +++ b/example/Makefile.inc @@ -1,6 +1,6 @@ include $(top_srcdir)/platform/@with_platform@/Makefile.inc LIB = $(top_builddir)/lib -LDADD = $(LIB)/libodp-linux.la $(LIB)/libodphelper-linux.la +LDADD = $(LIB)/libodp-linux.la $(LIB)/libodphelper-@with_helper_platform@.la AM_CFLAGS += \ -I$(srcdir) \ -I$(top_srcdir)/example \ diff --git a/helper/Makefile.am b/helper/Makefile.am index 62e55cc..9b2f18a 100644 --- a/helper/Makefile.am +++ b/helper/Makefile.am @@ -1,7 +1,7 @@ include $(top_srcdir)/platform/@with_platform@/Makefile.inc pkgconfigdir = $(libdir)/pkgconfig -pkgconfig_DATA = $(top_builddir)/pkgconfig/libodphelper-linux.pc +pkgconfig_DATA = $(top_builddir)/pkgconfig/libodphelper-linux-generic.pc LIB = $(top_builddir)/lib AM_CFLAGS = -I$(srcdir)/include @@ -25,6 +25,11 @@ helperinclude_HEADERS = \ $(srcdir)/include/odp/helper/threads.h \ $(srcdir)/include/odp/helper/udp.h +if helper_extn +helperinclude_HEADERS += \ + $(srcdir)/include/odp/helper/platform/@with_helper_platform@/threads_extn.h +endif + noinst_HEADERS = \ $(srcdir)/odph_debug.h \ $(srcdir)/odph_hashtable.h \ @@ -33,7 +38,7 @@ noinst_HEADERS = \ $(srcdir)/odph_list_internal.h \ $(srcdir)/odph_iplookuptable.h -__LIB__libodphelper_linux_la_SOURCES = \ +__LIB__libodphelper_@with_platform@_la_SOURCES = \ eth.c \ ip.c \ chksum.c \ @@ -43,4 +48,9 @@ __LIB__libodphelper_linux_la_SOURCES = \ iplookuptable.c \ threads.c -lib_LTLIBRARIES = $(LIB)/libodphelper-linux.la +if helper_extn +__LIB__libodphelper_@with_platform@_la_SOURCES += \ + platform/@with_helper_platform@/thread.c +endif + +lib_LTLIBRARIES = $(LIB)/libodphelper-@with_platform@.la diff --git a/helper/include/odp/helper/platform/linux-generic/threads_extn.h b/helper/include/odp/helper/platform/linux-generic/threads_extn.h new file mode 100644 index 0000000..1d4036d --- /dev/null +++ b/helper/include/odp/helper/platform/linux-generic/threads_extn.h @@ -0,0 +1,112 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +/** + * @file + * + * ODP Linux helper extension API + * + * This file is an optional helper to odp.h APIs. These functions are provided + * to ease common setups in a Linux system. User is free to implement the same + * setups in otherways (not via this API). + */ + +#ifndef ODPH_LINUX_EXT_H_ +#define ODPH_LINUX_EXT_H_ + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +/** @addtogroup odph_linux ODPH LINUX + * @{ + */ + +/** + * Creates and launches pthreads + * + * Creates, pins and launches threads to separate CPU's based on the cpumask. + * + * @param[out] pthread_tbl Table of pthread state information records. Table + * must have at least as many entries as there are + * CPUs in the CPU mask. + * @param mask CPU mask + * @param thr_params Linux helper thread parameters + * + * @return Number of threads created + */ +int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params); + +/** + * Waits pthreads to exit + * + * Returns when all threads have been exit. + * + * @param thread_tbl Thread table + * @param num Number of threads to create + * + */ +void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); + +/** + * Fork a process + * + * Forks and sets CPU affinity for the child process. Ignores 'start' and 'arg' + * thread parameters. + * + * @param[out] proc Pointer to process state info (for output) + * @param cpu Destination CPU for the child process + * @param thr_params Linux helper thread parameters + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, + const odph_linux_thr_params_t *thr_params); + +/** + * Fork a number of processes + * + * Forks and sets CPU affinity for child processes. Ignores 'start' and 'arg' + * thread parameters. + * + * @param[out] proc_tbl Process state info table (for output) + * @param mask CPU mask of processes to create + * @param thr_params Linux helper thread parameters + * + * @return On success: 1 for the parent, 0 for the child + * On failure: -1 for the parent, -2 for the child + */ +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params); + +/** + * Wait for a number of processes + * + * Waits for a number of child processes to terminate. Records process state + * change status into the process state info structure. + * + * @param proc_tbl Process state info table (previously filled by fork) + * @param num Number of processes to wait + * + * @return 0 on success, -1 on failure + */ +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); + +/** + * @} + */ + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/helper/include/odp/helper/threads.h b/helper/include/odp/helper/threads.h index b8d975a..5682bab 100644 --- a/helper/include/odp/helper/threads.h +++ b/helper/include/odp/helper/threads.h @@ -93,82 +93,6 @@ typedef struct { } odph_odpthread_t; /** - * Creates and launches pthreads - * - * Creates, pins and launches threads to separate CPU's based on the cpumask. - * - * @param[out] pthread_tbl Table of pthread state information records. Table - * must have at least as many entries as there are - * CPUs in the CPU mask. - * @param mask CPU mask - * @param thr_params Linux helper thread parameters - * - * @return Number of threads created - */ -int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params); - -/** - * Waits pthreads to exit - * - * Returns when all threads have been exit. - * - * @param thread_tbl Thread table - * @param num Number of threads to create - * - */ -void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num); - -/** - * Fork a process - * - * Forks and sets CPU affinity for the child process. Ignores 'start' and 'arg' - * thread parameters. - * - * @param[out] proc Pointer to process state info (for output) - * @param cpu Destination CPU for the child process - * @param thr_params Linux helper thread parameters - * - * @return On success: 1 for the parent, 0 for the child - * On failure: -1 for the parent, -2 for the child - */ -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, - const odph_linux_thr_params_t *thr_params); - - -/** - * Fork a number of processes - * - * Forks and sets CPU affinity for child processes. Ignores 'start' and 'arg' - * thread parameters. - * - * @param[out] proc_tbl Process state info table (for output) - * @param mask CPU mask of processes to create - * @param thr_params Linux helper thread parameters - * - * @return On success: 1 for the parent, 0 for the child - * On failure: -1 for the parent, -2 for the child - */ -int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params); - - -/** - * Wait for a number of processes - * - * Waits for a number of child processes to terminate. Records process state - * change status into the process state info structure. - * - * @param proc_tbl Process state info table (previously filled by fork) - * @param num Number of processes to wait - * - * @return 0 on success, -1 on failure - */ -int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num); - -/** * Creates and launches odpthreads (as linux threads or processes) * * Creates, pins and launches threads to separate CPU's based on the cpumask. diff --git a/helper/m4/configure.m4 b/helper/m4/configure.m4 index 7b208fa..38c95d9 100644 --- a/helper/m4/configure.m4 +++ b/helper/m4/configure.m4 @@ -8,5 +8,16 @@ AC_ARG_ENABLE([test-helper], test_helper=yes fi]) +########################################################################## +# Enable/disable helper-ext +# platform specific non portable extensions +########################################################################## +helper_extn=no +AC_ARG_ENABLE([helper-extn], + [ --enable-helper-extn build helper platform extensions (not portable)], + [if test "x$enableval" = "xyes"; then + helper_extn=yes + fi]) + AC_CONFIG_FILES([helper/Makefile helper/test/Makefile]) diff --git a/helper/platform/linux-generic/thread.c b/helper/platform/linux-generic/thread.c new file mode 100644 index 0000000..90fa42a --- /dev/null +++ b/helper/platform/linux-generic/thread.c @@ -0,0 +1,313 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include "odph_debug.h" + +static void *_odph_run_start_routine(void *arg) +{ + odph_linux_thr_params_t *thr_params = arg; + + /* ODP thread local init */ + if (odp_init_local(thr_params->instance, thr_params->thr_type)) { + ODPH_ERR("Local init failed\n"); + return NULL; + } + + void *ret_ptr = thr_params->start(thr_params->arg); + int ret = odp_term_local(); + + if (ret < 0) + ODPH_ERR("Local term failed\n"); + + return ret_ptr; +} + +static void *_odph_thread_run_start_routine(void *arg) +{ + int status; + int ret; + odph_odpthread_params_t *thr_params; + + odph_odpthread_start_args_t *start_args = arg; + + thr_params = &start_args->thr_params; + + /* ODP thread local init */ + if (odp_init_local(thr_params->instance, thr_params->thr_type)) { + ODPH_ERR("Local init failed\n"); + if (start_args->linuxtype == ODPTHREAD_PROCESS) + _exit(EXIT_FAILURE); + return (void *)-1; + } + + ODPH_DBG("helper: ODP %s thread started as linux %s. (pid=%d)\n", + thr_params->thr_type == ODP_THREAD_WORKER ? + "worker" : "control", + (start_args->linuxtype == ODPTHREAD_PTHREAD) ? + "pthread" : "process", + (int)getpid()); + + status = thr_params->start(thr_params->arg); + ret = odp_term_local(); + + if (ret < 0) + ODPH_ERR("Local term failed\n"); + + /* for process implementation of odp threads, just return status... */ + if (start_args->linuxtype == ODPTHREAD_PROCESS) + _exit(status); + + /* threads implementation return void* pointers: cast status to that. */ + return (void *)(intptr_t)status; +} + +int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params) +{ + int i; + int num; + int cpu_count; + int cpu; + int ret; + + num = odp_cpumask_count(mask); + + memset(pthread_tbl, 0, num * sizeof(odph_linux_pthread_t)); + + cpu_count = odp_cpu_count(); + + if (num < 1 || num > cpu_count) { + ODPH_ERR("Invalid number of threads:%d (%d cores available)\n", + num, cpu_count); + return 0; + } + + cpu = odp_cpumask_first(mask); + for (i = 0; i < num; i++) { + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pthread_attr_init(&pthread_tbl[i].attr); + + pthread_tbl[i].cpu = cpu; + + pthread_attr_setaffinity_np(&pthread_tbl[i].attr, + sizeof(cpu_set_t), &cpu_set); + + pthread_tbl[i].thr_params.start = thr_params->start; + pthread_tbl[i].thr_params.arg = thr_params->arg; + pthread_tbl[i].thr_params.thr_type = thr_params->thr_type; + pthread_tbl[i].thr_params.instance = thr_params->instance; + + ret = pthread_create(&pthread_tbl[i].thread, + &pthread_tbl[i].attr, + _odph_run_start_routine, + &pthread_tbl[i].thr_params); + if (ret != 0) { + ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); + break; + } + + cpu = odp_cpumask_next(mask, cpu); + } + + return i; +} + +void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) +{ + int i; + int ret; + + for (i = 0; i < num; i++) { + /* Wait thread to exit */ + ret = pthread_join(thread_tbl[i].thread, NULL); + if (ret != 0) { + ODPH_ERR("Failed to join thread from cpu #%d\n", + thread_tbl[i].cpu); + } + pthread_attr_destroy(&thread_tbl[i].attr); + } +} + +int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, + const odp_cpumask_t *mask, + const odph_linux_thr_params_t *thr_params) +{ + pid_t pid; + int num; + int cpu_count; + int cpu; + int i; + + num = odp_cpumask_count(mask); + + memset(proc_tbl, 0, num * sizeof(odph_linux_process_t)); + + cpu_count = odp_cpu_count(); + + if (num < 1 || num > cpu_count) { + ODPH_ERR("Bad num\n"); + return -1; + } + + cpu = odp_cpumask_first(mask); + for (i = 0; i < num; i++) { + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pid = fork(); + + if (pid < 0) { + ODPH_ERR("fork() failed\n"); + return -1; + } + + /* Parent continues to fork */ + if (pid > 0) { + proc_tbl[i].pid = pid; + proc_tbl[i].cpu = cpu; + + cpu = odp_cpumask_next(mask, cpu); + continue; + } + + /* Child process */ + + /* Request SIGTERM if parent dies */ + prctl(PR_SET_PDEATHSIG, SIGTERM); + /* Parent died already? */ + if (getppid() == 1) + kill(getpid(), SIGTERM); + + if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { + ODPH_ERR("sched_setaffinity() failed\n"); + return -2; + } + + if (odp_init_local(thr_params->instance, + thr_params->thr_type)) { + ODPH_ERR("Local init failed\n"); + return -2; + } + + return 0; + } + + return 1; +} + +int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, + const odph_linux_thr_params_t *thr_params) +{ + odp_cpumask_t mask; + + odp_cpumask_zero(&mask); + odp_cpumask_set(&mask, cpu); + return odph_linux_process_fork_n(proc, &mask, thr_params); +} + +int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) +{ + pid_t pid; + int i, j; + int status = 0; + + for (i = 0; i < num; i++) { + pid = wait(&status); + + if (pid < 0) { + ODPH_ERR("wait() failed\n"); + return -1; + } + + for (j = 0; j < num; j++) { + if (proc_tbl[j].pid == pid) { + proc_tbl[j].status = status; + break; + } + } + + if (j == num) { + ODPH_ERR("Bad pid:%d\n", (int)pid); + return -1; + } + + /* Examine the child process' termination status */ + if (WIFEXITED(status) && WEXITSTATUS(status) != EXIT_SUCCESS) { + ODPH_ERR("Child exit status:%d (pid:%d)\n", + WEXITSTATUS(status), (int)pid); + return -1; + } + if (WIFSIGNALED(status)) { + int signo = WTERMSIG(status); + + ODPH_ERR("Child term signo:%d - %s (pid:%d)\n", + signo, strsignal(signo), (int)pid); + return -1; + } + } + + return 0; +} + +/* + * Create a single ODPthread as a linux thread + */ +static int odph_linux_thread_create(odph_odpthread_t *thread_tbl, + int cpu, + const odph_odpthread_params_t *thr_params) +{ + int ret; + cpu_set_t cpu_set; + + CPU_ZERO(&cpu_set); + CPU_SET(cpu, &cpu_set); + + pthread_attr_init(&thread_tbl->thread.attr); + + thread_tbl->cpu = cpu; + + pthread_attr_setaffinity_np(&thread_tbl->thread.attr, + sizeof(cpu_set_t), &cpu_set); + + thread_tbl->start_args.thr_params = *thr_params; /* copy */ + thread_tbl->start_args.linuxtype = ODPTHREAD_PTHREAD; + + ret = pthread_create(&thread_tbl->thread.thread_id, + &thread_tbl->thread.attr, + _odph_thread_run_start_routine, + &thread_tbl->start_args); + if (ret != 0) { + ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); + thread_tbl->start_args.linuxtype = ODPTHREAD_NOT_STARTED; + return ret; + } + + return 0; +} diff --git a/helper/test/Makefile.am b/helper/test/Makefile.am index 2bf6765..7d55ae6 100644 --- a/helper/test/Makefile.am +++ b/helper/test/Makefile.am @@ -7,12 +7,19 @@ TESTS_ENVIRONMENT += TEST_DIR=${builddir} EXECUTABLES = chksum$(EXEEXT) \ cuckootable$(EXEEXT) \ - table$(EXEEXT) \ - thread$(EXEEXT) \ parse$(EXEEXT)\ - process$(EXEEXT) \ + table$(EXEEXT) \ iplookuptable$(EXEEXT) +#These are platform specific extensions that are not portable +#They are a convenience to app writers who have chosen to +#restrict their application to Linux. + +if helper_extn +EXECUTABLES += @with_helper_platform@/thread$(EXEEXT) \ + @with_helper_platform@/process$(EXEEXT) +endif + COMPILE_ONLY = odpthreads TESTSCRIPTS = odpthreads_as_processes \ @@ -31,11 +38,7 @@ EXTRA_DIST = odpthreads_as_processes odpthreads_as_pthreads dist_chksum_SOURCES = chksum.c dist_cuckootable_SOURCES = cuckootable.c dist_odpthreads_SOURCES = odpthreads.c -odpthreads_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la -dist_thread_SOURCES = thread.c -thread_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la -dist_process_SOURCES = process.c +odpthreads_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la dist_parse_SOURCES = parse.c -process_LDADD = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la dist_table_SOURCES = table.c dist_iplookuptable_SOURCES = iplookuptable.c diff --git a/helper/test/linux-generic/Makefile.am b/helper/test/linux-generic/Makefile.am new file mode 100644 index 0000000..28d54a8 --- /dev/null +++ b/helper/test/linux-generic/Makefile.am @@ -0,0 +1,5 @@ + +thread_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la +dist_thread_SOURCES = thread.c +dist_process_SOURCES = process.c +process_LDADD = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la diff --git a/helper/test/process.c b/helper/test/linux-generic/process.c similarity index 97% rename from helper/test/process.c rename to helper/test/linux-generic/process.c index f3c6d50..f641128 100644 --- a/helper/test/process.c +++ b/helper/test/linux-generic/process.c @@ -6,7 +6,7 @@ #include #include -#include +#include #define NUMBER_WORKERS 16 /* 0 = max */ diff --git a/helper/test/thread.c b/helper/test/linux-generic/thread.c similarity index 97% rename from helper/test/thread.c rename to helper/test/linux-generic/thread.c index da94b49..f1ca1b7 100644 --- a/helper/test/thread.c +++ b/helper/test/linux-generic/thread.c @@ -6,7 +6,7 @@ #include #include -#include +#include #define NUMBER_WORKERS 16 static void *worker_fn(void *arg TEST_UNUSED) diff --git a/helper/threads.c b/helper/threads.c index 690b6ca..fe5d2bd 100644 --- a/helper/threads.c +++ b/helper/threads.c @@ -9,16 +9,10 @@ #endif #include #include -#include #include #include #include -#include -#include -#include -#include - #include #include #include "odph_debug.h" @@ -27,225 +21,11 @@ static struct { int proc; /* true when process mode is required, false otherwise */ } helper_options; -static void *odp_run_start_routine(void *arg) -{ - odph_linux_thr_params_t *thr_params = arg; - - /* ODP thread local init */ - if (odp_init_local(thr_params->instance, thr_params->thr_type)) { - ODPH_ERR("Local init failed\n"); - return NULL; - } - - void *ret_ptr = thr_params->start(thr_params->arg); - int ret = odp_term_local(); - - if (ret < 0) - ODPH_ERR("Local term failed\n"); - - return ret_ptr; -} - -int odph_linux_pthread_create(odph_linux_pthread_t *pthread_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params) -{ - int i; - int num; - int cpu_count; - int cpu; - int ret; - - num = odp_cpumask_count(mask); - - memset(pthread_tbl, 0, num * sizeof(odph_linux_pthread_t)); - - cpu_count = odp_cpu_count(); - - if (num < 1 || num > cpu_count) { - ODPH_ERR("Invalid number of threads:%d (%d cores available)\n", - num, cpu_count); - return 0; - } - - cpu = odp_cpumask_first(mask); - for (i = 0; i < num; i++) { - cpu_set_t cpu_set; - - CPU_ZERO(&cpu_set); - CPU_SET(cpu, &cpu_set); - - pthread_attr_init(&pthread_tbl[i].attr); - - pthread_tbl[i].cpu = cpu; - - pthread_attr_setaffinity_np(&pthread_tbl[i].attr, - sizeof(cpu_set_t), &cpu_set); - - pthread_tbl[i].thr_params.start = thr_params->start; - pthread_tbl[i].thr_params.arg = thr_params->arg; - pthread_tbl[i].thr_params.thr_type = thr_params->thr_type; - pthread_tbl[i].thr_params.instance = thr_params->instance; - - ret = pthread_create(&pthread_tbl[i].thread, - &pthread_tbl[i].attr, - odp_run_start_routine, - &pthread_tbl[i].thr_params); - if (ret != 0) { - ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); - break; - } - - cpu = odp_cpumask_next(mask, cpu); - } - - return i; -} - -void odph_linux_pthread_join(odph_linux_pthread_t *thread_tbl, int num) -{ - int i; - int ret; - - for (i = 0; i < num; i++) { - /* Wait thread to exit */ - ret = pthread_join(thread_tbl[i].thread, NULL); - if (ret != 0) { - ODPH_ERR("Failed to join thread from cpu #%d\n", - thread_tbl[i].cpu); - } - pthread_attr_destroy(&thread_tbl[i].attr); - } -} - -int odph_linux_process_fork_n(odph_linux_process_t *proc_tbl, - const odp_cpumask_t *mask, - const odph_linux_thr_params_t *thr_params) -{ - pid_t pid; - int num; - int cpu_count; - int cpu; - int i; - - num = odp_cpumask_count(mask); - - memset(proc_tbl, 0, num * sizeof(odph_linux_process_t)); - - cpu_count = odp_cpu_count(); - - if (num < 1 || num > cpu_count) { - ODPH_ERR("Bad num\n"); - return -1; - } - - cpu = odp_cpumask_first(mask); - for (i = 0; i < num; i++) { - cpu_set_t cpu_set; - - CPU_ZERO(&cpu_set); - CPU_SET(cpu, &cpu_set); - - pid = fork(); - - if (pid < 0) { - ODPH_ERR("fork() failed\n"); - return -1; - } - - /* Parent continues to fork */ - if (pid > 0) { - proc_tbl[i].pid = pid; - proc_tbl[i].cpu = cpu; - - cpu = odp_cpumask_next(mask, cpu); - continue; - } - - /* Child process */ - - /* Request SIGTERM if parent dies */ - prctl(PR_SET_PDEATHSIG, SIGTERM); - /* Parent died already? */ - if (getppid() == 1) - kill(getpid(), SIGTERM); - - if (sched_setaffinity(0, sizeof(cpu_set_t), &cpu_set)) { - ODPH_ERR("sched_setaffinity() failed\n"); - return -2; - } - - if (odp_init_local(thr_params->instance, - thr_params->thr_type)) { - ODPH_ERR("Local init failed\n"); - return -2; - } - - return 0; - } - - return 1; -} - -int odph_linux_process_fork(odph_linux_process_t *proc, int cpu, - const odph_linux_thr_params_t *thr_params) -{ - odp_cpumask_t mask; - - odp_cpumask_zero(&mask); - odp_cpumask_set(&mask, cpu); - return odph_linux_process_fork_n(proc, &mask, thr_params); -} - -int odph_linux_process_wait_n(odph_linux_process_t *proc_tbl, int num) -{ - pid_t pid; - int i, j; - int status = 0; - - for (i = 0; i < num; i++) { - pid = wait(&status); - - if (pid < 0) { - ODPH_ERR("wait() failed\n"); - return -1; - } - - for (j = 0; j < num; j++) { - if (proc_tbl[j].pid == pid) { - proc_tbl[j].status = status; - break; - } - } - - if (j == num) { - ODPH_ERR("Bad pid:%d\n", (int)pid); - return -1; - } - - /* Examine the child process' termination status */ - if (WIFEXITED(status) && WEXITSTATUS(status) != EXIT_SUCCESS) { - ODPH_ERR("Child exit status:%d (pid:%d)\n", - WEXITSTATUS(status), (int)pid); - return -1; - } - if (WIFSIGNALED(status)) { - int signo = WTERMSIG(status); - - ODPH_ERR("Child term signo:%d - %s (pid:%d)\n", - signo, strsignal(signo), (int)pid); - return -1; - } - } - - return 0; -} - /* * wrapper for odpthreads, either implemented as linux threads or processes. * (in process mode, if start_routine returns NULL, the process return FAILURE). */ -static void *odpthread_run_start_routine(void *arg) +static void *_odph_thread_run_start_routine(void *arg) { int status; int ret; @@ -287,9 +67,9 @@ static void *odpthread_run_start_routine(void *arg) /* * Create a single ODPthread as a linux process */ -static int odph_linux_process_create(odph_odpthread_t *thread_tbl, - int cpu, - const odph_odpthread_params_t *thr_params) +static int _odph_linux_process_create(odph_odpthread_t *thread_tbl, + int cpu, + const odph_odpthread_params_t *thr_params) { cpu_set_t cpu_set; pid_t pid; @@ -327,7 +107,7 @@ static int odph_linux_process_create(odph_odpthread_t *thread_tbl, return -2; } - odpthread_run_start_routine(&thread_tbl->start_args); + _odph_thread_run_start_routine(&thread_tbl->start_args); return 0; /* never reached */ } @@ -357,7 +137,7 @@ static int odph_linux_thread_create(odph_odpthread_t *thread_tbl, ret = pthread_create(&thread_tbl->thread.thread_id, &thread_tbl->thread.attr, - odpthread_run_start_routine, + _odph_thread_run_start_routine, &thread_tbl->start_args); if (ret != 0) { ODPH_ERR("Failed to start thread on cpu #%d\n", cpu); @@ -401,9 +181,9 @@ int odph_odpthreads_create(odph_odpthread_t *thread_tbl, thr_params)) break; } else { - if (odph_linux_process_create(&thread_tbl[i], - cpu, - thr_params)) + if (_odph_linux_process_create(&thread_tbl[i], + cpu, + thr_params)) break; } diff --git a/pkgconfig/libodphelper-linux.pc.in b/pkgconfig/libodphelper-linux-generic.pc.in similarity index 72% rename from pkgconfig/libodphelper-linux.pc.in rename to pkgconfig/libodphelper-linux-generic.pc.in index 3987f4c..cab7be2 100644 --- a/pkgconfig/libodphelper-linux.pc.in +++ b/pkgconfig/libodphelper-linux-generic.pc.in @@ -3,9 +3,9 @@ exec_prefix=@exec_prefix@ libdir=@libdir@ includedir=@includedir@ -Name: libodphelper-linux +Name: libodphelper-linux-generic Description: Helper for the ODP packet processing engine Version: @PKGCONFIG_VERSION@ -Libs: -L${libdir} -lodphelper-linux +Libs: -L${libdir} -lodphelper-linux-generic Libs.private: Cflags: -I${includedir} diff --git a/test/Makefile.inc b/test/Makefile.inc index 1ebc047..243a616 100644 --- a/test/Makefile.inc +++ b/test/Makefile.inc @@ -4,7 +4,7 @@ LIB = $(top_builddir)/lib #in the following line, the libs using the symbols should come before #the libs containing them! The includer is given a chance to add things #before libodp by setting PRE_LDADD before the inclusion. -LDADD = $(PRE_LDADD) $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LDADD = $(PRE_LDADD) $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la INCFLAGS = \ -I$(top_builddir)/platform/@with_platform@/include \ diff --git a/test/common_plat/validation/api/Makefile.inc b/test/common_plat/validation/api/Makefile.inc index ffba620..a0afd26 100644 --- a/test/common_plat/validation/api/Makefile.inc +++ b/test/common_plat/validation/api/Makefile.inc @@ -13,4 +13,4 @@ AM_LDFLAGS += -static LIBCUNIT_COMMON = $(COMMON_DIR)/libcunit_common.la LIBCPUMASK_COMMON = $(COMMON_DIR)/libcpumask_common.la LIBTHRMASK_COMMON = $(COMMON_DIR)/libthrmask_common.la -LIBODP = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LIBODP = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la diff --git a/test/linux-generic/Makefile.inc b/test/linux-generic/Makefile.inc index 36745fe..2a49076 100644 --- a/test/linux-generic/Makefile.inc +++ b/test/linux-generic/Makefile.inc @@ -6,7 +6,7 @@ AM_LDFLAGS += -static LIBCUNIT_COMMON = $(top_builddir)/test/common_plat/common/libcunit_common.la LIB = $(top_builddir)/lib -LIBODP = $(LIB)/libodphelper-linux.la $(LIB)/libodp-linux.la +LIBODP = $(LIB)/libodphelper-@with_helper_platform@.la $(LIB)/libodp-linux.la INCCUNIT_COMMON = -I$(top_srcdir)/test/common_plat/common INCODP = \