| Message ID | 20181210211957.210189-1-arnd@arndb.de |
|---|---|
| State | Superseded |
| Headers | show |
| Series | test_rhashtable: remove semaphore usage | expand |
From: Arnd Bergmann <arnd@arndb.de> Date: Mon, 10 Dec 2018 22:17:20 +0100 > This is one of only two files that initialize a semaphore to a negative > value. We don't really need the two semaphores here at all, but can do > the same thing in more conventional and more effient way, by using a > single waitqueue and an atomic thread counter. > > This gets us a little bit closer to eliminating classic semaphores from > the kernel. It also fixes a corner case where we fail to continue after > one of the threads fails to start up. > > An alternative would be to use a split kthread_create()+wake_up_process() > and completely eliminate the separate synchronization. > > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > --- > This is part of a longer, untested, series to remove semaphores > from the kernel, please review as such before applying. This looks fine to me, although it seems kinda weird how the synchronization works in that the N - 1 child threads will be awoken two times, once when child N decrements the count to zero and once when the parent decrements the count to -1 which lets us past the wait_event_interruptibel(). Nevertheless it should work just fine and I have no problems with it. Want me to apply this to net-next? Thanks.
On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote: > This is one of only two files that initialize a semaphore to a negative > value. We don't really need the two semaphores here at all, but can do > the same thing in more conventional and more effient way, by using a > single waitqueue and an atomic thread counter. > > This gets us a little bit closer to eliminating classic semaphores from > the kernel. It also fixes a corner case where we fail to continue after > one of the threads fails to start up. > > An alternative would be to use a split kthread_create()+wake_up_process() > and completely eliminate the separate synchronization. > > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > --- > This is part of a longer, untested, series to remove semaphores > from the kernel, please review as such before applying. > --- > lib/test_rhashtable.c | 28 ++++++++++++++++------------ > 1 file changed, 16 insertions(+), 12 deletions(-) This was created by Phil Sutter so I am adding him to the cc list. > diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c > index 18de5ff1255b..12bdea4f6c20 100644 > --- a/lib/test_rhashtable.c > +++ b/lib/test_rhashtable.c > @@ -20,11 +20,11 @@ > #include <linux/module.h> > #include <linux/rcupdate.h> > #include <linux/rhashtable.h> > -#include <linux/semaphore.h> > #include <linux/slab.h> > #include <linux/sched.h> > #include <linux/random.h> > #include <linux/vmalloc.h> > +#include <linux/wait.h> > > #define MAX_ENTRIES 1000000 > #define TEST_INSERT_FAIL INT_MAX > @@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = { > .automatic_shrinking = false, > }; > > -static struct semaphore prestart_sem, startup_sem; > +static atomic_t startup_count; > +static DECLARE_WAIT_QUEUE_HEAD(startup_wait); > > static int insert_retry(struct rhashtable *ht, struct test_obj *obj, > const struct rhashtable_params params) > @@ -635,8 +636,9 @@ static int threadfunc(void *data) > int i, step, err = 0, insert_retries = 0; > struct thread_data *tdata = data; > > - up(&prestart_sem); > - if (down_interruptible(&startup_sem)) > + if (atomic_dec_and_test(&startup_count)) > + wake_up(&startup_wait); > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1)) > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id); > > for (i = 0; i < tdata->entries; i++) { > @@ -756,8 +758,7 @@ static int __init test_rht_init(void) > > pr_info("Testing concurrent rhashtable access from %d threads\n", > tcount); > - sema_init(&prestart_sem, 1 - tcount); > - sema_init(&startup_sem, 0); > + atomic_set(&startup_count, tcount); > tdata = vzalloc(array_size(tcount, sizeof(struct thread_data))); > if (!tdata) > return -ENOMEM; > @@ -783,15 +784,18 @@ static int __init test_rht_init(void) > tdata[i].objs = objs + i * entries; > tdata[i].task = kthread_run(threadfunc, &tdata[i], > "rhashtable_thrad[%d]", i); > - if (IS_ERR(tdata[i].task)) > + if (IS_ERR(tdata[i].task)) { > pr_err(" kthread_run failed for thread %d\n", i); > - else > + atomic_dec(&startup_count); > + } else { > started_threads++; > + } > } > - if (down_interruptible(&prestart_sem)) > - pr_err(" down interruptible failed\n"); > - for (i = 0; i < tcount; i++) > - up(&startup_sem); > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0)) > + pr_err(" wait_event interruptible failed\n"); > + /* count is 0 now, set it to -1 and wake up all threads together */ > + atomic_dec(&startup_count); > + wake_up_all(&startup_wait); > for (i = 0; i < tcount; i++) { > if (IS_ERR(tdata[i].task)) > continue; > -- > 2.20.0 > -- Email: Herbert Xu <herbert@gondor.apana.org.au> Home Page: http://gondor.apana.org.au/~herbert/ PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Hi, On Tue, Dec 11, 2018 at 01:45:52PM +0800, Herbert Xu wrote: > On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote: > > This is one of only two files that initialize a semaphore to a negative > > value. We don't really need the two semaphores here at all, but can do > > the same thing in more conventional and more effient way, by using a > > single waitqueue and an atomic thread counter. > > > > This gets us a little bit closer to eliminating classic semaphores from > > the kernel. It also fixes a corner case where we fail to continue after > > one of the threads fails to start up. > > > > An alternative would be to use a split kthread_create()+wake_up_process() > > and completely eliminate the separate synchronization. > > > > Signed-off-by: Arnd Bergmann <arnd@arndb.de> > > --- > > This is part of a longer, untested, series to remove semaphores > > from the kernel, please review as such before applying. > > --- > > lib/test_rhashtable.c | 28 ++++++++++++++++------------ > > 1 file changed, 16 insertions(+), 12 deletions(-) > > This was created by Phil Sutter so I am adding him to the cc list. Thanks, I would have missed it otherwise. Just a minor nit: > > diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c > > index 18de5ff1255b..12bdea4f6c20 100644 > > --- a/lib/test_rhashtable.c > > +++ b/lib/test_rhashtable.c [...] > > @@ -635,8 +636,9 @@ static int threadfunc(void *data) > > int i, step, err = 0, insert_retries = 0; > > struct thread_data *tdata = data; > > > > - up(&prestart_sem); > > - if (down_interruptible(&startup_sem)) > > + if (atomic_dec_and_test(&startup_count)) > > + wake_up(&startup_wait); > > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1)) > > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id); The error message should probably be adjusted as well. Apart from that: Acked-by: Phil Sutter <phil@nwl.cc> Thanks, Phil
From: Arnd Bergmann <arnd@arndb.de> Date: Mon, 10 Dec 2018 22:17:20 +0100 > @@ -635,8 +636,9 @@ static int threadfunc(void *data) > int i, step, err = 0, insert_retries = 0; > struct thread_data *tdata = data; > > - up(&prestart_sem); > - if (down_interruptible(&startup_sem)) > + if (atomic_dec_and_test(&startup_count)) > + wake_up(&startup_wait); > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1)) > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id); Arnd, please adjust this pr_err() text to match the new code as pointed out by Phil Sutter. Thank you.
On Fri, Dec 14, 2018 at 10:25 PM David Miller <davem@davemloft.net> wrote: > > From: Arnd Bergmann <arnd@arndb.de> > Date: Mon, 10 Dec 2018 22:17:20 +0100 > > > @@ -635,8 +636,9 @@ static int threadfunc(void *data) > > int i, step, err = 0, insert_retries = 0; > > struct thread_data *tdata = data; > > > > - up(&prestart_sem); > > - if (down_interruptible(&startup_sem)) > > + if (atomic_dec_and_test(&startup_count)) > > + wake_up(&startup_wait); > > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1)) > > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id); > > Arnd, please adjust this pr_err() text to match the new code as pointed out by > Phil Sutter. Done, and resent as v2. I also noticed that the version I sent did not apply cleanly on current kernels, so I fixed that, and also added a 'goto out' in the kthread after printing the error message, so an interrupted thread would exist straight away. Arnd
diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c index 18de5ff1255b..12bdea4f6c20 100644 --- a/lib/test_rhashtable.c +++ b/lib/test_rhashtable.c @@ -20,11 +20,11 @@ #include <linux/module.h> #include <linux/rcupdate.h> #include <linux/rhashtable.h> -#include <linux/semaphore.h> #include <linux/slab.h> #include <linux/sched.h> #include <linux/random.h> #include <linux/vmalloc.h> +#include <linux/wait.h> #define MAX_ENTRIES 1000000 #define TEST_INSERT_FAIL INT_MAX @@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = { .automatic_shrinking = false, }; -static struct semaphore prestart_sem, startup_sem; +static atomic_t startup_count; +static DECLARE_WAIT_QUEUE_HEAD(startup_wait); static int insert_retry(struct rhashtable *ht, struct test_obj *obj, const struct rhashtable_params params) @@ -635,8 +636,9 @@ static int threadfunc(void *data) int i, step, err = 0, insert_retries = 0; struct thread_data *tdata = data; - up(&prestart_sem); - if (down_interruptible(&startup_sem)) + if (atomic_dec_and_test(&startup_count)) + wake_up(&startup_wait); + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1)) pr_err(" thread[%d]: down_interruptible failed\n", tdata->id); for (i = 0; i < tdata->entries; i++) { @@ -756,8 +758,7 @@ static int __init test_rht_init(void) pr_info("Testing concurrent rhashtable access from %d threads\n", tcount); - sema_init(&prestart_sem, 1 - tcount); - sema_init(&startup_sem, 0); + atomic_set(&startup_count, tcount); tdata = vzalloc(array_size(tcount, sizeof(struct thread_data))); if (!tdata) return -ENOMEM; @@ -783,15 +784,18 @@ static int __init test_rht_init(void) tdata[i].objs = objs + i * entries; tdata[i].task = kthread_run(threadfunc, &tdata[i], "rhashtable_thrad[%d]", i); - if (IS_ERR(tdata[i].task)) + if (IS_ERR(tdata[i].task)) { pr_err(" kthread_run failed for thread %d\n", i); - else + atomic_dec(&startup_count); + } else { started_threads++; + } } - if (down_interruptible(&prestart_sem)) - pr_err(" down interruptible failed\n"); - for (i = 0; i < tcount; i++) - up(&startup_sem); + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0)) + pr_err(" wait_event interruptible failed\n"); + /* count is 0 now, set it to -1 and wake up all threads together */ + atomic_dec(&startup_count); + wake_up_all(&startup_wait); for (i = 0; i < tcount; i++) { if (IS_ERR(tdata[i].task)) continue;
This is one of only two files that initialize a semaphore to a negative value. We don't really need the two semaphores here at all, but can do the same thing in more conventional and more effient way, by using a single waitqueue and an atomic thread counter. This gets us a little bit closer to eliminating classic semaphores from the kernel. It also fixes a corner case where we fail to continue after one of the threads fails to start up. An alternative would be to use a split kthread_create()+wake_up_process() and completely eliminate the separate synchronization. Signed-off-by: Arnd Bergmann <arnd@arndb.de> --- This is part of a longer, untested, series to remove semaphores from the kernel, please review as such before applying. --- lib/test_rhashtable.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) -- 2.20.0