From patchwork Thu May 1 15:49:34 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 29509 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f197.google.com (mail-ve0-f197.google.com [209.85.128.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2EF8E20B8D for ; Thu, 1 May 2014 15:49:58 +0000 (UTC) Received: by mail-ve0-f197.google.com with SMTP id sa20sf11382412veb.4 for ; Thu, 01 May 2014 08:49:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=hnRNt/WeV6M8WoUaLDQMFKmXiGxxIzD7L5KtM56WOP4=; b=RhEFgMrT/r59t0di1c1H06JKYbvMdWFle2R2zi9bdFD169vyg11CQJBnLJrZYnLXDU vMd47xq4UH8y6hz26MZLP2/pZys8nfM26u9xOb8H5SoksyvObb4xbRQO6kbACqjme0+b lO8WGcvcKTu9Bc79GNLaW1/9tgzqMtq5glNMR9/XM3dqUFsDccdBMYvPK8abLea+tOvQ QVZHcWL2BpdA4/tml0QjiWnwpiE6ndYDyWXJdvdheiKHO210c8Sy085Z9CqNnnG5f0gD xCwwjOMxALCiE8rGr65cAC088Gw3JjaWJw6cOu6UqQX12kRB80lRdsfHaEP/EkEvAtPl 38YA== X-Gm-Message-State: ALoCoQm2PvNbEKu6R/rDYIGH+L5gi5LfHKgDY5jM2yymR5eVBY674u/YAfOdm95qKbd24ujDag0Y X-Received: by 10.236.171.169 with SMTP id r29mr5223635yhl.32.1398959398633; Thu, 01 May 2014 08:49:58 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.86.170 with SMTP id p39ls1093708qgd.49.gmail; Thu, 01 May 2014 08:49:58 -0700 (PDT) X-Received: by 10.58.77.238 with SMTP id v14mr2092641vew.27.1398959398493; Thu, 01 May 2014 08:49:58 -0700 (PDT) Received: from mail-ve0-f169.google.com (mail-ve0-f169.google.com [209.85.128.169]) by mx.google.com with ESMTPS id p8si6131551vef.14.2014.05.01.08.49.58 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 01 May 2014 08:49:58 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.128.169; Received: by mail-ve0-f169.google.com with SMTP id jx11so4178608veb.28 for ; Thu, 01 May 2014 08:49:58 -0700 (PDT) X-Received: by 10.52.166.18 with SMTP id zc18mr372647vdb.65.1398959398326; Thu, 01 May 2014 08:49:58 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp33990vcb; Thu, 1 May 2014 08:49:57 -0700 (PDT) X-Received: by 10.50.79.226 with SMTP id m2mr3920252igx.11.1398959396200; Thu, 01 May 2014 08:49:56 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id to1si17435586pab.240.2014.05.01.08.49.55; Thu, 01 May 2014 08:49:55 -0700 (PDT) Received-SPF: none (google.com: linux-crypto-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754014AbaEAPty (ORCPT ); Thu, 1 May 2014 11:49:54 -0400 Received: from mail-we0-f174.google.com ([74.125.82.174]:65373 "EHLO mail-we0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751833AbaEAPtw (ORCPT ); Thu, 1 May 2014 11:49:52 -0400 Received: by mail-we0-f174.google.com with SMTP id k48so3374361wev.33 for ; Thu, 01 May 2014 08:49:51 -0700 (PDT) X-Received: by 10.180.210.238 with SMTP id mx14mr2625837wic.61.1398959390966; Thu, 01 May 2014 08:49:50 -0700 (PDT) Received: from ards-macbook-pro.local (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by mx.google.com with ESMTPSA id jd2sm4457326wic.9.2014.05.01.08.49.49 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 01 May 2014 08:49:50 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, linux-crypto@vger.kernel.org Cc: catalin.marinas@arm.com, will.deacon@arm.com, steve.capper@linaro.org, Ard Biesheuvel Subject: [PATCH resend 02/15] arm64: add abstractions for FPSIMD state manipulation Date: Thu, 1 May 2014 17:49:34 +0200 Message-Id: <1398959381-8126-3-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1398959381-8126-1-git-send-email-ard.biesheuvel@linaro.org> References: <1398959381-8126-1-git-send-email-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , There are two tacit assumptions in the FPSIMD handling code that will no longer hold after the next patch that optimizes away some FPSIMD state restores: . the FPSIMD registers of this CPU contain the userland FPSIMD state of task 'current'; . when switching to a task, its FPSIMD state will always be restored from memory. This patch adds the following functions to abstract away from straight FPSIMD register file saves and restores: - fpsimd_preserve_current_state -> ensure current's FPSIMD state is saved - fpsimd_restore_current_state -> ensure current's FPSIMD state is loaded - fpsimd_update_current_state -> replace current's FPSIMD state - fpsimd_flush_task_state -> invalidate live copies of a task's FPSIMD state Where necessary, the ptrace, signal handling and fork code are updated to use the above wrappers instead of poking into the FPSIMD registers directly. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/fpsimd.h | 6 ++++++ arch/arm64/kernel/fpsimd.c | 33 +++++++++++++++++++++++++++++++++ arch/arm64/kernel/process.c | 2 +- arch/arm64/kernel/ptrace.c | 2 ++ arch/arm64/kernel/signal.c | 9 +++------ arch/arm64/kernel/signal32.c | 9 +++------ 6 files changed, 48 insertions(+), 13 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index c43b4ac13008..9f9b8e438546 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -58,6 +58,12 @@ extern void fpsimd_load_state(struct fpsimd_state *state); extern void fpsimd_thread_switch(struct task_struct *next); extern void fpsimd_flush_thread(void); +extern void fpsimd_preserve_current_state(void); +extern void fpsimd_restore_current_state(void); +extern void fpsimd_update_current_state(struct fpsimd_state *state); + +extern void fpsimd_flush_task_state(struct task_struct *target); + #endif #endif diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 4aef42a04bdc..86ac6a9bc86a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -87,6 +87,39 @@ void fpsimd_flush_thread(void) preempt_enable(); } +/* + * Save the userland FPSIMD state of 'current' to memory + */ +void fpsimd_preserve_current_state(void) +{ + fpsimd_save_state(¤t->thread.fpsimd_state); +} + +/* + * Load the userland FPSIMD state of 'current' from memory + */ +void fpsimd_restore_current_state(void) +{ + fpsimd_load_state(¤t->thread.fpsimd_state); +} + +/* + * Load an updated userland FPSIMD state for 'current' from memory + */ +void fpsimd_update_current_state(struct fpsimd_state *state) +{ + preempt_disable(); + fpsimd_load_state(state); + preempt_enable(); +} + +/* + * Invalidate live CPU copies of task t's FPSIMD state + */ +void fpsimd_flush_task_state(struct task_struct *t) +{ +} + #ifdef CONFIG_KERNEL_MODE_NEON /* diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 6391485f342d..c5693163408c 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -205,7 +205,7 @@ void release_thread(struct task_struct *dead_task) int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) { - fpsimd_save_state(¤t->thread.fpsimd_state); + fpsimd_preserve_current_state(); *dst = *src; return 0; } diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 6a8928bba03c..f8700eca24e7 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -517,6 +517,7 @@ static int fpr_set(struct task_struct *target, const struct user_regset *regset, return ret; target->thread.fpsimd_state.user_fpsimd = newstate; + fpsimd_flush_task_state(target); return ret; } @@ -764,6 +765,7 @@ static int compat_vfp_set(struct task_struct *target, uregs->fpcr = fpscr & VFP_FPSCR_CTRL_MASK; } + fpsimd_flush_task_state(target); return ret; } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 890a591f75dd..06448a77ff53 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -51,7 +51,7 @@ static int preserve_fpsimd_context(struct fpsimd_context __user *ctx) int err; /* dump the hardware registers to the fpsimd_state structure */ - fpsimd_save_state(fpsimd); + fpsimd_preserve_current_state(); /* copy the FP and status/control registers */ err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs)); @@ -86,11 +86,8 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx) __get_user_error(fpsimd.fpcr, &ctx->fpcr, err); /* load the hardware registers from the fpsimd_state structure */ - if (!err) { - preempt_disable(); - fpsimd_load_state(&fpsimd); - preempt_enable(); - } + if (!err) + fpsimd_update_current_state(&fpsimd); return err ? -EFAULT : 0; } diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c index b3fc9f5ec6d3..ac7e237d0bda 100644 --- a/arch/arm64/kernel/signal32.c +++ b/arch/arm64/kernel/signal32.c @@ -219,7 +219,7 @@ static int compat_preserve_vfp_context(struct compat_vfp_sigframe __user *frame) * Note that this also saves V16-31, which aren't visible * in AArch32. */ - fpsimd_save_state(fpsimd); + fpsimd_preserve_current_state(); /* Place structure header on the stack */ __put_user_error(magic, &frame->magic, err); @@ -282,11 +282,8 @@ static int compat_restore_vfp_context(struct compat_vfp_sigframe __user *frame) * We don't need to touch the exception register, so * reload the hardware state. */ - if (!err) { - preempt_disable(); - fpsimd_load_state(&fpsimd); - preempt_enable(); - } + if (!err) + fpsimd_update_current_state(&fpsimd); return err ? -EFAULT : 0; }