| Message ID | 20250508125011.3455696-1-zhuwei@sangfor.com.cn |
|---|---|
| State | New |
| Headers | show |
| Series | scsi: smartpqi: Fix the race condition between pqi_tmf_worker and pqi_sdev_destroy | expand |
---------- Forwarded message --------- From: Zhu Wei <zhuwei@sangfor.com.cn> Date: Thu, May 8, 2025 at 7:57 AM Subject: [PATCH] scsi: smartpqi: Fix the race condition between pqi_tmf_worker and pqi_sdev_destroy To: <don.brace@microchip.com>, <kevin.barnett@microchip.com> Cc: <dinghui@sangfor.com.cn>, <zengzhicong@sangfor.com.cn>, <James.Bottomley@hansenpartnership.com>, <martin.petersen@oracle.com>, <storagedev@microchip.com>, <linux-scsi@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <stable@vger.kernel.org>, Zhu Wei <zhuwei@sangfor.com.cn> There is a race condition between pqi_sdev_destroy and pqi_tmf_worker. After pqi_free_device is released, pqi_tmf_worker will still use device. Don: Thank-you for your patch, however we recently applied a similar patch to our internal repo. Don: But more checking is done for removed devices. Don: When this patch has been tested internally, we will post it up for review. Don: I will add a Reported-By tag with your name. Don: So Nak. kasan report: [ 1933.765810] ================================================================== [ 1933.771862] scsi 15:0:20:0: Direct-Access ATA WDC WUH722222AL WTS2 PQ: 0 ANSI: 6 [ 1933.779190] BUG: KASAN: use-after-free in pqi_device_wait_for_pending_io+0x9e/0x600 [smartpqi] [ 1933.779194] Read of size 4 at addr ffff88954c490480 by task kworker/2:2/518186 [ 1933.779201] CPU: 2 PID: 518186 Comm: kworker/2:2 Kdump: loaded Tainted: G U OE 4.19.90-89.16.v2401.osc.sfc.6.11.1.0108.ky10.x86_64+debug #1 [ 1933.779203] Source Version: v6.11.1.0108+0~65323201.20250328 [ 1933.779205] Hardware name: SANGFOR S2122-S12L/ASERVER-P-2000, BIOS TYR.2.00.0100 05/18/2019 [ 1933.779213] Workqueue: events pqi_tmf_worker [smartpqi] [ 1933.779216] Call Trace: [ 1933.779225] dump_stack+0x8b/0xb9 [ 1933.779239] print_address_description+0x65/0x2b0 [ 1933.779249] kasan_report+0x14b/0x290 [ 1933.779255] pqi_device_wait_for_pending_io+0x9e/0x600 [smartpqi] [ 1933.779264] pqi_device_reset_handler+0x174f/0x1f30 [smartpqi] [ 1933.779284] process_one_work+0x65f/0x12d0 [ 1933.806306] worker_thread+0x87/0xb50 [ 1933.806315] kthread+0x2e9/0x3a0 [ 1933.806323] ret_from_fork+0x1f/0x40 [ 1933.843994] Allocated by task 579094: [ 1933.875361] save_stack+0x19/0x80 [ 1933.892366] kasan_kmalloc+0xa0/0xd0 [ 1933.892373] kmem_cache_alloc+0xbb/0x1c0 [ 1933.892378] getname_flags+0xba/0x500 [ 1933.892384] user_path_at_empty+0x1d/0x40 [ 1933.892389] vfs_statx+0xb9/0x140 [ 1933.892397] __do_sys_newstat+0x77/0xd0 [ 1933.913179] do_syscall_64+0xa4/0x430 [ 1933.913187] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [ 1933.933381] Freed by task 579094: [ 1933.933389] save_stack+0x19/0x80 [ 1933.933392] __kasan_slab_free+0x130/0x180 [ 1933.933396] kmem_cache_free+0x78/0x1e0 [ 1933.933401] filename_lookup+0x216/0x400 [ 1933.933405] vfs_statx+0xb9/0x140 [ 1933.933407] __do_sys_newstat+0x77/0xd0 [ 1933.933417] do_syscall_64+0xa4/0x430 [ 1933.933432] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [ 1933.956837] [ 1933.956842] The buggy address belongs to the object at ffff88954c490000 which belongs to the cache names_cache of size 4096 [ 1933.956845] The buggy address is located 1152 bytes inside of 4096-byte region [ffff88954c490000, ffff88954c491000) [ 1933.956851] The buggy address belongs to the page: [ 1934.297352] page:ffffea0055312400 count:1 mapcount:0 mapping:ffff888100580f00 index:0x0 compound_mapcount: 0 [ 1934.309566] flags: 0x57ffffc0008100(slab|head) [ 1934.316370] raw: 0057ffffc0008100 0000000000000000 dead000000000200 ffff888100580f00 [ 1934.326518] raw: 0000000000000000 0000000000070007 00000001ffffffff 0000000000000000 [ 1934.338191] page dumped because: kasan: bad access detected [ 1934.346177] [ 1934.350031] Memory state around the buggy address: [ 1934.357220] ffff88954c490380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.366854] ffff88954c490400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.376474] >ffff88954c490480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.386094] ^ [ 1934.391684] ffff88954c490500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.402601] ffff88954c490580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.412228] ================================================================== Before pqi_sdev_destroy executes pqi_free_device, cancel the pqi_tmf_worker of the corresponding device. Fixes: 2d80f4054f7f ("scsi: smartpqi: Update deleting a LUN via sysfs") Signed-off-by: Zhu Wei <zhuwei@sangfor.com.cn> --- drivers/scsi/smartpqi/smartpqi_init.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/scsi/smartpqi/smartpqi_init.c b/drivers/scsi/smartpqi/smartpqi_init.c index 8a26eca4fdc9..102ed7501f08 100644 --- a/drivers/scsi/smartpqi/smartpqi_init.c +++ b/drivers/scsi/smartpqi/smartpqi_init.c @@ -6581,6 +6581,8 @@ static void pqi_sdev_destroy(struct scsi_device *sdev) struct pqi_scsi_dev *device; int mutex_acquired; unsigned long flags; + unsigned int lun; + struct pqi_tmf_work *tmf_work; ctrl_info = shost_to_hba(sdev->host); @@ -6607,6 +6609,8 @@ static void pqi_sdev_destroy(struct scsi_device *sdev) mutex_unlock(&ctrl_info->scan_mutex); pqi_dev_info(ctrl_info, "removed", device); + for (lun = 0, tmf_work = device->tmf_work; lun < PQI_MAX_LUNS_PER_DEVICE; lun++, tmf_work++) + cancel_work_sync(&tmf_work->work_struct); pqi_free_device(device); } -- 2.43.0
On 2025/5/9 23:19, Don.Brace@microchip.com wrote: > > ---------- Forwarded message --------- > From: Zhu Wei <zhuwei@sangfor.com.cn> > Date: Thu, May 8, 2025 at 7:57 AM > Subject: [PATCH] scsi: smartpqi: Fix the race condition between pqi_tmf_worker and pqi_sdev_destroy > To: <don.brace@microchip.com>, <kevin.barnett@microchip.com> > Cc: <dinghui@sangfor.com.cn>, <zengzhicong@sangfor.com.cn>, <James.Bottomley@hansenpartnership.com>, <martin.petersen@oracle.com>, <storagedev@microchip.com>, <linux-scsi@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <stable@vger.kernel.org>, Zhu Wei <zhuwei@sangfor.com.cn> > > > There is a race condition between pqi_sdev_destroy and pqi_tmf_worker. > After pqi_free_device is released, pqi_tmf_worker will still use device. > > Don: Thank-you for your patch, however we recently applied a similar patch to our internal repo. > Don: But more checking is done for removed devices. > Don: When this patch has been tested internally, we will post it up for review. > Don: I will add a Reported-By tag with your name. Ok, hope smartpqi gets better. > Don: So Nak. > > > > kasan report: > [ 1933.765810] ================================================================== > [ 1933.771862] scsi 15:0:20:0: Direct-Access ATA WDC WUH722222AL WTS2 PQ: 0 ANSI: 6 > [ 1933.779190] BUG: KASAN: use-after-free in pqi_device_wait_for_pending_io+0x9e/0x600 [smartpqi] > ...... > -- > 2.43.0 >
diff --git a/drivers/scsi/smartpqi/smartpqi_init.c b/drivers/scsi/smartpqi/smartpqi_init.c index 8a26eca4fdc9..102ed7501f08 100644 --- a/drivers/scsi/smartpqi/smartpqi_init.c +++ b/drivers/scsi/smartpqi/smartpqi_init.c @@ -6581,6 +6581,8 @@ static void pqi_sdev_destroy(struct scsi_device *sdev) struct pqi_scsi_dev *device; int mutex_acquired; unsigned long flags; + unsigned int lun; + struct pqi_tmf_work *tmf_work; ctrl_info = shost_to_hba(sdev->host); @@ -6607,6 +6609,8 @@ static void pqi_sdev_destroy(struct scsi_device *sdev) mutex_unlock(&ctrl_info->scan_mutex); pqi_dev_info(ctrl_info, "removed", device); + for (lun = 0, tmf_work = device->tmf_work; lun < PQI_MAX_LUNS_PER_DEVICE; lun++, tmf_work++) + cancel_work_sync(&tmf_work->work_struct); pqi_free_device(device); }
There is a race condition between pqi_sdev_destroy and pqi_tmf_worker. After pqi_free_device is released, pqi_tmf_worker will still use device. kasan report: [ 1933.765810] ================================================================== [ 1933.771862] scsi 15:0:20:0: Direct-Access ATA WDC WUH722222AL WTS2 PQ: 0 ANSI: 6 [ 1933.779190] BUG: KASAN: use-after-free in pqi_device_wait_for_pending_io+0x9e/0x600 [smartpqi] [ 1933.779194] Read of size 4 at addr ffff88954c490480 by task kworker/2:2/518186 [ 1933.779201] CPU: 2 PID: 518186 Comm: kworker/2:2 Kdump: loaded Tainted: G U OE 4.19.90-89.16.v2401.osc.sfc.6.11.1.0108.ky10.x86_64+debug #1 [ 1933.779203] Source Version: v6.11.1.0108+0~65323201.20250328 [ 1933.779205] Hardware name: SANGFOR S2122-S12L/ASERVER-P-2000, BIOS TYR.2.00.0100 05/18/2019 [ 1933.779213] Workqueue: events pqi_tmf_worker [smartpqi] [ 1933.779216] Call Trace: [ 1933.779225] dump_stack+0x8b/0xb9 [ 1933.779239] print_address_description+0x65/0x2b0 [ 1933.779249] kasan_report+0x14b/0x290 [ 1933.779255] pqi_device_wait_for_pending_io+0x9e/0x600 [smartpqi] [ 1933.779264] pqi_device_reset_handler+0x174f/0x1f30 [smartpqi] [ 1933.779284] process_one_work+0x65f/0x12d0 [ 1933.806306] worker_thread+0x87/0xb50 [ 1933.806315] kthread+0x2e9/0x3a0 [ 1933.806323] ret_from_fork+0x1f/0x40 [ 1933.843994] Allocated by task 579094: [ 1933.875361] save_stack+0x19/0x80 [ 1933.892366] kasan_kmalloc+0xa0/0xd0 [ 1933.892373] kmem_cache_alloc+0xbb/0x1c0 [ 1933.892378] getname_flags+0xba/0x500 [ 1933.892384] user_path_at_empty+0x1d/0x40 [ 1933.892389] vfs_statx+0xb9/0x140 [ 1933.892397] __do_sys_newstat+0x77/0xd0 [ 1933.913179] do_syscall_64+0xa4/0x430 [ 1933.913187] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [ 1933.933381] Freed by task 579094: [ 1933.933389] save_stack+0x19/0x80 [ 1933.933392] __kasan_slab_free+0x130/0x180 [ 1933.933396] kmem_cache_free+0x78/0x1e0 [ 1933.933401] filename_lookup+0x216/0x400 [ 1933.933405] vfs_statx+0xb9/0x140 [ 1933.933407] __do_sys_newstat+0x77/0xd0 [ 1933.933417] do_syscall_64+0xa4/0x430 [ 1933.933432] entry_SYSCALL_64_after_hwframe+0x5c/0xc1 [ 1933.956837] [ 1933.956842] The buggy address belongs to the object at ffff88954c490000 which belongs to the cache names_cache of size 4096 [ 1933.956845] The buggy address is located 1152 bytes inside of 4096-byte region [ffff88954c490000, ffff88954c491000) [ 1933.956851] The buggy address belongs to the page: [ 1934.297352] page:ffffea0055312400 count:1 mapcount:0 mapping:ffff888100580f00 index:0x0 compound_mapcount: 0 [ 1934.309566] flags: 0x57ffffc0008100(slab|head) [ 1934.316370] raw: 0057ffffc0008100 0000000000000000 dead000000000200 ffff888100580f00 [ 1934.326518] raw: 0000000000000000 0000000000070007 00000001ffffffff 0000000000000000 [ 1934.338191] page dumped because: kasan: bad access detected [ 1934.346177] [ 1934.350031] Memory state around the buggy address: [ 1934.357220] ffff88954c490380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.366854] ffff88954c490400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.376474] >ffff88954c490480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.386094] ^ [ 1934.391684] ffff88954c490500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.402601] ffff88954c490580: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 1934.412228] ================================================================== Before pqi_sdev_destroy executes pqi_free_device, cancel the pqi_tmf_worker of the corresponding device. Fixes: 2d80f4054f7f ("scsi: smartpqi: Update deleting a LUN via sysfs") Signed-off-by: Zhu Wei <zhuwei@sangfor.com.cn> --- drivers/scsi/smartpqi/smartpqi_init.c | 4 ++++ 1 file changed, 4 insertions(+)