| Message ID | 20240119094944.26763-9-benjamin.gaignard@collabora.com |
|---|---|
| State | New |
| Headers | show |
| Series | Add DELETE_BUF ioctl | expand |
On 19/01/2024 10:49, Benjamin Gaignard wrote: > Allow to delete buffers on capture queue because it the one which > own the decoded buffers. After a dynamic resolution change lot of > them could remain allocated but won't be used anymore so deleting > them save memory. > Do not add this feature on output queue because the buffers are > smaller, fewer and always recycled even after a dynamic resolution > change. > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> > --- > drivers/media/platform/verisilicon/hantro_drv.c | 1 + > drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + > 2 files changed, 2 insertions(+) > > diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c > index db3df6cc4513..f6b0a676a740 100644 > --- a/drivers/media/platform/verisilicon/hantro_drv.c > +++ b/drivers/media/platform/verisilicon/hantro_drv.c > @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) > dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; > dst_vq->lock = &ctx->dev->vpu_mutex; > dst_vq->dev = ctx->dev->v4l2_dev.dev; > + src_vq->supports_delete_bufs = true; As I mentioned, I remain unconvinced by this. It is just making the API inconsistent since if you support delete_bufs, then why support it for one queue only and not both? > > return vb2_queue_init(dst_vq); > } > diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c > index 941fa23c211a..34eab90e8a42 100644 > --- a/drivers/media/platform/verisilicon/hantro_v4l2.c > +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c > @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { > .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, > .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, > .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, > + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, > .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, > > .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, In my view setting vidioc_delete_bufs should enable this feature, and if for some strange reason only one queue support it, then make a wrapper callback that returns an error when used with the wrong queue. Also note that patch 6/8 never checks for q->supports_delete_bufs in vb2_core_delete_bufs(), which is wrong! Regards, Hans
On 24/01/2024 16:35, Benjamin Gaignard wrote: > > Le 24/01/2024 à 13:52, Hans Verkuil a écrit : >> On 19/01/2024 10:49, Benjamin Gaignard wrote: >>> Allow to delete buffers on capture queue because it the one which >>> own the decoded buffers. After a dynamic resolution change lot of >>> them could remain allocated but won't be used anymore so deleting >>> them save memory. >>> Do not add this feature on output queue because the buffers are >>> smaller, fewer and always recycled even after a dynamic resolution >>> change. >>> >>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> >>> --- >>> drivers/media/platform/verisilicon/hantro_drv.c | 1 + >>> drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + >>> 2 files changed, 2 insertions(+) >>> >>> diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c >>> index db3df6cc4513..f6b0a676a740 100644 >>> --- a/drivers/media/platform/verisilicon/hantro_drv.c >>> +++ b/drivers/media/platform/verisilicon/hantro_drv.c >>> @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) >>> dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; >>> dst_vq->lock = &ctx->dev->vpu_mutex; >>> dst_vq->dev = ctx->dev->v4l2_dev.dev; >>> + src_vq->supports_delete_bufs = true; >> As I mentioned, I remain unconvinced by this. It is just making the API inconsistent >> since if you support delete_bufs, then why support it for one queue only and not both? > > Because the both queues don't handle the same type of data. > For example for a stateless decoder, for me, it makes sense to allow delete decoded frames > if they won't be used anymore but that won't makes sense for bitstream buffers. But is there any reason why you wouldn't support this feature? We support VIDIOC_CREATE_BUFS as well, even though most drivers do not need it, but it is cheap to add. Deleting buffers is a generic feature, and I don't see why you wouldn't just offer it for both queues. Regards, Hans > >> >>> return vb2_queue_init(dst_vq); >>> } >>> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c >>> index 941fa23c211a..34eab90e8a42 100644 >>> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c >>> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c >>> @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { >>> .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, >>> .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, >>> .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, >>> + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, >>> .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, >>> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, >> In my view setting vidioc_delete_bufs should enable this feature, and if >> for some strange reason only one queue support it, then make a wrapper >> callback that returns an error when used with the wrong queue. >> >> Also note that patch 6/8 never checks for q->supports_delete_bufs in >> vb2_core_delete_bufs(), which is wrong! > > I will fix that in next version. > Regards, > Benjamin > >> >> Regards, >> >> Hans >>
Le 24/01/2024 à 16:44, Hans Verkuil a écrit : > On 24/01/2024 16:35, Benjamin Gaignard wrote: >> Le 24/01/2024 à 13:52, Hans Verkuil a écrit : >>> On 19/01/2024 10:49, Benjamin Gaignard wrote: >>>> Allow to delete buffers on capture queue because it the one which >>>> own the decoded buffers. After a dynamic resolution change lot of >>>> them could remain allocated but won't be used anymore so deleting >>>> them save memory. >>>> Do not add this feature on output queue because the buffers are >>>> smaller, fewer and always recycled even after a dynamic resolution >>>> change. >>>> >>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> >>>> --- >>>> drivers/media/platform/verisilicon/hantro_drv.c | 1 + >>>> drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + >>>> 2 files changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c >>>> index db3df6cc4513..f6b0a676a740 100644 >>>> --- a/drivers/media/platform/verisilicon/hantro_drv.c >>>> +++ b/drivers/media/platform/verisilicon/hantro_drv.c >>>> @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) >>>> dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; >>>> dst_vq->lock = &ctx->dev->vpu_mutex; >>>> dst_vq->dev = ctx->dev->v4l2_dev.dev; >>>> + src_vq->supports_delete_bufs = true; >>> As I mentioned, I remain unconvinced by this. It is just making the API inconsistent >>> since if you support delete_bufs, then why support it for one queue only and not both? >> Because the both queues don't handle the same type of data. >> For example for a stateless decoder, for me, it makes sense to allow delete decoded frames >> if they won't be used anymore but that won't makes sense for bitstream buffers. > But is there any reason why you wouldn't support this feature? We support VIDIOC_CREATE_BUFS > as well, even though most drivers do not need it, but it is cheap to add. > > Deleting buffers is a generic feature, and I don't see why you wouldn't just offer it > for both queues. You want me to remove supports_delete_bufs and V4L2_BUF_CAP_SUPPORTS_DELETE_BUFS ? This way we can remove buffers from the both queues. Sound good for you ? Regards, Benjamin > > Regards, > > Hans > >>>> return vb2_queue_init(dst_vq); >>>> } >>>> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> index 941fa23c211a..34eab90e8a42 100644 >>>> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { >>>> .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, >>>> .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, >>>> .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, >>>> + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, >>>> .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, >>>> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, >>> In my view setting vidioc_delete_bufs should enable this feature, and if >>> for some strange reason only one queue support it, then make a wrapper >>> callback that returns an error when used with the wrong queue. >>> >>> Also note that patch 6/8 never checks for q->supports_delete_bufs in >>> vb2_core_delete_bufs(), which is wrong! >> I will fix that in next version. >> Regards, >> Benjamin >> >>> Regards, >>> >>> Hans >>> >
Hi, Le mercredi 24 janvier 2024 à 16:35 +0100, Benjamin Gaignard a écrit : > Le 24/01/2024 à 13:52, Hans Verkuil a écrit : > > On 19/01/2024 10:49, Benjamin Gaignard wrote: > > > Allow to delete buffers on capture queue because it the one which > > > own the decoded buffers. After a dynamic resolution change lot of > > > them could remain allocated but won't be used anymore so deleting > > > them save memory. > > > Do not add this feature on output queue because the buffers are > > > smaller, fewer and always recycled even after a dynamic resolution > > > change. > > > > > > Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> > > > --- > > > drivers/media/platform/verisilicon/hantro_drv.c | 1 + > > > drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + > > > 2 files changed, 2 insertions(+) > > > > > > diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c > > > index db3df6cc4513..f6b0a676a740 100644 > > > --- a/drivers/media/platform/verisilicon/hantro_drv.c > > > +++ b/drivers/media/platform/verisilicon/hantro_drv.c > > > @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) > > > dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; > > > dst_vq->lock = &ctx->dev->vpu_mutex; > > > dst_vq->dev = ctx->dev->v4l2_dev.dev; > > > + src_vq->supports_delete_bufs = true; > > As I mentioned, I remain unconvinced by this. It is just making the API inconsistent > > since if you support delete_bufs, then why support it for one queue only and not both? > > Because the both queues don't handle the same type of data. > For example for a stateless decoder, for me, it makes sense to allow delete decoded frames > if they won't be used anymore but that won't makes sense for bitstream buffers. I personally think that for stateless and stateful decoder bitstream queue this can be useful. We currently guess the size we need, and there is no way to allocate bigger ones without the driver forgetting about reference frames. In stateful, some drivers allow to split the bitstream (I tested wave5 notably), but I was told this is not always the case. A bit of a gray zone in that API, with lack of capabilities to describe it. On stateless, the entire bitstream slice must be in one buffer. Though, for the asymmetry, most stateful decoders won't allow delete bufs on capture, because the buffers are registered in the firmware ahead of time. wave5 can't even implement create_bufs on capture. We had an argument about having create_bufs on only one queue for that specific driver, as we wanted something upstream, we flex to removing create bufs completely. I think the all or nothing rule on per queue create/delete_bufs is not aligned with the reality. Nicolas > > > > > > > > > return vb2_queue_init(dst_vq); > > > } > > > diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c > > > index 941fa23c211a..34eab90e8a42 100644 > > > --- a/drivers/media/platform/verisilicon/hantro_v4l2.c > > > +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c > > > @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { > > > .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, > > > .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, > > > .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, > > > + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, > > > .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, > > > > > > .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, > > In my view setting vidioc_delete_bufs should enable this feature, and if > > for some strange reason only one queue support it, then make a wrapper > > callback that returns an error when used with the wrong queue. > > > > Also note that patch 6/8 never checks for q->supports_delete_bufs in > > vb2_core_delete_bufs(), which is wrong! > > I will fix that in next version. > Regards, > Benjamin > > > > > Regards, > > > > Hans > > >
On 25/01/2024 17:27, Nicolas Dufresne wrote: > Hi, > > Le mercredi 24 janvier 2024 à 16:35 +0100, Benjamin Gaignard a écrit : >> Le 24/01/2024 à 13:52, Hans Verkuil a écrit : >>> On 19/01/2024 10:49, Benjamin Gaignard wrote: >>>> Allow to delete buffers on capture queue because it the one which >>>> own the decoded buffers. After a dynamic resolution change lot of >>>> them could remain allocated but won't be used anymore so deleting >>>> them save memory. >>>> Do not add this feature on output queue because the buffers are >>>> smaller, fewer and always recycled even after a dynamic resolution >>>> change. >>>> >>>> Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> >>>> --- >>>> drivers/media/platform/verisilicon/hantro_drv.c | 1 + >>>> drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + >>>> 2 files changed, 2 insertions(+) >>>> >>>> diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c >>>> index db3df6cc4513..f6b0a676a740 100644 >>>> --- a/drivers/media/platform/verisilicon/hantro_drv.c >>>> +++ b/drivers/media/platform/verisilicon/hantro_drv.c >>>> @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) >>>> dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; >>>> dst_vq->lock = &ctx->dev->vpu_mutex; >>>> dst_vq->dev = ctx->dev->v4l2_dev.dev; >>>> + src_vq->supports_delete_bufs = true; >>> As I mentioned, I remain unconvinced by this. It is just making the API inconsistent >>> since if you support delete_bufs, then why support it for one queue only and not both? >> >> Because the both queues don't handle the same type of data. >> For example for a stateless decoder, for me, it makes sense to allow delete decoded frames >> if they won't be used anymore but that won't makes sense for bitstream buffers. > > I personally think that for stateless and stateful decoder bitstream queue this > can be useful. We currently guess the size we need, and there is no way to > allocate bigger ones without the driver forgetting about reference frames. > > In stateful, some drivers allow to split the bitstream (I tested wave5 notably), > but I was told this is not always the case. A bit of a gray zone in that API, > with lack of capabilities to describe it. On stateless, the entire bitstream > slice must be in one buffer. > > Though, for the asymmetry, most stateful decoders won't allow delete bufs on > capture, because the buffers are registered in the firmware ahead of time. wave5 > can't even implement create_bufs on capture. We had an argument about having > create_bufs on only one queue for that specific driver, as we wanted something > upstream, we flex to removing create bufs completely. I think the all or nothing > rule on per queue create/delete_bufs is not aligned with the reality. I think the default should be that it supports DELETE_BUFS for both queues, but it should still be possible to only have it on one queue. When v18 is posted I want to play around with that, because I am not certain what the easiest way is to implement this. Another thing that needs to be added is a check that DELETE_BUFS is only enabled if CREATE_BUFS is also present, it makes no sense otherwise. Finally I want to take another look at the work required to make a CREATE_BUFS replacement since that ioctl is horrible. Whether that will become part of this series or done as a follow-up I am not sure about, but this series should definitely make it possible to cleanly integrate it. Regards, Hans > > Nicolas >> >>> >>>> >>>> return vb2_queue_init(dst_vq); >>>> } >>>> diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> index 941fa23c211a..34eab90e8a42 100644 >>>> --- a/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c >>>> @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { >>>> .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, >>>> .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, >>>> .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, >>>> + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, >>>> .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, >>>> >>>> .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, >>> In my view setting vidioc_delete_bufs should enable this feature, and if >>> for some strange reason only one queue support it, then make a wrapper >>> callback that returns an error when used with the wrong queue. >>> >>> Also note that patch 6/8 never checks for q->supports_delete_bufs in >>> vb2_core_delete_bufs(), which is wrong! >> >> I will fix that in next version. >> Regards, >> Benjamin >> >>> >>> Regards, >>> >>> Hans >>> >> >
diff --git a/drivers/media/platform/verisilicon/hantro_drv.c b/drivers/media/platform/verisilicon/hantro_drv.c index db3df6cc4513..f6b0a676a740 100644 --- a/drivers/media/platform/verisilicon/hantro_drv.c +++ b/drivers/media/platform/verisilicon/hantro_drv.c @@ -248,6 +248,7 @@ queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; dst_vq->lock = &ctx->dev->vpu_mutex; dst_vq->dev = ctx->dev->v4l2_dev.dev; + src_vq->supports_delete_bufs = true; return vb2_queue_init(dst_vq); } diff --git a/drivers/media/platform/verisilicon/hantro_v4l2.c b/drivers/media/platform/verisilicon/hantro_v4l2.c index 941fa23c211a..34eab90e8a42 100644 --- a/drivers/media/platform/verisilicon/hantro_v4l2.c +++ b/drivers/media/platform/verisilicon/hantro_v4l2.c @@ -756,6 +756,7 @@ const struct v4l2_ioctl_ops hantro_ioctl_ops = { .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_delete_bufs = v4l2_m2m_ioctl_delete_bufs, .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, .vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
Allow to delete buffers on capture queue because it the one which own the decoded buffers. After a dynamic resolution change lot of them could remain allocated but won't be used anymore so deleting them save memory. Do not add this feature on output queue because the buffers are smaller, fewer and always recycled even after a dynamic resolution change. Signed-off-by: Benjamin Gaignard <benjamin.gaignard@collabora.com> --- drivers/media/platform/verisilicon/hantro_drv.c | 1 + drivers/media/platform/verisilicon/hantro_v4l2.c | 1 + 2 files changed, 2 insertions(+)