| Message ID | 20211217111836.225013-1-tomi.valkeinen@ideasonboard.com |
|---|---|
| Headers | show |
| Series | v4l: subdev active state | expand |
Hi Tomi, On Fri, Dec 17, 2021 at 01:18:35PM +0200, Tomi Valkeinen wrote: > All suitable subdev ops are now passed either the TRY or the ACTIVE > state by the v4l2 core. However, other subdev drivers can still call the > ops passing NULL as the state, implying the active case. > > For all current upstream drivers this doesn't matter, as they do not > expect to get a valid state for ACTIVE case. But future drivers which > support multiplexed streaming and routing will depend on getting a state > for both active and try cases. > > For new drivers we can mandate that the pipelines where the drivers are > used need to pass the state properly, or preferably, not call such > subdev ops at all. > > However, if an existing subdev driver is changed to support multiplexed > streams, the driver has to consider cases where its ops will be called > with NULL state. The problem can easily be solved by using the > v4l2_subdev_lock_and_return_state() helper, introduced here. > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > include/media/v4l2-subdev.h | 35 +++++++++++++++++++++++++++++++++++ > 1 file changed, 35 insertions(+) > > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h > index 80cd0ffb60d1..4bc3cc04cc6e 100644 > --- a/include/media/v4l2-subdev.h > +++ b/include/media/v4l2-subdev.h > @@ -8,6 +8,7 @@ > #ifndef _V4L2_SUBDEV_H > #define _V4L2_SUBDEV_H > > +#include "linux/dev_printk.h" > #include <linux/types.h> > #include <linux/v4l2-subdev.h> > #include <media/media-entity.h> > @@ -1321,4 +1322,38 @@ void v4l2_subdev_lock_state(struct v4l2_subdev_state *state); > */ > void v4l2_subdev_unlock_state(struct v4l2_subdev_state *state); > > +/** > + * v4l2_subdev_lock_and_return_state() - Gets locked try or active subdev state > + * @sd: subdevice > + * @state: subdevice state as passed to the subdev op > + * > + * Due to legacy reasons, when subdev drivers call ops in other subdevs they use > + * NULL as the state parameter, as subdevs always used to have their active > + * state stored privately. > + * > + * However, newer state-aware subdev drivers, which store their active state in > + * a common place, subdev->active_state, expect to always get a proper state as > + * a parameter. > + * > + * These state-aware drivers can use v4l2_subdev_lock_and_return_state() instead > + * of v4l2_subdev_lock_state(). v4l2_subdev_lock_and_return_state() solves the > + * issue by using subdev->state in case the passed state is NULL. > + * > + * This is a temporary helper function, and should be removed when we can ensure > + * that all drivers pass proper state when calling other subdevs. > + */ > +static inline struct v4l2_subdev_state * > +v4l2_subdev_lock_and_return_state(struct v4l2_subdev *sd, > + struct v4l2_subdev_state *state) > +{ > + if (!state) > + dev_notice_once(sd->dev, "source subdev should be ported to new state management\n"); I agree the message is improvable. Maybe dev_notice_once(sd->dev, "The provided state is NULL. Adjusting to the subdev active state.\n"); dev_notice_once(sd->dev, "Please consid er porting your driver to the new state management API.\n); I don't mind multiple messages, other might. Anyway, as long as we warn, I'm fine with this helper! Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org> Thanks j > + > + state = state ? state : sd->active_state; > + > + v4l2_subdev_lock_state(state); > + > + return state; > +} > + > #endif > -- > 2.25.1 >
On 17/12/2021 12:18, Tomi Valkeinen wrote: > Add documentation about centrally managed subdev state. > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> > --- > .../driver-api/media/v4l2-subdev.rst | 58 +++++++++++++++++++ > 1 file changed, 58 insertions(+) > > diff --git a/Documentation/driver-api/media/v4l2-subdev.rst b/Documentation/driver-api/media/v4l2-subdev.rst > index 08ea2673b19e..f0ba04c80563 100644 > --- a/Documentation/driver-api/media/v4l2-subdev.rst > +++ b/Documentation/driver-api/media/v4l2-subdev.rst > @@ -518,6 +518,64 @@ The :c:func:`v4l2_i2c_new_subdev` function will call > :c:type:`i2c_board_info` structure using the ``client_type`` and the > ``addr`` to fill it. > > +Centrally managed subdev active state > +------------------------------------- > + > +Traditionally V4L2 subdev drivers maintained internal state for the active > +configuration for the subdev. This is often implemented e.g. as an array of > +struct v4l2_mbus_framefmt, one entry for each pad, and similarly for cropping > +and composition using struct v4l2_rect. > + > +In addition to the active configuration, each subdev filehandle has an array of > +struct v4l2_subdev_pad_config, managed by V4L2 core, which contains the try > +configuration. > + > +To simplify the subdev drivers the V4L2 subdev API now optionally supports a > +centrally managed active configuration represented by > +:c:type:`v4l2_subdev_state`. One instance of state, which contains the active > +device configuration, is associated with the sub-device itself as part of > +the :c:type:`v4l2_subdev` structure, while the core associates to each open > +file handle a try state, which contains the configuration valid in the > +file-handle context only. > + > +Sub-device drivers can opt-in and use state to manage their active configuration > +by initializing the subdevice state with a call to v4l2_subdev_init_finalize() > +before registering the sub-device. They must also call v4l2_subdev_cleanup() > +to release all the acquired resources before unregistering the sub-device. > +The core automatically initializes a state for each open file handle where to > +store the try configurations and releases them at file handle closing time. > + > +V4L2 sub-device operations that operate on both the :ref:`ACTIVE and TRY formats "operations that operate on": yuck. How about: "operations that use" > +<v4l2-subdev-format-whence>` receive the correct state to operate on an > +operation parameter. The sub-device driver can access and modify the "operate on an operation parameter"? I think you mean: "operate on as a 'state' parameter" > +configuration stored in the provided state after having locked it by calling > +:c:func:`v4l2_subdev_lock_state()`. The driver must then call > +:c:func:`v4l2_subdev_unlock_state()` to unlock the state when done. > + > +Operations that do not receive a state parameter implicitly operate on the > +subdevice active state, which drivers can exclusively access by > +calling :c:func:`v4l2_subdev_lock_active_state()`. The sub-device active state > +should equally be released by calling should -> must > +:c:func:`v4l2_subdev_unlock_state()`. > + > +In no occasions driver should try to manually access the state stored In no occasions driver should try" -> "Drivers must never try" > +in the :c:type:`v4l2_subdev` or in the file handle without going > +through the designated helpers. > + > +The V4L2 core will pass either the try- or active-state to various subdev ops. > +Unfortunately not all the callers comply with this yet, and may pass NULL as > +the active-state. This is only a problem for subdev drivers which use the > +centrally managed active-state and are used in media pipelines with older > +subdev drivers. In these cases the called subdev ops must also handle the NULL > +case. This can be easily managed by the use of > +v4l2_subdev_validate_and_lock_state() helper. > + > +v4l2_subdev_validate_and_lock_state() should only be used when porting an > +existing driver to the new state management when it cannot be guaranteed that > +the current callers will pass the state properly. The function prints a notice > +when the passed state is NULL to encourage the porting of the callers to the > +new state management. > + > V4L2 sub-device functions and data structures > --------------------------------------------- > > Regards, Hans
Hi Tomi, On Fri, Dec 17, 2021 at 01:18:32PM +0200, Tomi Valkeinen wrote: > Add a new 'active_state' field to struct v4l2_subdev to which we can > store the active state of a subdev. This will place the subdev > configuration into a known place, allowing us to use the state directly > from the v4l2 framework, thus simplifying the drivers. > > Also add functions v4l2_subdev_init_finalize() and > v4l2_subdev_cleanup(), which will allocate and free the active state. > The functions are named in a generic way so that they can be also used > for other subdev initialization work. > > Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> > Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl> > --- > drivers/media/v4l2-core/v4l2-subdev.c | 21 ++++++++++ > include/media/v4l2-subdev.h | 58 +++++++++++++++++++++++++++ > 2 files changed, 79 insertions(+) > > diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c > index fe49c86a9b02..de160140d63b 100644 > --- a/drivers/media/v4l2-core/v4l2-subdev.c > +++ b/drivers/media/v4l2-core/v4l2-subdev.c > @@ -943,3 +943,24 @@ void v4l2_subdev_notify_event(struct v4l2_subdev *sd, > v4l2_subdev_notify(sd, V4L2_DEVICE_NOTIFY_EVENT, (void *)ev); > } > EXPORT_SYMBOL_GPL(v4l2_subdev_notify_event); > + > +int v4l2_subdev_init_finalize(struct v4l2_subdev *sd) > +{ > + struct v4l2_subdev_state *state; > + > + state = __v4l2_subdev_state_alloc(sd); > + if (IS_ERR(state)) > + return PTR_ERR(state); > + > + sd->active_state = state; > + > + return 0; > +} > +EXPORT_SYMBOL_GPL(v4l2_subdev_init_finalize); > + > +void v4l2_subdev_cleanup(struct v4l2_subdev *sd) > +{ > + __v4l2_subdev_state_free(sd->active_state); > + sd->active_state = NULL; > +} > +EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup); > diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h > index e52bf508c75b..eddf72768e10 100644 > --- a/include/media/v4l2-subdev.h > +++ b/include/media/v4l2-subdev.h > @@ -645,6 +645,9 @@ struct v4l2_subdev_ir_ops { > * This structure only needs to be passed to the pad op if the 'which' field > * of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For > * %V4L2_SUBDEV_FORMAT_ACTIVE it is safe to pass %NULL. > + * > + * Note: This struct is also used in active state, and the try_ prefix is > + * historical and to be removed. > */ > struct v4l2_subdev_pad_config { > struct v4l2_mbus_framefmt try_fmt; > @@ -898,6 +901,9 @@ struct v4l2_subdev_platform_data { > * @subdev_notifier: A sub-device notifier implicitly registered for the sub- > * device using v4l2_async_register_subdev_sensor(). > * @pdata: common part of subdevice platform data > + * @active_state: Active state for the subdev (NULL for subdevs tracking the > + * state internally). Initialized by calling > + * v4l2_subdev_init_finalize(). Is alignement a little off or is it my mail client ? Nit apart: Reviewed-by: Jacopo Mondi <jacopo+renesas@jmondi.org> Thanks j > * > * Each instance of a subdev driver should create this struct, either > * stand-alone or embedded in a larger struct. > @@ -929,6 +935,19 @@ struct v4l2_subdev { > struct v4l2_async_notifier *notifier; > struct v4l2_async_notifier *subdev_notifier; > struct v4l2_subdev_platform_data *pdata; > + > + /* > + * The fields below are private, and should only be accessed via > + * appropriate functions. > + */ > + > + /* > + * TODO: active_state should most likely be changed from a pointer to an > + * embedded field. For the time being it's kept as a pointer to more > + * easily catch uses of active_state in the cases where the driver > + * doesn't support it. > + */ > + struct v4l2_subdev_state *active_state; > }; > > > @@ -1217,4 +1236,43 @@ extern const struct v4l2_subdev_ops v4l2_subdev_call_wrappers; > void v4l2_subdev_notify_event(struct v4l2_subdev *sd, > const struct v4l2_event *ev); > > +/** > + * v4l2_subdev_init_finalize() - Finalizes the initialization of the subdevice > + * @sd: The subdev > + * > + * This function finalizes the initialization of the subdev, including > + * allocation of the active state for the subdev. > + * > + * This function must be called by the subdev drivers that use the centralized > + * active state, after the subdev struct has been initialized and > + * media_entity_pads_init() has been called, but before registering the > + * subdev. > + * > + * The user must call v4l2_subdev_cleanup() when the subdev is being removed. > + */ > +int v4l2_subdev_init_finalize(struct v4l2_subdev *sd); > + > +/** > + * v4l2_subdev_cleanup() - Releases the resources needed by the subdevice > + * @sd: The subdevice > + * > + * This function will release the resources allocated in > + * v4l2_subdev_init_finalize. > + */ > +void v4l2_subdev_cleanup(struct v4l2_subdev *sd); > + > +/** > + * v4l2_subdev_get_active_state() - Returns the active subdev state for > + * subdevice > + * @sd: The subdevice > + * > + * Returns the active state for the subdevice, or NULL if the subdev does not > + * support active state. > + */ > +static inline struct v4l2_subdev_state * > +v4l2_subdev_get_active_state(struct v4l2_subdev *sd) > +{ > + return sd->active_state; > +} > + > #endif > -- > 2.25.1 >
On 17/12/2021 14:07, Jacopo Mondi wrote: > Hi Tomi, > > On Fri, Dec 17, 2021 at 01:18:32PM +0200, Tomi Valkeinen wrote: >> Add a new 'active_state' field to struct v4l2_subdev to which we can >> store the active state of a subdev. This will place the subdev >> configuration into a known place, allowing us to use the state directly >> from the v4l2 framework, thus simplifying the drivers. >> >> Also add functions v4l2_subdev_init_finalize() and >> v4l2_subdev_cleanup(), which will allocate and free the active state. >> The functions are named in a generic way so that they can be also used >> for other subdev initialization work. >> >> Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com> >> Reviewed-by: Hans Verkuil <hverkuil-cisco@xs4all.nl> >> --- >> drivers/media/v4l2-core/v4l2-subdev.c | 21 ++++++++++ >> include/media/v4l2-subdev.h | 58 +++++++++++++++++++++++++++ >> 2 files changed, 79 insertions(+) >> >> diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c >> index fe49c86a9b02..de160140d63b 100644 >> --- a/drivers/media/v4l2-core/v4l2-subdev.c >> +++ b/drivers/media/v4l2-core/v4l2-subdev.c >> @@ -943,3 +943,24 @@ void v4l2_subdev_notify_event(struct v4l2_subdev *sd, >> v4l2_subdev_notify(sd, V4L2_DEVICE_NOTIFY_EVENT, (void *)ev); >> } >> EXPORT_SYMBOL_GPL(v4l2_subdev_notify_event); >> + >> +int v4l2_subdev_init_finalize(struct v4l2_subdev *sd) >> +{ >> + struct v4l2_subdev_state *state; >> + >> + state = __v4l2_subdev_state_alloc(sd); >> + if (IS_ERR(state)) >> + return PTR_ERR(state); >> + >> + sd->active_state = state; >> + >> + return 0; >> +} >> +EXPORT_SYMBOL_GPL(v4l2_subdev_init_finalize); >> + >> +void v4l2_subdev_cleanup(struct v4l2_subdev *sd) >> +{ >> + __v4l2_subdev_state_free(sd->active_state); >> + sd->active_state = NULL; >> +} >> +EXPORT_SYMBOL_GPL(v4l2_subdev_cleanup); >> diff --git a/include/media/v4l2-subdev.h b/include/media/v4l2-subdev.h >> index e52bf508c75b..eddf72768e10 100644 >> --- a/include/media/v4l2-subdev.h >> +++ b/include/media/v4l2-subdev.h >> @@ -645,6 +645,9 @@ struct v4l2_subdev_ir_ops { >> * This structure only needs to be passed to the pad op if the 'which' field >> * of the main argument is set to %V4L2_SUBDEV_FORMAT_TRY. For >> * %V4L2_SUBDEV_FORMAT_ACTIVE it is safe to pass %NULL. >> + * >> + * Note: This struct is also used in active state, and the try_ prefix is >> + * historical and to be removed. >> */ >> struct v4l2_subdev_pad_config { >> struct v4l2_mbus_framefmt try_fmt; >> @@ -898,6 +901,9 @@ struct v4l2_subdev_platform_data { >> * @subdev_notifier: A sub-device notifier implicitly registered for the sub- >> * device using v4l2_async_register_subdev_sensor(). >> * @pdata: common part of subdevice platform data >> + * @active_state: Active state for the subdev (NULL for subdevs tracking the >> + * state internally). Initialized by calling >> + * v4l2_subdev_init_finalize(). > > Is alignement a little off or is it my mail client ? Looks correct to me. It's using tabs there. I don't like, I think it should be spaces, but I followed the convention used elsewhere. Tomi