| Message ID | 20231118062754.2453-1-quic_luoj@quicinc.com |
|---|---|
| Headers | show |
| Series | add qca8084 ethernet phy driver | expand |
On Sat, Nov 18, 2023 at 02:27:51PM +0800, Luo Jie wrote: > Add qca8084 PHY support, which is four-port PHY with maximum > link capability 2.5G, the features of each port is almost same > as QCA8081 and slave seed config is not needed. > > Three kind of interface modes supported by qca8084. > PHY_INTERFACE_MODE_10G_QXGMII, PHY_INTERFACE_MODE_2500BASEX and > PHY_INTERFACE_MODE_SGMII. Sorry for joining the conversation late. I'm trying to get my head around QXGMII. Let me describe what i think is happening, and then you can correct me.... You have 4 MACs, probably in a switch. The MII interfaces from these MACs go into a multiplexer, and out comes QXGMII? You then have a SERDES interface out of the switch and into the PHY package. Inside the PHY package there is a demultiplexor, giving you four MII interfaces, one to each PHY in the package. If you have the PHY SERDES running in 2500BaseX, you have a single MAC, no mux/demux, and only one PHY is used? The other three are idle. Same from SGMII? So the interface mode QXGMII is a property of the package. It is not really a property of one PHY. Having one PHY using QXGMII and another SGMII does not work? Andrew
On Sat, Nov 18, 2023 at 04:51:42PM +0100, Andrew Lunn wrote: > On Sat, Nov 18, 2023 at 02:27:51PM +0800, Luo Jie wrote: > > Add qca8084 PHY support, which is four-port PHY with maximum > > link capability 2.5G, the features of each port is almost same > > as QCA8081 and slave seed config is not needed. > > > > Three kind of interface modes supported by qca8084. > > PHY_INTERFACE_MODE_10G_QXGMII, PHY_INTERFACE_MODE_2500BASEX and > > PHY_INTERFACE_MODE_SGMII. > > Sorry for joining the conversation late. > > I'm trying to get my head around QXGMII. Let me describe what i think > is happening, and then you can correct me.... > > You have 4 MACs, probably in a switch. The MII interfaces from these > MACs go into a multiplexer, and out comes QXGMII? You then have a > SERDES interface out of the switch and into the PHY package. Inside > the PHY package there is a demultiplexor, giving you four MII > interfaces, one to each PHY in the package. > > If you have the PHY SERDES running in 2500BaseX, you have a single > MAC, no mux/demux, and only one PHY is used? The other three are idle. > Same from SGMII? > > So the interface mode QXGMII is a property of the package. It is not > really a property of one PHY. Having one PHY using QXGMII and another > SGMII does not work? 10G_QXGMII is defined in the Cisco USXGMII multi-port document as one of several possibilities for a USXGMII-M link. The Cisco document can be a little confusing beause it states that 10G_QXGMII supports 10M, 100M, 1G and 2.5G, and then only talks about a 10G and 100M/1G MAC. For 10G_QXGMII, there are 4 MAC interfaces. These are connected to a rate "adaption" through symbol replication block, and then on to a clause 49 PCS block. There is then a port MUX and framing block, followed by the PMA serdes which communicates with the remote end over a single pair of transmit/receive serdes lines. Each interface also has its own clause 37 autoneg block. So, for an interface to operate in SGMII mode, it would have to be muxed to a different path before being presented to the USXGMII-M block since each interface does not have its own external data lane - thus that's out of scope of USXGMII-M as documented by Cisco. Hope this helps.
> 10G_QXGMII is defined in the Cisco USXGMII multi-port document as one > of several possibilities for a USXGMII-M link. The Cisco document can > be a little confusing beause it states that 10G_QXGMII supports 10M, > 100M, 1G and 2.5G, and then only talks about a 10G and 100M/1G MAC. > > For 10G_QXGMII, there are 4 MAC interfaces. These are connected to a > rate "adaption" through symbol replication block, and then on to a > clause 49 PCS block. > > There is then a port MUX and framing block, followed by the PMA > serdes which communicates with the remote end over a single pair of > transmit/receive serdes lines. > > Each interface also has its own clause 37 autoneg block. > > So, for an interface to operate in SGMII mode, it would have to be > muxed to a different path before being presented to the USXGMII-M > block since each interface does not have its own external data lane > - thus that's out of scope of USXGMII-M as documented by Cisco. Hi Russell I think it helps. Where i'm having trouble is deciding if this is actually an interface mode. Interface mode is a per PHY property. Where as it seems 10G_QXGMII is a property of the USXGMII-M link? Should we be representing the package with 4 PHYs in it, and specify the package has a PMA which is using 10G_QXGMII over USXGMII-M? The PHY interface mode is then internal? Its just the link between the PHY and the MUX? By saying the interface mode is 10G_QXGMII and not describing the PMA mode, are we setting ourselves up for problems in the future? Could there be a PMA interface which could carry different PHY interface modes? If we decide we do want to use 10G_QXGMII as an interface made, i think the driver should be doing some validation. If asked to do anything else, it should return -EINVAL. And i don't yet understand how it can also do 1000BaseX and 2500BaseX and SGMII? Andrew
On 11/21/2023 12:18 AM, Russell King (Oracle) wrote: > On Mon, Nov 20, 2023 at 04:34:55PM +0100, Andrew Lunn wrote: >> Are you saying there is a USXGMII-M level link change status? The link >> between the SoC and the PHY package is up/down? If it is down, all >> four MAC-PHY links are down. If it is up, it is possible to carry >> frames between the SoC and the PHY package, but maybe the PHYs >> themselves are down? > > It shouldn't do. Each "channel" in the USXGMII-M link has its own > autoneg block at both ends, each conveys link status independently. > > The MAC side structure is: > > > +----------+ +-----+ > .-XGMII-> Rate | PCS | | > MAC1 <-MDI-> PHY <-+ | Adaption <--> Clause 49 <-> | > `-GMII--> | | | > +-----^----+ | | > | | | > +-----v---- + | | > | Autoneg | | | > | Clause 37 | | | > +-----------+ | | > | Mux <--> PMA <--> > | | > ....... USXGMII-M > > <------------------------------------------------------> > These blocks are repeated for each channel > > The spec goes on to state that there must be a USXGMII enable bit that > defaults to disabled and the PHY should assume normal XGMII/XFI > operation. When enabled, autoneg follows a slight modification of > clause 37-6. > > As far as the USXGMII-M link, I believe 2.7.8 in the USXGMII-M > documentation covers this, which is "hardware autoneg programming > sequence". It states that "if 10G link is lost or regained, the > software is expected to disable autoneg and re-enable autoneg". I > think "10G link" refers to the USXGMII-M connection, which means > the loss of that link shold cause software to intervene in each > of the PCS autoneg blocks. It is, however, rather unclear. > The link status of PHY is updated, software should do the corresponding QXGMII mode configuration per channel for this PHY. The PCS QXGMII configuration reflects the current link status of the connected PHY.
On Tue, Nov 21, 2023 at 07:10:08PM +0800, Jie Luo wrote: > when pcs is configured to SGMII mode, the fourth PHY can reach to > maximum speed 2.5G(2500BaseT) that is reached by increasing the clock > rate to 312.5MHZ from 125MHZ of 1G speed, but there is no corresponding > interface mode can be used to reflect this 2.5G speed mode(sgmii+) So this comes up again. 2.5G SGMII? What is that? Let's start off with the basics. SGMII is Cisco's modification of 1000base-X. The two are broadly compatible in that they can communicate with each other provided that the inband control word is disregarded. 2500base-X is generally implemented as 1000base-X over-clocked by 2.5x. Some manufacturers state that the inband control word is not supported. Others say it can be used. This disparity comes from the lack of early IEEE standardisation of this protocol. Cisco SGMII as defined is a 10M/100M/1G protocol operating at 125MHz with a fixed underlying baud rate of 1250Mbaud. Slower speeds are achieved via symbol replication by 10x or 100x. The inband control word is modified in order to convey this speed information, as well as duplex and sometimes also other vendor extensions. Switching SGMII to be clocked 2.5x faster means that a partner that expects SGMII at normal speed sees garbage - it can't recognise the waveform. Therefore, it is not possible for inband to convey any information. Many vendors explicitly state that symbol replication is not supported when "SGMII" is clocked at 2.5x. All variants of whatever the vendor calls the 2.5G mode tend to use the SGMII term because... it's Serial Gigabit... and SGMII even gets used by vendors to describe the interface used for 1000base-X. Vendors use terms like "HS-SGMII" and other stuff to describe their 2.5x mode. Some use "2500base-X". Yours seems to use "SGMII+". SGMII without inband signalling is basically the same as 1000base-X. Therefore, SGMII clocked at 2.5x the speed is basically the same as 2500base-X without inband signalling. So, the whole area is totally confused, and one should not get too hung up on the terminology that vendors are using, but go back to precisely what's going on at the hardware level. We have raised this point almost every time someone talks about an up-clocked "SGMII". > Actually we should add a new interface mode such as sgmii+ > to reflect this 2.5G speed of sgmii Only if there really is something different about it. For example, if it were Cisco SGMII modified to operate always at 312.5MHz with inband signalling updated to signal the four speeds. That would definitely be a different protocol. However, it's not that. What it actually is is Cisco SGMII when operating at 10M/100M/1G speeds, and 2500base-X without inband signalling when operating at 2.5G speed. We have PHYs that support this (and more) which we support. PHYs that switch between 10GBASE-R, 5GBASE-R, 2500BASE-X and Cisco SGMII depending on the speed that was negotiated on the media. There is no definition of a single interface mode that covers all those, because it isn't a single interface mode. It's four separate modes that the PHY switches between - and this is no different from what is happening with your PHY. Ultimately, you will need a way to use inband signalling with Cisco SGMII for 10M/100M/1G speeds, and then switch to 2500base-X when operating at 2.5G speeds, and that is done via the PHY driver updating phydev->interface. What we do need is some way for the PHY to also tell the PCS/MAC whether inband should be used. This is something I keep bringing up and now that we have PCS drivers revised to use the value from phylink_pcs_neg_mode() _and_ a consistent implementation amongst them we can now think about signalling to PCS drivers whether inband mode needs to be turned off when switching between modes. There have been patches in the past that allow inband mode to be queried from phylib, and this is another important component in properly dealing with PHYs that need to use inband signalling with Cisco SGMII, but do not support inband signalling when operating at 2.5G speeds. The problem when operating at 2.5G speed is that the base-X protocols are normally for use over fibre, which is the media, and therefore the ethtool Autoneg bit should define whether inband gets used or not. However, in the case of a PHY using 2500base-X, the Autoneg bit continues to define whether autonegotiation should be used on the media, and in this case it's the media side of the PHY rather than the 2500base-X link. So, when using a 2500base-X link to a PHY, we need to disregard the Autoneg bit, but that then raises the question about how we should configure it - and one solution to that would be to entire of phylib what the PHY wants to do. Another is to somehow ask the PCS driver whether it supports inband signalling at 2500base-X, and resolve those capabilities. That is my view where we need to get to in order to properly resolve the ongoing issues about 2500base-X and PHYs that make use of that.
On Thu, Nov 23, 2023 at 06:57:59PM +0800, Jie Luo wrote: > On 11/21/2023 7:52 PM, Russell King (Oracle) wrote: > > Ultimately, you will need a way to use inband signalling with Cisco > > SGMII for 10M/100M/1G speeds, and then switch to 2500base-X when > > operating at 2.5G speeds, and that is done via the PHY driver > > updating phydev->interface. > > > > What we do need is some way for the PHY to also tell the PCS/MAC > > whether inband should be used. This is something I keep bringing up > > and now that we have PCS drivers revised to use the value from > > phylink_pcs_neg_mode() _and_ a consistent implementation amongst them > > we can now think about signalling to PCS drivers whether inband mode > > needs to be turned off when switching between modes. > > Yes, we can switch the interface mode according to the current link > speed in the pcs driver. > but the issue is that the phy-mode i specified for the PHYLINK, > if phy-mode is sgmii, the support capability is limited to maximum > capability 1G during the PHYLINK setup and i can't configure it to 2.5G > dynamically, if the phy-mode is 2500base-x, then PHY capability will > be modified to only support 2.5G, other speeds can't be linked up. So you need my patches that add "possible_interfaces" to phylib so you can tell phylink that you will be switching between SGMII and 2500base-X. Please see the RFC posting of those patches I sent yesterday and try them out - you will need to modify your phylib driver to fill in phydev->possible_interfaces. > > There have been patches in the past that allow inband mode to be > > queried from phylib, and this is another important component in > > properly dealing with PHYs that need to use inband signalling with > > Cisco SGMII, but do not support inband signalling when operating at > > 2.5G speeds. The problem when operating at 2.5G speed is that the > > base-X protocols are normally for use over fibre, which is the media, > > and therefore the ethtool Autoneg bit should define whether inband > > gets used or not. However, in the case of a PHY using 2500base-X, > > the Autoneg bit continues to define whether autonegotiation should > > be used on the media, and in this case it's the media side of the > > PHY rather than the 2500base-X link. > > > > So, when using a 2500base-X link to a PHY, we need to disregard the > > Autoneg bit, but that then raises the question about how we should > > configure it - and one solution to that would be to entire of phylib > > what the PHY wants to do. Another is to somehow ask the PCS driver > > whether it supports inband signalling at 2500base-X, and resolve > > those capabilities. > > For the qca808x PHY, when it is linked in 2.5G, the autoneg is also > disabled in PCS hardware, so the sgmii+ of qca808x PHY is almost > same as 2500base-X. Not "almost". It _is_ the same. This is the point I've been trying to get across to you. Without inband signalling, 1000base-X and SGMII (when operating at 1G) are _identical_ and entirely compatible. You've said that your 2.5G "SGMII" mode has inband signalling disabled, and thus it without inband signalling, 2500base-X and this 2.5G mode are again identical and entirely compatible. There's no "almost" about it.
On 11/23/2023 8:01 PM, Russell King (Oracle) wrote: > On Thu, Nov 23, 2023 at 06:57:59PM +0800, Jie Luo wrote: >> On 11/21/2023 7:52 PM, Russell King (Oracle) wrote: >>> Ultimately, you will need a way to use inband signalling with Cisco >>> SGMII for 10M/100M/1G speeds, and then switch to 2500base-X when >>> operating at 2.5G speeds, and that is done via the PHY driver >>> updating phydev->interface. >>> >>> What we do need is some way for the PHY to also tell the PCS/MAC >>> whether inband should be used. This is something I keep bringing up >>> and now that we have PCS drivers revised to use the value from >>> phylink_pcs_neg_mode() _and_ a consistent implementation amongst them >>> we can now think about signalling to PCS drivers whether inband mode >>> needs to be turned off when switching between modes. >> >> Yes, we can switch the interface mode according to the current link >> speed in the pcs driver. >> but the issue is that the phy-mode i specified for the PHYLINK, >> if phy-mode is sgmii, the support capability is limited to maximum >> capability 1G during the PHYLINK setup and i can't configure it to 2.5G >> dynamically, if the phy-mode is 2500base-x, then PHY capability will >> be modified to only support 2.5G, other speeds can't be linked up. > > So you need my patches that add "possible_interfaces" to phylib so you > can tell phylink that you will be switching between SGMII and > 2500base-X. Please see the RFC posting of those patches I sent > yesterday and try them out - you will need to modify your phylib > driver to fill in phydev->possible_interfaces. Your patches work on my board, thanks Russell. > >>> There have been patches in the past that allow inband mode to be >>> queried from phylib, and this is another important component in >>> properly dealing with PHYs that need to use inband signalling with >>> Cisco SGMII, but do not support inband signalling when operating at >>> 2.5G speeds. The problem when operating at 2.5G speed is that the >>> base-X protocols are normally for use over fibre, which is the media, >>> and therefore the ethtool Autoneg bit should define whether inband >>> gets used or not. However, in the case of a PHY using 2500base-X, >>> the Autoneg bit continues to define whether autonegotiation should >>> be used on the media, and in this case it's the media side of the >>> PHY rather than the 2500base-X link. >>> >>> So, when using a 2500base-X link to a PHY, we need to disregard the >>> Autoneg bit, but that then raises the question about how we should >>> configure it - and one solution to that would be to entire of phylib >>> what the PHY wants to do. Another is to somehow ask the PCS driver >>> whether it supports inband signalling at 2500base-X, and resolve >>> those capabilities. >> >> For the qca808x PHY, when it is linked in 2.5G, the autoneg is also >> disabled in PCS hardware, so the sgmii+ of qca808x PHY is almost >> same as 2500base-X. > > Not "almost". It _is_ the same. This is the point I've been trying > to get across to you. Without inband signalling, 1000base-X and SGMII > (when operating at 1G) are _identical_ and entirely compatible. > > You've said that your 2.5G "SGMII" mode has inband signalling disabled, > and thus it without inband signalling, 2500base-X and this 2.5G mode > are again identical and entirely compatible. There's no "almost" about > it. > > Yes, confirmed with HW guy, they work on the same way.
On 11/24/2023 5:53 PM, Russell King (Oracle) wrote: > On Fri, Nov 24, 2023 at 05:47:04PM +0800, Jie Luo wrote: >> >> >> On 11/23/2023 8:01 PM, Russell King (Oracle) wrote: >>> On Thu, Nov 23, 2023 at 06:57:59PM +0800, Jie Luo wrote: >>>> On 11/21/2023 7:52 PM, Russell King (Oracle) wrote: >>>>> Ultimately, you will need a way to use inband signalling with Cisco >>>>> SGMII for 10M/100M/1G speeds, and then switch to 2500base-X when >>>>> operating at 2.5G speeds, and that is done via the PHY driver >>>>> updating phydev->interface. >>>>> >>>>> What we do need is some way for the PHY to also tell the PCS/MAC >>>>> whether inband should be used. This is something I keep bringing up >>>>> and now that we have PCS drivers revised to use the value from >>>>> phylink_pcs_neg_mode() _and_ a consistent implementation amongst them >>>>> we can now think about signalling to PCS drivers whether inband mode >>>>> needs to be turned off when switching between modes. >>>> >>>> Yes, we can switch the interface mode according to the current link >>>> speed in the pcs driver. >>>> but the issue is that the phy-mode i specified for the PHYLINK, >>>> if phy-mode is sgmii, the support capability is limited to maximum >>>> capability 1G during the PHYLINK setup and i can't configure it to 2.5G >>>> dynamically, if the phy-mode is 2500base-x, then PHY capability will >>>> be modified to only support 2.5G, other speeds can't be linked up. >>> >>> So you need my patches that add "possible_interfaces" to phylib so you >>> can tell phylink that you will be switching between SGMII and >>> 2500base-X. Please see the RFC posting of those patches I sent >>> yesterday and try them out - you will need to modify your phylib >>> driver to fill in phydev->possible_interfaces. >> >> Your patches work on my board, thanks Russell. > > Please can you reply to the covering email for that series giving your > tested-by? Thanks. > Ok.