Quality of Service (QoS) is defined as the minimal end-to-end performance that is guaranteed in advance by a service level agreement (SLA) to a workload. A workload may be a single application, a group of applications, a virtual machine, a group of virtual machines, or a combination of those.
The RISC-V Capacity and Bandwidth QoS Register Interface (CBQRI) cite:[CBQRI] specification specifies mechanisms to a) associate QoS identifiers with requests made by workloads to resources controlled by capacity and bandwidth controllers, b) memory-mapped register interface for capacity allocation and capacity usage monitoring in capacity controllers in shared resources such as shared caches, c) memory-mapped register interface for bandwidth allocation and bandwidth usage monitoring in bandwidth controllers in shared resources such as memory controllers.
The Ssqosid extension cite:[SSQOSID] provides a read/write S/HS-mode register
(srmcfg) to configure QoS Identifiers to be used with requests made by the
hart to shared resources. The CBQRI specification specifies the IOMMU extension
to associate QoS Identifiers with requests made by devices to shared resources.
With supervisor domains, the resource accesses from an SD or the RDSM must not be observable by entities that are not within their TCB using the resource usage monitors. Similarly, the resource allocations for an SD or the RDSM must not be influenced by entities outside their TCB.
To support this security objective, the following capabilities are required:
-
Multiple QoS register interfaces (QRIs) in the capacity and bandwidth controllers within the SoC.
-
The RDSM may allocate a QRI exclusively to an SD or may mediate access to a QRI among multiple SDs. A non-ISA supervisor domain QRI extension is specified in Supervisor Domain QoS Register interfaces (QRI) to provide this capability.
-
-
Associating a QRI ID (QRID) with requests originating from SDs and RDSM.
-
The QRID, along with the
RCIDandAT, is used to identify the resource allocation configurations in a capacity or bandwidth controller. The QRID, along with theMCID, is used to identify the ID of the counter used to monitor resource usage in a capacity or bandwidth controller. An ISA extension named Smqosid is specified in Smqosid - Supervisor Domain Quality-of-Service Identifiers to provide this capability.
-
The Smqosid introduces a read/write M-mode register (mrmcfg) to configure
QoS identifiers for requests made by a hart with effective privilege mode equal
to M.
The mrmcfg register is an MXLEN-bit read/write register used to configure a
Resource Control ID (RCID), Monitoring Counter ID (MCID), and a Machine QRID
(MQRID). The MQRID, RCID, and MCID are WARL fields. The register is
formatted as shown in Machine Resource Management Configuration (mrmcfg) register for MXLEN=64 when MXLEN=64 and Machine Resource Management Configuration (mrmcfg) register for MXLEN=32 when MXLEN=32.
The CSR number is TBA.
The MQRID, RCID, and MCID values accompany each request made with effective
privilege mode equal to M by the hart to shared resource controllers. The MQRID
and RCID values are used to determine the resource allocations (e.g., cache
occupancy limits, memory bandwidth limits, etc.) to enforce. The MQRID and
MCID values are used to identify a counter to monitor resource usage.
|
Note
|
A suggested reset value of 0 for the The |
When Smqosid is implemented, the configurations in the srmcfg register
introduced by Ssqosid extension only apply to requests made with effective
privilege modes less than M.
mrmcfg) register for MXLEN=64{reg: [
{bits: 12, name: 'RCID'},
{bits: 4, name: 'WPRI'},
{bits: 12, name: 'MCID'},
{bits: 4, name: 'MQRID'},
{bits: 32, name: 'WPRI'},
], config:{lanes: 2, hspace:1024}}
mrmcfg) register for MXLEN=32{reg: [
{bits: 12, name: 'RCID'},
{bits: 4, name: 'WPRI'},
{bits: 12, name: 'MCID'},
{bits: 4, name: 'MQRID'},
], config:{lanes: 1, hspace:1024}}
The Smqosid extension enables the scenario where two or more security domains
share a common set of capacity and bandwidth controllers. In this setup, access
to the QRI of these controllers is restricted to RDSM, which then provides
mediated access for these security domains to configure capacity or bandwidth
allocation and read the corresponding monitoring counters. A common QRID is
associated with requests from these domains, and the RDSM allocates sets of
RCID and MCID for each domain’s use. Supervisor domains are then restricted
to selecting a RCID and MCID in srmcfg from the allocated range. Although
the RDSM could limit access to srmcfg to prevent direct domain configuration,
enabling domains to program srmcfg directly is preferred for performance
reasons.
To facilitate delegation of srmcfg to the SD, the Smqosid extension introduces
SRL (supervisor-RCID-length) and SML (supervisor-MCID-length) fields in the
msdcfg register to configure the usable range of RCID and MCID. These
fields are WARL, with legal values from 0 to 12. Furthermore, a
supervisor-srmcfg-mode (SSM) field is introduced, with legal values of 0 and
1. The SSM, along with SRL and SML settings, determine the range of RCID
and MCID values an SD can program into the srmcfg register. The RDSM is
expected to configure these fields such that each SD can select from a disjoint
range of values for RCID and MCID.
msdcfg register{reg: [
{bits: 6, name: 'SDICN'},
{bits: 1, name: 'SSM'},
{bits: 1, name: 'SDEDBGALW'},
{bits: 1, name: 'SDETRCALW'},
{bits: 11, name: 'WPRI'},
{bits: 4, name: 'SRL'},
{bits: 4, name: 'SML'},
{bits: 4, name: 'SQRID'},
], config:{lanes: 4, hspace:1024}}
The srmcfg CSR is provided with a privilege mode dependent read and write
logic which enables the RDSM to configure supervisor domain specific prefixes for
RCID and MCID fields. These prefixes remain invisible to supervisor domains
which can only configure RCID and MCID suffixes within the boundaries set by
RDSM in msdcfg.
When SSM is 0, the SRL configuration enables the RDSM to configure a RCID
prefix value in the srmcfg.RCID field while allowing the supervisor domain to
program the low order SRL bits of the field. When SSM is 1, the supervisor
domain may only program a RCID value in the range 0 through
2RCIDLEN - 2SRL - 1 where RCIDLEN is the number of implemented bits of
the RCID field.
When SSM is 0, the SML configuration enables the RDSM to configure a MCID
prefix value in the srmcfg.MCID field while allowing the supervisor domain to
program the low order SML bits of the field. When SSM is 1, the supervisor
domain may only program a MCID value in the range 0 through
2MCIDLEN - 2SML - 1 where MCIDLEN is the number of implemented bits of
the MCID field.
When the srmcfg CSR is read at privilege modes less than M, the value returned
for the RCID and MCID fields of the register is computed as follows:
RCID and MCID read value computationSRL_MASK = (1 << SRL) - 1
if SSM == 0
RCID-value = srmcfg.RCID & SRL_MASK
else
RCID-value = srmcfg.RCID
endif
SML_MASK = (1 << SML) - 1
if SSM == 0
MCID-value = srmcfg.MCID & SML_MASK
else
MCID-value = srmcfg.MCID
endif
On a write to the srmcfg CSR at privilege modes less than M, the value
stored in the RCID and MCID fields of the register are computed as
follows:
RCID and MCID write value processingSRL_MASK = (1 << SRL) - 1
if SSM == 0
srmcfg.RCID = (srmcfg.RCID & ~SRL_MASK) | (RCID-value & SRL_MASK)
else
if ((RCID-value & ~SRL_MASK) | SRL_MASK) != ((1 << RCIDLEN) - 1)
srmcfg.RCID = RCID-value
else
srmcfg.RCID = RCID-value & ((1 << RCIDLEN) - (1 << SRL) - 1)
endif
endif
SML_MASK = (1 << SML) - 1
if SSM == 0
srmcfg.MCID = (srmcfg.MCID & ~SML_MASK) | (MCID-value & SML_MASK)
else
if ((MCID-value & ~SML_MASK) | SML_MASK) != ((1 << MCIDLEN) - 1)
srmcfg.MCID = MCID-value
else
srmcfg.MCID = MCID-value & ((1 << MCIDLEN) - (1 << SML) - 1)
endif
endif
|
Note
|
Consider a QRI that supports 32 RCIDs and is mediated by RDSM between two SDs.
The RDSM may allocate 24 RCIDs to the first SD and 8 RCIDs to the second SD. The
|
The RCID and MCID values that accompany requests, when made with an effective
privilege mode equal to M are stored in the mrmcfg register. Conversely, for
requests made with effective privilege modes less than M, these values are
stored in the srmcfg register.
The Smqosid extension introduces a SQRID field in the msdcfg register to
hold the QRID for requests made by the hart with effective privilege mode less
than M respectively. The SQRID value along with the RCID and MCID values
accompanies each such request made by the hart to the shared resource
controllers. The SQRID and RCID values are used to determine the resource
allocations (e.g., cache occupancy limits, memory bandwidth limits, etc.) to
enforce. The SQRID and MCID values are used to identify a counter to monitor
resource usage.
|
Note
|
The |
Capacity and bandwidth controllers that support supervisor domains provide one or more memory-mapped QoS register interfaces (QRI). A QRI may be made exclusively accessible to a supervisor domain or the RDSM may mediate access to the QRI using an SBI. The RDSM can control access to the QRI from supervisor domains using MPT and/or PMP. The RDSM controls access to the QRI from devices using IOMPT and/or IOPMP.
The number of RCID and MCID supported by the controllers for each QRI need
not be identical. For maximal flexibility in allocation of RCID and MCID
values, it is recommended that the number of RCID and MCID supported for a
given QRID be identical in all capacity and bandwidth controllers in the
system.
The capacity and bandwidth controllers use the configurations that were
established for the RCID and AT in the request through the QRI corresponding
to the QRID in the request. Likewise the counters to count resource usage are
selected using MCID and the QRID in the request and a QRI can be used to
access counters associated with the corresponding QRID.
By default all resources in the capacity and bandwidth controllers may be allocated using any of the QRI. The controllers may optionally support reservation of resources for use by a QRI. When such reservation is supported the capacity or bandwidth reserved for a QRI may only be used by requests that have the corresponding QRID. Supporting resource reservation capability allows effective partitioning of the shared resources among SDs sharing the resources.
To reserve capacity to a QRI, the capacity controllers support a new operation
called CONFIG_QRI_LIMIT (OP=4). The capacity to be reserved is specified
using the cc_block_mask register. The AT and RCID fields of the
cc_alloc_ctl register are ignored by this operation. The mask specified in
cc_block_mask for this operation must have a contiguous run of 1s and an
implementation may require the mask to reserve at least one capacity block;
else the operation will fail with STATUS=5. The CONFIG_QRI_LIMIT operation
may be requested once following reset. If the operation is requested again then
the operation will fail with STATUS=2. On successful completion of the
operation, the cc_capabilities.NCBLKS shall update to a value that is the
number of 1 bits in value held in the cc_block_mask and only bits NCBLKS-1:0
are writable in cc_block_mask.
|
Note
|
The |
|
Note
|
Let’s consider a cache with
|
To reserve bandwidth to a QRI, the bandwidth controllers support a new operation
called CONFIG_QRI_LIMIT (OP=4). The AT and RCID fields of the
bc_alloc_ctl register are ignored by this operation. Likewise, the sharedAT
and useShared fields of the bc_bw_alloc register are ignored by this
operation. The bandwidth to be reserved is specified using the Rbwb field of
the bc_bw_alloc register and Mweight, when not equal to 0, represents a
proportional share of non-reserved or unused bandwidth that may be used by the
RCIDs associated with this QRI. When the Mweight is equal to 0, the Rbwb is
a hart limit and the RCIDs associated with this QRI are not eligible to use
unused or non-reserved bandwidth. The CONFIG_QRI_LIMIT operation may be
requested once following reset. If the operation is requested again then the
operation will fail with STATUS=2. On successful completion of the operation,
the bc_capabilities.MRBWB shall update to a value specified in Rbwb. When
Mweight for a QRI is not 0, then a two level weighted sharing of unused or
non-reserved bandwidth occurs. When the Mweight parameter for a QRI is not set
to 0, the amount of unused bandwidth allocated to QRID=x during contention
with another QRI that is also permitted to use unused bandwidth is determined
by dividing the Mweight of QRID=q by the sum of the Mweight of all other
contending QRIs. This ratio Pq is determined by equation (1). This weight share
of the unused bandwidth made available to a QRI is then shared among the
contending RCIDs of that QRI using the weights configured for the RCIDs.
|
Note
|
Consider a bandwidth controller that supports two QRIs. For brevity, this
example controller does not support bandwidth allocation by access-type
|
|
Note
|
The bandwidth enforcement is typically work-conserving, meaning that it allows
unused bandwidth to be used by QRIs enabled to use it even if they have consumed
their When contending for unused bandwidth, the weighted share is typically computed among the QRIs that are actively generating requests in that accounting interval and have a non-zero weight programmed. |