-
Notifications
You must be signed in to change notification settings - Fork 31
AEMO Tables
This page summarise the different AEMO tables available using NEMOSIS. It provides information on the contents of the tables, where the raw data can be found online and how you should expect the tables to behave when filtered. The behaviour described here should be taken as the idealised behaviour, or how NEMOSIS is intended to behave. If you observe something different, congratulations! This is your opportunity to contribute, raise an issue and help make NEMOSIS better.
Most tables available through NEMOSIS are dynamic tables. This means they are tables that when queried will return different results depending on the start and end time provided, i.e the contents of the tables change with respect to time.
Many of the dynamic tables are from AEMO's Market Management System (MMS) dataset. A full summary of these tables is available in AEMO's MMS Data Model Report.
The summaries that follow are based partially on this document, but are also interpretive in nature as a precise definition of each table is not always provided. The names of the following sections aim to be descriptive, with the actual MMS name provided in parenthesis. Note: NEMOSIS uses the MMS name.
The data contained in each table is briefly summarised by referring to columns they contain. For more detail on each column, see the column summary.
The columns in many of the tables that are obtained by NEMOSIS are a subset of all the columns available via the MMS dataset. The full set can be accessed using lower level NEMOSIS functions. To view a list of the full set of columns search for the table name in the MMS data model report.
The following tables all contain the column SETTLEMENTDATE. In the context of these tables, this column defines the particular 5 min dispatch interval or 30 min trading interval (prior to 5-minute settlement) that a data record applies to. Therefore, when NEMOSIS queries these tables, only records when the SETTLEMENDATE timestamp is between the start (exclusive) and end time (inclusive) will be returned. Some tables do contain this column but do not conform to this generalisation and details of exceptions are given on a per table basis.
This table reports the results of the central dispatch process by 5 min interval and by dispatch unit.
Columns: SETTLEMENTDATE, DUID, INTERVENTION, DISPATCHMODE, AGCSTATUS, INITIALMW, TOTALCLEARED, RAMPDOWNRATE, RAMPUPRATE, LOWER5MIN, LOWER60SEC, LOWER6SEC, RAISE5MIN, RAISE60SEC, RAISE6SEC, LOWERREG, RAISEREG, SEMIDISPATCHCAP, AVAILABILITY.
This table provides a summary of the dispatch results (as reported in DISPATCHLOAD) aggregated to the 30 min trading interval level by averaging.
Columns: As per the table DISPATCHLOAD
This table reports the results of the central dispatch process by 5 min interval and by interconnector.
Columns: SETTLEMENTDATE, INTERCONNECTORID, INTERVENTION, METEREDMWFLOW, MWFLOW, MWLOSSES.
This table provides a summary of the interconnector dispatch results (as reported in DISPATCHINTERCONNECTORRES), aggregated to the 30 min trading interval level by averaging.
Columns: As per the table TRADINGTRADINGINTERCONNECT
This table reports the results of the central dispatch process by 5 min interval and by market region.
Columns: SETTLEMENTDATE, REGIONID, INTERVENTION, TOTALDEMAND, AVAILABLEGENERATION, AVAILABLELOAD, DEMANDFORECAST, DISPATCHABLEGENERATION, DISPATCHABLELOAD, NETINTERCHANGE, EXCESSGENERATION, LOWER5MINLOCALDISPATCH, LOWER60SECLOCALDISPATCH, LOWER6SECLOCALDISPATCH, RAISE5MINLOCALDISPATCH, RAISE60SECLOCALDISPATCH, RAISE6SECLOCALDISPATCH, LOWERREGLOCALDISPATCH, RAISEREGLOCALDISPATCH, INITIALSUPPLY, CLEAREDSUPPLY, TOTALINTERMITTENTGENERATION, DEMAND_AND_NONSCHEDGEN, UIGF, SEMISCHEDULE_CLEAREDMW, SEMISCHEDULE_COMPLIANCEMW.
This table provides a summary of the region dispatch results (as reported in DISPATCHREGIONSUM), aggregated to the 30 min trading interval level by averaging.
Columns: As per the table DISPATCHREGIONSUM
This table reports the price of various market services by 5 min interval and by market region.
Columns: SETTLEMENTDATE, REGIONID, INTERVENTION, RRP, RAISE6SECRRP, RAISE60SECRRP, RAISE5MINRRP, RAISEREGRRP, LOWER6SECRRP, LOWER60SECRRP, LOWERREGRRP, PRICE_STATUS.
This table provides a summary of the region dispatch results (as reported in DISPATCHPRICE), aggregated to the 30 min trading interval level by averaging.
Columns: As per the table DISPATCHPRICE
This table reports the SCADA values of scheduled, semi-scheduled and non-scheduled units where available.
Columns: SETTLEMENTDATE, DUID, SCADAVALUE.
This table reports generic constraints imposed on the dispatch process by 5 min interval, to the best of the authors knowledge it is the definitive information on which constraints were used in each dispatch interval. The definition of the constraints set out in this table can be found in the tables GENCONDATA, SPDREGIONCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and SPDINTERCONNECTORCONSTRAINT. The mapping between tables is from CONSTRAINTID to GENCONID_EFFECTIVEDATE, GENCONID_VERSIONNO to GECONID and EFFECTIVEDATE and VERSIONNO.
Columns: SETTLEMENTDATE, RUNNO, CONSTRAINTID, INTERVENTION, RHS, MARGINALVALUE, VIOLATIONDEGREE, LASTCHANGED, GENCONID_EFFECTIVEDATE, GENCONID_VERSIONNO, LHS.
This table sets out the capacity bids used in the central dispatch process. The records are by 5 min interval, by dispatch unit ID and by bid type. It also contains important constraining variables that generators submit with their bids.
Columns: DUID, BANDAVAIL1, BANDAVAIL2, BANDAVAIL3, BANDAVAIL4, BANDAVAIL5,BANDAVAIL6, BANDAVAIL7, BANDAVAIL8, BANDAVAIL9, BANDAVAIL10, MAXAVAIL, RAMPUPRATE, RAMPDOWNRATE, BIDTYPE, SETTLEMENTDATE, ENABLEMENTMIN, ENABLEMENTMAX, LOWBREAKPOINT, HIGHBREAKPOINT, INTERVAL_DATETIME.
This table set out the price bids used in the central dispatch process. The records are by day, but apply to all dispatch intervals on that market day. Note the definition of a market day in the NEM, as starting at 04:00 AM, means that a record with SETTLEMENTDATE "2017/01/01 00:00:00" applies to all intervals between (inclusive) "2017/01/01 04:05:00" and "2017/01/02 04:00:00".
Columns: SETTLEMENTDATE, DUID, BIDTYPE, OFFERDATE, VERSIONNO, PRICEBAND1, PRICEBAND2, PRICEBAND3, PRICEBAND4, PRICEBAND5, PRICEBAND6, PRICEBAND7, PRICEBAND8, PRICEBAND9, PRICEBAND10, T1, T2, T3, T4
The filtering by date for these tables occurs on the EFFECTIVEDATE column. The most recent effective dates applicable are returned, including multiple versions if they exist. For longer query windows (i.e greater time between start and end time) multiple effective dates for the same record maybe returned, as different records will apply to different times within the window.
This table defines constraint matrix coefficients for regions, the absence of factor implies a zero coefficient, Note the table just defines constraints as they relate to regions, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: REGIONID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, LASTCHANGED, BIDTYPE.
This table defines constraint matrix coefficients for connection points, the absence of factor implies a zero coefficient. Note the table just defines constraints as they relate to connection points, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: CONNECTIONPOINTID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, BIDTYPE, LASTCHANGED.
This table defines constraint matrix coefficients for interconnectors, the absence of factor implies a zero coefficient. Note the table just defines constraints as they relate to interconnectors, a full definition of any one constrain may require combining data from GENCONDATA, SPDREGIONCONSTRAINT, SPDINTERCONNECTORCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINT and DISPATCHCONSTRAINT.
Columns: INTERCONNECTORID, EFFECTIVEDATE, VERSIONNO, GENCONID, FACTOR, LASTCHANGED
This table defines general information about constraints applied to the central dispatch process. It maps to SPDREGIONCONSTRAINT, SPDCONNECTIONPOINTCONSTRAINTS and SPDINTERCONNECTORCONSTRAINT with the columns GECONID, EFFECTIVEDATE and VERSIONNO. Note the RHS values given in this table may not always be the ones used in dispatch, some RHS values are dynamically determined, for actual RHS values used in dispatch see the table DISPATCHCONSTRAINT.
Columns: GENCONID, EFFECTIVEDATE, VERSIONNO, CONSTRAINTTYPE, CONSTRAINTVALUE, DESCRIPTION, GENERICCONSTRAINTWEIGHT, LASTCHANGED, DISPATCH, PREDISPATCH, STPASA, MTPASA, LIMITTYPE, REASON
This table provides information on dispatch units, other dispatch unit information can be found in the table DUDETAILSUMMARY. Other useful information on the units that is not directly relevant to dispatch is more likely to be found in the Registration Master List.
Columns: DUID, EFFECTIVEDATE, VERSIONNO, CONNECTIONPOINTID, REGISTEREDCAPACITY, AGCCAPABILITY, DISPATCHTYPE, MAXCAPACITY,LASTCHANGED,STARTTYPE. NORMALLYONFLAG.
This table provides a second source of information on dispatch units. Other useful information on the units that is not directly relevant to dispatch is more likely to be found in the Registration Master List. This table is by START_DATE and END_DATE rather than EFFECTIVEDATE, but the behaviour is similar. The difference in behaviour is that records with an END_DATE before the start time specified are excluded from the results.
Columns: DUID, START_DATE, END_DATE, DISPATCHTYPE, CONNECTIONPOINTID, REGIONID, STATIONID, PARTICIPANTID, LASTCHANGED, TRANSMISSIONLOSSFACTOR, STARTTYPE, DISTRIBUTIONLOSSFACTOR, SCHEDULE_TYPE, MAX_RAMP_RATE_UP, MAX_RAMP_RATE_DOWN.
This table is used in the determination of recovery payments for regulation services, more information is available from AEMO here. This data maps to Elements_FCAS table by ELEMENTNUMBER column and the Variables_FCAS by VARAIBLENUMBER column. Note NEMOSIS complies this table from current and historical data published by AEMO, however there is a gap in data availability between the current and historical data sets and this may result in NEMOSIS failing to compile data from this table.
Columns: TIMESTAMP, ELEMENTNUMBER, VARIABLENUMBER, VALUE, VALUEQUALITY
While these tables may be periodically updated by AEMO, they do not contain a time based column and hence will not be filtered by any start and end time. If you require the most up to date tables from AEMO delete or rename the corresponding tables from your local data cache and NEMOSIS will download the most recent versions. Note the names provided in parenthesis below are NEMOSIS names not AEMO names.
This table defines the element numbers used in the 4 second FCAS data. Note it appears AEMO has deleted the market names from this data set and no publicly available mapping of element numbers to market names appears to exist. More information is available from AEMO here
Columns: ELEMENTNUMBER, ELEMENTNAME, ELEMENTTYPE, NAME.
This table defines the variable numbers used in the 4 second FCAS data. More information is available from AEMO here
Columns: VARIABLENUMBER, VARIABLETYPE.
This table provides registration data for generators and loads in the NEM. It is derived from excel sheet provide by AEMO here. Note NEMOSIS uses the Generators and Scheduled Loads tab, but drops duplicate entries based on DUID values, this removes unit level information such as capacity, but enables the dataset to be more easily combined with MMS datasets. AEMO also removes retired generators from this dataset which can be problematic for historical analysis, we are looking to overcome this issue by hosting historical versions of the Registration and Exemption file on GitHub but this has not been implemented yet.
Columns: Participant, Station Name, Region, Dispatch Type, Category, Classification, Fuel Source - Primary, Fuel Source - Descriptor, Technology Type - Primary, Technology Type - Descriptor, Aggregation, DUID.