Migration to DAM happens in several stages and with full migrations and delta migrations a few weeks apart. This page describes how DAM creates timelines for each entity and how new data transferred on delta migrations replace or cancel out previously migrated data.
Keys used in establishing timelines
For each of the migration file types, different keys will be used to identify a unique entity timeline. This will be a combination of the GLN in the file name and one or two fields in the actual files.
|MP - Metering Point – Målepunkt|
|CUS - Customer– Kunde|
|CO - Contract - Kontrakt|
Dimensions of time and overwriting data in migration
The final timeline migrated into Elhub is established by working with two dimensions of time. DAM will will let new data overwrite old data using predefined rules.
As established by ValidFrom and ValidTo in the migration files.
It is important to emphasize that each change of one of the represented values in the files should result in one new record. For example if the meter constant on the metering point has been changed it should result in two separate rows. One row representing the old state, with an end date, and another row with an empty end date representing the current new state.
Where in the sequence of migration is the data transmitted. Based on when the file was transmitted and where in the file the data is located.
- Oldest: First row in the first transmitted file
- Newest: Last row in the last transmitted file
Overwriting data in delta migration
For each timeline, new data will overwrite old data (migration sequence) for the validity period of the new data. This is the key feature of delta migration. This can be either updates, corrections, cancellations or any other changes to the data.
For metering points, the new data will only overwrite for the transmitted validity period, allowing partly overwritten data to still be seen as OK for validity periods not covered by the new data. Gaps or partly overwritten data will be reported as errors to grid owners, but not to balance suppliers.
For contracts, the new data will overwrite all older migrated data that is partly overlapping the validity period of the new data row.
Resubmitting whole metering points on delta migration
The overwriting rules defined above gives possibilities, but also limitations, in sending in full metering point history on delta migration for single metering points. This means that metering points can be flagged for delta in the source system and the whole metering point, with contracts and customers, are transmitted to DAM on delta migration. But there is also a need to consider the need to explicit cancellation of data.
|File||Market party||Gaps allowed||Partly overlapping data allowed||Must have current data - i.e. row with empty ValidTo|
|MP - Metering Point – Målepunkt||Grid Owner|
The exception to the rule is where a historic customer address has been classified as "secret". This will however result in gap errors reported since these metering points should never be flagged as special in any way.
|No||Yes. Elhub must allways know the current state of the metering point. Else it will not be transmitted into Elhub.|
|MP - Metering Point – Målepunkt||Balance Supplier||Yes||Yes||No|
|CUS - Customer– Kunde||Grid Owner||Not relevant - Only the newest data is used||Not relevant - Only the newest data is used||Should. Only the newest data about a customer is used by DAM. If only older validity period is transmitted, then this data will be used.|
|CUS - Customer– Kunde||Balance Supplier|
|CO - Contract - Kontrakt||Grid Owner||Yes||Yes, DAM will cancel the partly overlapping row.||No|
|CO - Contract - Kontrakt||Balance Supplier|
Cancellation of migrated data
As of delta migration has been removed from the GoLive plan for Elhub. This functionality will remain in DAM, but should not be used.
Cancellation of migrated data is used when previously migrated data has to be removed from the timeline of a metering point or a contract. Cancellation of customer data is not relevant because only the newest data is used and only active customers for non-cancelled contracts on valid metering points will be transferred into Elhub.
Cancellation of a full timeline
Relevant for metering points and contracts:
- cancelling the metering point and contract history completely ( it should never have been migrated if the first place )
- the amount or nature of changes done on the metering point or contract timeline requires a "reset" of the timeline before transmitting new data
To cancel a full timeline, send a row with empty ValidFrom and ValidTo in the future. All other data in the row should be correct to eliminate the possibility that any format validation errors are triggered.
If this solution is used to "reset" timeline before transmitting new data, the order of which the data is transmitted is very important. The cancellation row must be earlier in the file than the new data.
Cancellation of contracts
If a migrated contract is cancelled between full and delta migration, it needs to be explicitly cancelled in the migration files. This is done by using special values in CO.ContractCode (GC, BC, LC)
The cancellation contract will cancel all previously migrated data that is partly (or fully) overlapping the validity period of the cancellation contract, according to the general rules for overwriting stated above.
This PDF contains drawn examples of how the timeline is checked and established in DAM (Norwegian only).