Skip to main content

Improve eTRM Sync Resilience for Invalid Measure Package Data


CEDARS synchronizes draft Measure Package data from eTRM to support Load Shapes uploads. We have encountered Measure Packages containing data that does not meet the assumptions required for processing within CEDARS. Examples include unexpected data formats, and invalid baseline data used for load shape calculations.

Because CEDARS does not control the quality or completeness of data maintained in eTRM, additional handling is needed to ensure that a single problematic Measure Package does not prevent the synchronization of all other valid Measure Packages.

Problem

Currently, when invalid or unexpected data is encountered during synchronization, the entire sync process can fail. This creates several issues:

  • Valid Measure Packages may not be synchronized into CEDARS.

  • Users have limited visibility into which Measure Packages caused the failure.

  • Troubleshooting requires investigation by development teams rather than providing actionable information to users and measure developers.


Proposed Solution

Enhance the synchronization process to be more resilient when invalid Measure Package data is encountered. When a Measure Package cannot be processed:

  • The sync should continue processing all remaining Measure Packages.

  • The problematic Measure Package should be excluded from synchronization.

  • The user should be informed which Measure Packages were excluded.

  • A clear reason for the exclusion should be provided where possible.

  • The system should support a broad range of validation failures and unexpected data conditions, rather than only handling known examples.

Select the module in CEDARS
Select user type
Status: Completed2 comments

Log in to comment and vote

Comments2

  • cedars team

    Team•

    Sep 16

    This work is available on cedars-2 for testing.

  • cedars team

    Team•

    Jul 23

    Attaching some ideas we had for surfacing which measure packages were skipped in the sun and why. Feedback would be appreciated and answers the following questions:

    1. Are there additional known eTRM data exclusions that aren’t on the mockup?

    2. Do we need to call out specific permutations or rows, or can we keep it more general to the measure package?

    3. Are there any other identifying fields from eTRM we should include here?

    4. Is a download of the exclusion list helpful?

    5. Do we want to keep a log of all exclusions or just the last sync?

      Load Shape skipped eTRM data.pdf

      1.7 MB• Document