How long do we want CEDARS to hold onto CET output files?
Question for the data reporting PCG:
The CET module was added to CEDARS in 2018 and to date more than 120K CET runs have been done in production. We currently keep CET input/output files from runs indefinitely. It would help us plan for future disk size updates to know if we want to continue to keep all files and make them available to the person who uploaded them indefinitely, or if we want to set a window where files uploaded more than X years ago are deleted.
- Select the module in CEDARS
Log in to comment and vote
Comments1
cedars team
Sep 3
•Merged request
•1 vote
CEDARS server storage strategy & future growth management
Discuss our approach to managing CET run storage and planning for future growth.
Key points to consider:
Current approach: All CET runs (inputs and outputs) are currently stored indefinitely.
Problem: This leads to increasing storage demands.
Discussion points:
Should we remove test runs after a certain period?
Partial deletion:
Should we keep inputs only and delete outputs after a period?
Note: CET versions change, which may make old inputs unusable for reruns or audits.
Should we apply retention policies only to community users while keeping full data for PAs?
Position on file storage:
Reinforce that users are expected to track and store files internally.
CET is not intended to act as a long-term file repository.