Witz Stephan

Abstract Reference: 30788
Identifier: O1.3
Presentation: Oral communication
Key Theme: 4 Long-term Management of Data Archives 

Towards a Self-Healing Archive

Authors:
Witz Stephan, Lyons Daniel, Plank Jen, Arora Jitin

The new NRAO Archive encompasses data from the Jansky VLA, the legacy VLA, the Green Bank Telescope and the VLBA while additionally providing access to ALMA data stored and managed separately. In this environment, metadata is extracted centrally but generated independently by different software for each instrument. Errors in metadata generation and extraction are unavoidable, but after fixing the bug, how do you correct the data? This paper introduces a self-healing approach that leverages otherwise idle archive storage nodes by having them continuously re-parse stored metadata with the latest software. Upon detecting a difference, the re-parser can take certain actions, such as updating incorrect records in the searchable metadata database, or broadcasting a notification. Data validity can be verified at the same time if desired.