Hosted on the Open Science Framework

A Clean Sweep: The Tools and Processes of a Successful Metadata Migration

It’s been a hectic few days since the flurry of posts last week about trying to access my ancient (1998!) PowerPoint 4.0 slides. I should have posted on the outcome much sooner, but Real Life…

The majority of information about science, culture, society, economy and the environment is born digital, yet the underlying technology is subject to rapid obsolescence. One solution to this obsolescence, format migration, is widely practiced and supported by many software packages, yet migration has well known risks. For example, newer formats – even where similar in function – do not generally support all of the features of their predecessors, and, where similar features exist, there  may be significant differences of interpretation.There appears to be a conflict between the wide use of migration and its known risks. In this paper  we explore a simple hypothesis – that, where migration paths exist, the majority of data files can be safely migrated leaving only a few that must be handled more carefully – in the context of several scientific data formats that are or were widely used. Our approach is to gather information about  potential migration mismatches and, using custom tools, evaluate a large collection of data files for the incidence of these risks. Our results support our  initial hypothesis, though with some caveats.  Further, we found that writing a tool to identify “risky” format features is considerably easier than  writing a migration tool.

Xena is a digital preservation tool that first identifies a digital object’s file format, and for certain formats then converts the object into an open format. Xena also creates a base64-encoded version of the file, (using only printable ASCII characters) with an XML metadata wrapper.