Dedicated indexing script

[aidanheerdegen]

A dedicated indexing script has a number of advantages:

• Implement complicated logic for optimising indexing, without having to alter the basic API which is ill suited for this task
• Could impose some restrictions on indexing to improve quality of the database meta-data. Specifically could require a metadata.yaml file to be present with some essential fields filled before indexing the data
• Requiring a metadata.yaml file would simplify the process of deciding what is an experiment, as it is then simply a directory containing a metadata.yaml file
• With above changes could use a yaml configuration file to define which directories to index
• A config file would allow for the concept of "collections" with associated meta-data, i.e. an extra layer in the hierarchy in the DB. This would allow for experiment name degeneracy, as experiment names might only have to be unique within collections.

[access-hive-bot]

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/cosima-cookbook-updating-needs/130/2