Supporting Archival Snapshots use-case

[travisperson]

Archival snapshots are just a classification of snapshots. They are produced basically in the same way as any lightweight snapshot. The main difference is their frequency and the size of the export itself. The Filecoin chain archiver can easily be configured to execute on archival snapshots by just running a job every 24 hours with an export of 2880 epochs. However, in addition to just creating the exports, we want some additional information to be stored alongside the exports.

This metadata is covered in issue #16. To ensure the metadata is correct we want to make use of a new lotus method ChainExportRange (filecoin-project/lotus#8641) that will provide better guarantees around ensuring data is available.

The current thought is that we could produce exports of exactly 2880 with no overlap. However, I think this risk is just not worth it and instead, we will want to just run exports with 3000, or an additional 120 (2 hours) of TipSet data. I still think we should use the new ChainExportRange so that we can provide accurate metadata information alongside the exports.

In addition the ChainExportRange will support CARv2 exports, which current exports do not support. Using CARv2 will provide an index that will make searching through the archives faster and allow for broader uses of the snapshots.

[travisperson]

• Implement ChainExportRange (Add ChainExportRange API method lotus#8641)
• Update Chain Archiver to use new ChainExportRange
• Write out metadata file alongside all exports (Snapshot metadata #16)
• Support creating exports over offline lotus repo