Gather requirements for a data loader for scivision (potential replacement for Intake)

[edwardchalstrey1]

Issue complete when: there is a draft SCIP with a spec

Coworking notes on this topic: https://github.com/alan-turing-institute/scivision-project-management/blob/main/meeting-notes/2023-01-12-coworking.md

Some more ideas:

• Make stardist datasource loadable via Python api #482
• Automate data repo creation #471
• [ENH] Datasets with labels (and in CSV format) #251
• [BUG] Support cloud-optimised datasets #625

[edwardchalstrey1]

Duplicate issue: #460 - will close to avoid clutter

[edwardchalstrey1]

@ots22 One thing that would be very handy would be if we could make it possible to do the following

Ideal version

data = load_dataset(<datasource url>)
model.predict(data)    # assuming this model var was already set up

Current version

as oppose to the intake version we have currently:

data_config = load_dataset(<datasource url>)
print([i for i in data_config.keys()])

choose a key... run to_dask .... and only then run model.predict

data = data_config.my_key().to_dask()
model.predict(data) 

[ots22]

Consider native support for common cases, like 'zip file of images' (would close #482)