Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document /vsigzip/ convention when reading gzip files in docstring, examples and tutorial #235

Open
3 tasks
EwoutH opened this issue Aug 25, 2024 · 3 comments · Fixed by #237
Open
3 tasks
Labels
docs Release notes label

Comments

@EwoutH
Copy link
Member

EwoutH commented Aug 25, 2024

Document the weird /vsigzip/ convention when reading gzip files in docstring, examples and tutorial. So basically this piece of infromation:

mesa-geo's from_file() immediately calls rasterio.open() without modifying the argument, you should do this:

raster_layer = mg.RasterLayer.from_file(
    f"/vsigzip/{elevation_file}", cell_cls=LakeCell, attr_name="elevation"
)

Rasterio (from GDAL) has gzip support built-in: https://gdal.org/user/virtual_file_systems.html#vsigzip-gzipped-file.

Here comes the plot twist: /vsigzip/ is not part of the data path, but a weird convention by GDAL:

/vsigzip/my.gz # (relative path to the .gz)
/vsigzip//home/even/my.gz # (absolute path to the .gz)
/vsigzip/c:\users\even\my.gz

This should be thoroughly documented in Mesa-geo, since this is a very specific piece of knowledge you won't otherwise find out and it's publicly exposed in our user example models.

Related tickets:

Todo

  • Mention in tutorial
  • Mention in docstring
  • Mention in examples
@EwoutH
Copy link
Member Author

EwoutH commented Sep 1, 2024

Closed by #237.

@EwoutH EwoutH closed this as completed Sep 1, 2024
@wang-boyu
Copy link
Member

This may or may not be useful - just found a method from geopandas: geopandas.io.util.vsi_path that adds a prefix to zipped file paths and does nothing otherwise. geopandas handles vector data, but this method could be handy in converting file paths for our raster data.

@EwoutH
Copy link
Member Author

EwoutH commented Sep 1, 2024

Oh that’s interesting!

@EwoutH EwoutH reopened this Sep 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs Release notes label
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants