Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add individual download progress #42

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Commits on Sep 19, 2024

  1. Add progress bars for individual downloads

    Additionally, this commit prevents the program from exiting if a single
    resource has an error. Instead, that error will be noted (and details
    will be given if the --verbose flag is passed), and downloading of
    subsequent resources will continue.
    
    One bug is that the `gdown` library manages its own progress bars, so
    they do not appear in the same way as the ones generated here. There are
    three possible solutions I see, from easiest to hardest
    
      1. Ignore it
    
      2. Pass the quiet option to gdown (eliminating any progress bar)
    
      3. Send a patch to gdown to accept a file descriptor to which to
         download, rather than a filename. (Unlikely to be accepted)
    
    Taking (1) for now!
    ptgolden committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    566a11e View commit details
    Browse the repository at this point in the history
  2. Don't call downloader functions from other downloader functions

    This prevents the resource from being logged twice. I thought about
    doing something more complex to allow the functions to call each other,
    but it was to messy to justify not repeating a couple lines of code.
    ptgolden committed Sep 19, 2024
    Configuration menu
    Copy the full SHA
    8c6fec8 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4f00931 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    a43956e View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2024

  1. Fix bug in s3 download code

    ptgolden committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    da03a63 View commit details
    Browse the repository at this point in the history