Maestral v1.2.1

This update provides bug fixes and some improvements to error handling. Major changes don't regard Maestral itself but its distribution: a Docker image is now available, thanks to @aries1980, and the macOS app bundle has been rebuilt with the macOS 11 SDK, providing full compatibility from macOS 10.14 Mojave to macOS 11.0 Big Sur.

Added:

• Added a Docker image. The docker image is based on Linux and does not currently include a GUI.
• Added -V, --version option to the command line interface to show the version and exit.

Changed:

• Improves handling of database related errors such as database integrity, missing read / write permissions for the database file, etc.
• Improves handling of errors when the keyring cannot be unlocked to delete credentials during an unlink.
• Improves handling of errors when the keyring where Dropbox credentials are stored becomes unavailable, e.g., has been uninstalled.
• Never start a subprocess when maestral is run with the -f, --foreground option. Previously, any required setup such as linking, etc, would still be performed in a subprocess.
• Minor tweaks and improvements to the macOS GUI.
• Allow sending desktop notifications in Linux before the daemon's event loop has started. This is useful for error messages which occur early during the initialization.
• Improves log messages when the connection to Dropbox is lost.
• Performance improvements to maestral activity in case of very large sync queues.

Fixes:

• Fixes a database integrity error due to an unfulfilled unique constraint.
• Fixes an issue when the daemon is launched with systemd where systemd would unexpectedly receive notifications from a subprocess instead of the main process.
• Fixes an issue which would prevent syncing from automatically resuming after moving the local Dropbox directory with maestral move-dir or through the GUI.
• Fixes a green background for sync issue views in the macOS GUI.
• Fixes an issue where the system tray icon in KDE Plasma could fall back to the regular app icon or not show up at all,
• Fixes an issue where the user may be asked to unlock or grant access to the system keyring twice on startup if access denied the first time.

Dependencies:

• Adds alembic dependency for database migrations.

Maestral v1.2.0

The local file index and sync history are now stored in a SQLite database. After the update, Maestral will reindex your Dropbox to populate the new index. This may take a few minutes, depending on the number of files in your Dropbox.

This change enables several improvements to the command line interface and GUI: The command maestral activity now shows the progress of individual uploads or downloads. maestral history has been added to list recent sync events. In the GUI, the recent changes menu now has been replaced by a "Activity" window which shows all sync events of the past week.

This release also introduces clickable desktop notifications, performance improvements to indexing of local file changes, and bug fixes and smaller changes listed below.

Finally, this release introduces support for macOS 11 (Big Sur).

Added:

• Added an option --external/-e to maestral log show to open the log in the platform's default program instead of showing it in the console.
• Added a CLI command history to show all sync events of the past week.
• Added a "Activity" window to show all sync events of the past week.
• Desktop notifications are now clickable: for a single file change, clicking the notification will show the file in the platform's file manager. For a deletion, the Dropbox website is opened to provide options for restoring the file or folder.
• Use entry points to discover GUI frontends. 3rd party GUIs can register a maestral_gui entry point to be launched with the maestral gui CLI command. If installed, maestral gui will default to the 1st party maestral-cocoa or maestral-qt GUIs on macOS and Linux, respectively.

Changed:

• Transition to short-lived auth tokens for newly linked accounts.
• Transition to OAuth scopes for app permissions.
• Save all sync history and local index in SQLite database.
• Reduce unnecessary path conversions during indexing of local changes.
• Improved performance on case-sensitive file systems.
• Sync remote changes in filename even if they are only a change in casing. Those changes where previously ignored.
• Attempt to preserve local file permissions when syncing unless the file id has changed. Dropbox servers do store file permissions but don't make them available through the public API. We therefore cannot sync file permissions and instead choose not to overwrite locally set permissions on every download.
• Changed return type of Maestral.get_activity from namedtuple to dict for better consistency throughout the API. Every uploading or downloading item will have 'size' and 'completed' entries to monitor the progress of syncing individual items.
• The CLI command maestral activity now shows the progress of uploads and downloads for individual files.
• Introduced type annotations throughout and fixed a few type-related bugs.
• Added a field "Sync threads" to the output of the CLI command maestral status.
• The output of maestral ls is now printed in a grid, similar to the ls command included with most platforms.
• The macOS app bundle now uses Python 3.8, leading to some performance improvements when moving or copying file system trees.
• Prepared the GUI for changes in macOS Big Sur: use native alerts and dialogs wherever possible and refactor loading of libraries.
• Use an asyncio event loop instead of Pyro's event loop to run the daemon. This enables integration with the Cocoa run loop and callbacks when clicking notifications.

Fixed:

• Fixes a bug where throttling of sync threads would raise an error when we cannot determine the CPU count.
• Fixes a bug where sending SIGTERM to the daemon process would raise an error when we cannot determine its PID. Now, Stop.Failed is returned instead.
• Fixes a bug which would result in incorrect systemd unit files for non-default config file names. Please disable and re-enable autostart with maestral autostart -Y|-N to replace old unit files.
• Fixes a possible race condition when creating the cache directory.
• Fixes error handling when a file is changed while uploading.

Removed:

• Support for config names with spaces. Spaces could cause issues with autostart entries on some platforms.
• The ability to run the daemon in a separate thread. The daemon must now always be run inits own process.

Dependencies:

• Replaced jeepney dependency on Linux with dbus-next.

Maestral v1.1.0

This release expands the CLI functionality and improves the handling of file modification times during upload and download (used for display purposes only). It also fixes bugs with the "start on login" functionality of the macOS app bundle. After updating, please toggle "start on login" in the GUI or maestral autostart in the CLI to replace any old login items on macOS.

Added:

• Added --include-deleted | -d option to maestral ls.
• Added --long | -l option to maestral ls to include metadata in listing.
• Added maestral revs command to list revisions of a file.
• Added maestral restore command to restore an old revision of a file.

Changed:

• Always create config directory if it does not yet exist.
• Improved performance of converting Dropbox paths to correctly cased local paths.
• Renamed macOS executable inside app bundle from "main" to "Maestral". This results in more informative process names.
• Local files are now created with the "last modified" time provided by Dropbox servers. This only applies to new downloads. To update existing modified times, you will need to delete and redownload your Dropbox folder.

Fixed:

• Fixes a thread-safety issue with desktop notifications.
• Fixes a thread-safety issue when two frontends try to start or stop syncing at the same time.
• Fixes an issue where Maestral could incorrectly identify a file system as case sensitive if the Dropbox folder and temporary directory are on partitions with different file systems.
• Fixes incorrect file modification times uploaded to Dropbox for timezones outside of UTC. Those times are used for display purposes only.
• Fixes an issue where the maestral autostart -Y CLI command would start the GUI on login in case of the macOS app bundle.

Maestral v1.0.3

Thanks to generous Github sponsors, the macOS app bundle is now properly signed and notarized.

Edit: The previous macOS app bundle had the wrong version number. This has been corrected now.

Changed:

• Both "-h" and "--help" can now be used to print help output for a command.
• Show both the daemon and GUI version in the GUI settings window.
• The command line tool bundled with the macOS app now provides proper help output.
• Significantly reduced CPU usage of the GUI on macOS.
• The macOS app now uses a hardened runtime and is properly signed and notarized.

Fixed:

• Fixes an issue which could lead to the local Dropbox folder being moved before syncing has been paused.
• Fixes an issue where download errors would show a rev number instead of the Dropbox path.
• Fixes a race condition when two processes try to start a sync daemon at the same time.
• Fixes an issue in the macOS GUI where updating the displayed sync issues could fail.
• Fixes truncated text in the macOS setup dialog.
• Fixes an issue on fresh macOS installs where creating autostart entries could fail if /Library/LaunchAgents does not yet exist.
• Fixes an issue in the macOS app bundle where installing the command line could tool would fail if /usr/local/bin is owned by root (as is default on a fresh install). Now, the user is asked for permission instead.

Dependencies:

• Removed lockfile dependency.
• Added fasteners dependency.

Maestral v1.0.3.beta1

Changed:

• Significantly reduced CPU usage of the GUI on macOS.
• Show both the daemon and GUI version in the GUI settings window.

Fixed:

• Fixes an issue which could lead to the local Dropbox folder being moved before syncing has been paused.
• Fixes an issue where download errors would show a rev number instead of the Dropbox path.
• Fixes a race condition when two processes try to start a sync daemon at the same time.
• Fixes an issue in the macOS GUI where updating the displayed sync issues could fail.

Dependencies:

• Removed lockfile dependency.
• Added fasteners dependency.

Maestral v1.0.2

This release fixes bugs in the command line interface.

Fixed:

• Fixes a crash of the CLI when an update is available due to incorrect formatting of the
  update message.
• Fixes an error when listing the contents of an empty directory with maestral ls.

Maestral v1.0.0

This is the first stable release of Maestral. There have been numerous bug fixes to error handling and platform integration as well as a few bug fixes to syncing itself. There are also a few outward facing changes: Pausing Maestral now cancels all pending sync jobs instead of waiting for them to complete. The macOS GUI switches from Qt to using a native Cocoa interface and the macOS app bundle finally includes a full command line interface which can be installed from the settings window.

Added:

• Command line tools are now bundled with the macOS app bundle and can be installed from the settings window.
• Added support for config names with spaces.
• Switch from Qt to a native Cocoa GUI on macOS. This reduces the size of the app bundle from 50 MB to 20 MB.
• Expanded test suite to include sync tests.

Changed:

• Added '.dropbox' and '.dropbox.cache' to always excluded paths.
• Pausing sync now cancels all pending uploads and downloads.
• Quicker detection of connection problems.
• Faster sync of local deletions.
• The GUI now always launches a separate daemon process instead of an in-process daemon.
• Temporary files during a download are now stored inside the Dropbox directory at '.maestral.cache'. This guarantees that temporary files always reside on the same partition as the Dropbox folder itself.
• System tray icons are no longer installed in the platform theme in Linux. This is part of a workaround for a Qt issue on Linux desktops which causes unnecessarily large pixmap transfers over dBus when HiDPI support is enabled. Manually installed icons will still be respected.
• Switch from implicit grant to PKCE OAuth2 flow.
• Added public API to link a Dropbox account: Maestral.get_auth_url and Maestral.link. Frontends no longer need to import maestral.oauth.
• Moved all command line dialogs from the main API to the CLI module.
• Bumped watchdog requirement to >= 10.0.0 for more consistent error handling.
• Added explicit jeepny dependency for Linux. This is a dependency of keyring but we use it by itself as well.
• Improved the reliability of ignoring file system events caused by Maestral itself.

Fixed:

• Fixes an issue where a dropped internet connection during startup could result in continuous retries until the connection is finally established.
• Fixes an issue where downloads of newly included folders would not resume after being interrupted.
• Fixes an issue which could lead to false conflicting copies of folders in some cases.
• Fixes the handling of inofify limit and permission errors when starting a file system watch.
• Fixes handling of errors from too long file names.
• Handle errors due to file names which are not allowed on the local file system.
• Fixes handling of some uncaught insufficient disk space errors.
• Fixes incorrect autostart entries on macOS.
• Fixes a crash when running Maestral as a systemd service without python-systemd installed.
• Fixes an issue when checking for updates if the list of releases from Github includes dev releases.
• Fixes an issue where only remote changes would be listed in 'Recent changes' in the GUI.
• Fixes the alignment of comboboxes in the Qt GUI on macOS.
• Fixes a crash on macOS when no notification center is available, for instance in a headless session or on Github test runners.
• Fixes a crash on Linux when the command line tool notify-send is not available.
• Fixes an issue where sync errors would have incomplete path information.
• Resolves an issue where indexing a large Dropbox folder with > 100,000 items would continuously timeout and restart in some cases.

Removed:

• Removed migration code for versions < 0.6.3. If you want to update to v1.0.0, please make sure to upgrade to at least version 0.6.3 first or unlink your Dropbox before updating to v1.0.0.
• Removed u-msgpack dependency.

Maestral v0.6.4

The release provides bug fixes and minor improvements to the command line and graphical user interfaces. Importantly, it fixes an issue where some files could accidentally become un-indexed, resulting in incorrect conflict resolution.

Added:

• Config option to set the keyring backend. This defaults to 'automatic' but can be used to specify a preferred backend such as keyrings.backends.kwallet.DBusKeyring. You will need to migrate your credentials manually to the new keyring if you change this setting.
• Added a --verbose flag to maestral start and maestral restart commands to print log output to stdout.
• Added an API documentation for developers, available on Read the Docs.

Changed:

• During initial CLI setup, give the option to sync the entire Dropbox without paginating through individual folders to exclude.
• Limit the number of notifications to keep in the notification center. This will only work for some desktop environments.
• Fall back to plain text credential storage if neither Gnome Keyring, KWallet or any other storage implementing the Secret Service API can be found. A warning is shown when plain text storage is used.
• Settings and setup windows are no longer always kept on top in Linux.
• maestral start --foreground no longer prints log messages to stdout by default.

Fixed:

• Properly handle errors when moving files, for instance for sync conflicts.
• Fixes an issue where some files could accidentally become un-indexed, resulting in incorrect conflict resolution.
• Fixes an issue with macOS app bundles where the migration of configuration files was omitted after an update. This would result in a failure to start the daemon.
• Correctly specify the required version of six to work around an upstream issue in Dropbox.
• Fixes an issue where stdout would end up in the systemd journal in addition to the structured log messages.
• Fixed a bug where XDG_DATA_HOME was ignored.

Maestral v0.6.3

This release fixes a critical error introduced when updating to v9.5 of the Dropbox Python SDK which prevented any remote changes from being downloaded.

Changed:

• Show release notes from all releases since last update in update dialog.
• Use our own method instead of the psuitl package to determine the CPU usage. This eliminates the psuitl dependency which can be difficult to install on some systems.

Fixed:

• Fixes an issue with downloads failing because Dropbox Metadata is longer hashable from v9.5 of the Dropbox Python SDK.
• Fixed a StopIteration exception on startup when the location of the maestral CLI script cannot be found in the package metadata.
• Fixes an error when restarting the daemon with the "foreground" option.
• Fixed incorrect button labels in the setup dialog when choosing whether to replace or keep an old Dropbox folder. The labels "Replace" and "Cancel" where switched.
• Fixes a bug where the option "Unlink & Quit" in the "Revoked Access" error dialog would unlink but not quit Maestral.

Maestral v0.6.2

This release enables excluding individual files from syncing and fixes an issue which led to continuously retrying failed downloads. It also contains significant performance improvements to indexing, reduces the CPU usage when syncing a large number of files and introduces weekly re-indexing.

This release also introduces support for an ".mignore" file with the same syntax as gitignore. This feature is considered 'alpha' and may change in the future. Feedback is welcome.

Added:

• Support excluding files from sync. This uses the same 'selective sync' interface as excluding folders. Excluded files will be removed from the local Dropbox folder.
• Introduces an ".mignore" file to specify files that Maestral should ignore. The ".mignore" file must be saved in the local Dropbox folder. When excluding files or folders with selective sync (maestral exclude), they will be removed from the local folder and kept in the cloud only. The ".mignore" file enables the reverse: files or folders which exist locally will not be uploaded to Dropbox. It uses the same syntax as gitignore files and, similar to gitignore, files which are already tracked by Maestral will not be affected. More details are given in the Wiki.
• Added a config option "max_cpu_percent" to adjust the target maximum CPU usage per CPU core. This defaults to 20%, i.e., 80% total for a quad core CPU. Maestral will aim to remain below that percentage but this is not guaranteed.

Changed:

• Replaced the excluded_files and excluded_folders settings from the config file with a unified excluded_items setting. Entries from excluded_folders will be migrated to the excluded_items setting.
• Renamed methods which exclude / include folders to exclude_item etc.
• Speed up creation of local folders.
• When trying to create a file or folder with the same path as an item excluded by selective sync, the new item is now renamed by appending "selective sync conflict" instead of raising a sync issue. This is closer the behaviour of the official client.
• Significant performance improvements to indexing and file event processing. Indexing a remote Dropbox with 20,000 to 30,000 files and comparing it a local folder now takes ~ 5 min, depending on on the average file size.
• Introduced periodic reindexing every week. This has been made possible by the above performance improvements.

Fixed:

• Don't immediately retry when a download fails. Instead, save failed downloads and retry only on pause / resume or restart.
• Fixes missing cursor and resulting unexpected ValidationError during sync startup.
• Wait until all sync activity has stopped before moving the Dropbox folder. This avoids errors when trying to convert local to dropbox paths and vice versa during the move.
• Fixes an issue which would prevent some conflicting copies created by Dropbox from being downloaded.
• Correctly handle when a local item is renamed to an always excluded file name such as ".DS_STORE": the item is now deleted from Dropbox.
• Fixes an issue where sharing an existing folder from the Dropbox website would result in the folder being deleted locally. This is because Dropbox actually removes the shared folder from the user's Dropbox and then re-mounts it as a shared drive / file system. We handle this correctly now by leaving the local folder alone or deleting and re-downloading it, depending on the time elapsed between removal and re-mounting.
• Improves conflict resolution when a folder has been been replaced with a file or vice versa and both the local and remote item have un-synced changes.
• Fixes an issue where maestral stop would block until all pending syncs have completed. This could potentially take a very long time for large downloads. Instead, any interrupted downloads will be restarted on next launch.