Skip to content

Commit

Permalink
Create soft links to all files when saving a ReloadedArray.
Browse files Browse the repository at this point in the history
This is more consistent with the behavior of the other reuse modes and
allows link-based deduplication inference in packages like gypsum.
  • Loading branch information
LTLA committed Jan 25, 2024
1 parent 15c9707 commit 5cde6b7
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 24 deletions.
4 changes: 2 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Package: alabaster.matrix
Title: Load and Save Artifacts from File
Version: 1.3.6
Date: 2024-01-17
Version: 1.3.7
Date: 2024-01-25
Authors@R: person("Aaron", "Lun", role=c("aut", "cre"), email="[email protected]")
License: MIT + file LICENSE
Description:
Expand Down
27 changes: 11 additions & 16 deletions R/ReloadedArray.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,13 @@
#'
#' One obvious optimization is the specialization of \code{\link{saveObject}} on ReloadedArray instances.
#' Instead of loading the array data back into the R session and saving it again, the \code{saveObject} method can just link or copy the existing files.
#' This behavior is controlled by the optional \code{reloadedarray.reuse.files} option, which can be one of:
#' This behavior is controlled by the optional \code{reloadedarray.reuse.files} option in the \code{saveObject} method, which can be one of:
#' \itemize{
#' \item \code{"copy"}: copy the files from the previous \code{path} (as stored in the ReloadedArray object) to the new \code{path} specified in \code{saveObject}.
#' \item \code{"link"}: create a hard link from the files in the previous \code{path} to the new \code{path}.
#' \item \code{"copy"}: copy the files from the original directory (as stored in the ReloadedArray object) to the new \code{path} specified in \code{saveObject}.
#' \item \code{"link"}: create a hard link from the files in the original directory to the new \code{path}.
#' If this fails, we silently fall back to a copy.
#' This mode is the default approach.
#' \item \code{"symlink"}: create a symbolic link from the previous directory at \code{path} to the new \code{path}.
#' If this fails, we fall back to a copy with a warning.
#' \item \code{"symlink"}: create a symbolic link from the files in the original directory to the new \code{path}.
#' \item \code{"none"}: ignore existing files and just save the contents by calling \code{"\link{saveObject,DelayedArray-method}"}.
#' }
#'
Expand Down Expand Up @@ -94,28 +93,24 @@ setMethod("saveObject", "ReloadedArray", function(x, path, reloadedarray.reuse.f
x <- DelayedArray(s@seed)
return(callNextMethod())
}

if (reloadedarray.reuse.files == "symlink") {
if (file.symlink(s@path, path)) {
return(invisible(NULL))
} else {
warning("failed to create a symlink from '", s@path, "' to '", path, "', falling back to copies")
reloadedarray.reuse.files <- "copy"
}
}

manifest <- list.files(s@path, recursive=TRUE)
dir.create(path)

if (reloadedarray.reuse.files == "link") {
if (reloadedarray.reuse.files == "symlink") {
fun <- file.symlink
msg <- "link"
} else if (reloadedarray.reuse.files == "link") {
fun <- function(from, to) file.link(from, to) || file.copy(from, to)
msg <- "copy or link"
} else {
fun <- file.copy
msg <- "copy"
}

for (y in manifest) {
if (!fun(file.path(s@path, y), file.path(path, y))) {
stop("failed to copy '", y, "' from '", s@path, "' to '", path, "'")
stop("failed to ", msg, " '", y, "' from '", s@path, "' to '", path, "'")
}
}

Expand Down
9 changes: 4 additions & 5 deletions man/ReloadedArraySeed.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion tests/testthat/test-ReloadedArray.R
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,11 @@ test_that("ReloadedArrays save correctly", {
saveObject(obj, tmp, reloadedarray.reuse.files="symlink")
expect_identical(as.array(readObject(tmp)), arr)
if (.Platform$OS.type=="unix") {
expect_identical(Sys.readlink(tmp), dir)
expect_identical(Sys.readlink(file.path(tmp, "array.h5")), file.path(dir, "array.h5"))
}

# file.info() doesn't report the inode number so we don't have an easy way
# to distinguish between hard links and a copy. Oh well.
tmp <- tempfile()
saveObject(obj, tmp, reloadedarray.reuse.files="link")
expect_identical(as.array(readObject(tmp)), arr)
Expand Down

0 comments on commit 5cde6b7

Please sign in to comment.