Templated files are NOT processed independently of each other

[paltherr]

Describe the bug

Any modification to the . dictionary done while processing a templated file is visible in the processing of further templated files. The processing of each templated file should start with an unmodified . dictionary.

To reproduce

File chezmoi/.chezmoitemplates/file.txt:

home={{ .chezmoi.homeDir }}
name={{ .filename }}.txt

File chezmoi/file1.txt.tmpl:

{{- template "file.txt" (merge . (dict "filename" "file1")) -}}

File chezmoi/file2.txt.tmpl:

{{- template "file.txt" (merge . (dict "filename" "file2")) -}}

$ chezmoi apply file1.txt file2.txt
$ tail -1 file1.txt
name=file1.txt
$ tail -1 file2.txt
name=file1.txt

Note that file2.txt contains file1.

$ chezmoi apply file2.txt  
$ tail -1 file2.txt      
name=file2.txt

Nowfile2.txt contains file2.

% chezmoi apply          
paltherr@maloja ~ % tail -1 file2.txt
name=file1.txt

And it's back to file1!?!

Expected behavior

The file file2.txt should always contain file2.

Output of chezmoi doctor

$ chezmoi doctor
RESULT    CHECK                MESSAGE
ok        version              v2.35.2, commit 44814515eb8034b4d153569db86e58e278148e05, built at 2023-07-19T08:02:28Z, built by Homebrew
warning   latest-version       v2.36.0
ok        os-arch              darwin/amd64
ok        uname                Darwin maloja 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:51:50 PDT 2023; root:xnu-8796.121.2~5/RELEASE_X86_64 x86_64
ok        go-version           go1.20.6 (gc)
ok        executable           /usr/local/bin/chezmoi
ok        upgrade-method       replace-executable
ok        config-file          ~/.config/chezmoi/chezmoi.toml, last modified 2023-07-28T21:23:59+02:00
ok        source-dir           ~/account-customization/chezhome is a directory
warning   suspicious-entries   ~/account-customization/chezhome/.chezmoitemplates/chezmoi.toml, ~/account-customization/chezhome/dot_config/chezmoi/private_chezmoi.toml.tmpl, ~/account-customization/chezhome/dot_config/chezroot/private_chezmoi.toml.tmpl, and ~/account-customization/chezhome/private_Library/private_Preferences/.chezmoiignore~
warning   working-tree         ~/account-customization is a git working tree (dirty)
ok        dest-dir             ~ is a directory
ok        umask                022
ok        cd-command           found /bin/zsh
ok        cd-args              /bin/zsh
info      diff-command         not set
ok        edit-command         found /usr/local/bin/emacs
ok        edit-args            emacs
ok        git-command          found /usr/local/bin/git, version 2.41.0
ok        merge-command        found /usr/bin/vimdiff
ok        shell-command        found /bin/zsh
ok        shell-args           /bin/zsh
info      age-command          age not found in $PATH
info      gpg-command          gpg not found in $PATH
info      pinentry-command     not set
info      1password-command    op not found in $PATH
info      bitwarden-command    bw not found in $PATH
info      dashlane-command     dcli not found in $PATH
info      gopass-command       gopass not found in $PATH
info      keepassxc-command    keepassxc-cli not found in $PATH
info      keepassxc-db         not set
info      keeper-command       keeper not found in $PATH
info      lastpass-command     lpass not found in $PATH
info      pass-command         pass not found in $PATH
info      passhole-command     ph not found in $PATH
info      rbw-command          rbw not found in $PATH
info      vault-command        vault not found in $PATH
info      vlt-command          vlt not found in $PATH
info      secret-command       not set

[paltherr]

Note that I found that I could avoid the problem by switching the order of the two merge arguments in the .tmpl files. However, it still looks wrong that modifying . in one .tmpl file may affect the processing of other .tmpl files.

[twpayne]

This is because you are using the merge template function. According to the documentation:

merge, mustMerge

Merge two or more dictionaries into one, giving precedence to the dest dictionary:

$newdict := merge $dest $source1 $source2

This is a deep merge operation but not a deep copy operation. Nested objects that are merged are the same instance on both dicts. If you want a deep copy along with the merge than use the deepCopy function along with merging. For example,

deepCopy $source | merge $dest

mustMerge will return an error in case of unsuccessful merge.

You need to use deepCopy.

[paltherr]

Sure, that avoids the problem but isn't it problematic that chezmoi generates different results for the same target depending on which files it's applied to? With my example, running chezmoi apply file1.txt file2.txt generates a file2.txt that contains file1 and running chezmoi apply file2.txt generates a file2.txt that contains file2. Shouldn't chezmoi always generate the same result for the same target? Couldn't it achieve that by processing each file with a preventive deepCopy of .?

Users could still shoot themselves in the foot, for example by calling two templates with . where the first template modifies . and thus impacts the second template. But at least chezmoi would then be self-consistent and always generate the same result for a given target.

[twpayne]

You're right. The template execution should be independent. Should be fixed with #3129. Thank you for spotting and reporting this :)