The InvokeAI User Interface is very nice and slick, but sometimes you want to access your outputs from the command-line or from another tool, and the trouble is that InvokeAI just throws all its outputs in one big directory. There is structure there, and indeed InvokeAI presents the pictures organized into “boards” which you create, and so on, but that structure is held in InvokeAI’s database and not reflected in the actual directory structure.
InvokeFuse is a Filesystem in Userspace (FUSE) script, written in Python, which attempts to give you access to your big outputs/images directory in a more organized form, broken down by “board” and also by what model or prompt you used to create the image, or the date you made it on.
It sort of accreted features here and there, and doesn’t necessarily confirm that you’re accessing things the way it expects, so it is fairly easy to wind up trying to access a path that gives you unexpected or erroneous results, but if you keep within its limits you should get expected results. In no case should InvokeFuse ever actually mess up your InvokeAI database; it does not write anything there.
Please read these docs. At least the Directory Structure section. They aren’t that long, and things will be all confusing and won’t work as you expect until you update your expectations with knowledge.
It’s probably simplest just to run it from your InvokeAI installation directory:
$ python /path/to/invokefuse.py -o dbfile="$PWD"/databases/invokeai.db ./mnt
(You should create a mnt/ subdirectory there to mount it.) Basically, you need to give it a path to the invokeai.db file in the dbfile option. Make sure it is an absolute path. FUSE does not do so well with relative paths (but InvokeFuse will convert the ./mnt mountpoint to absolute form.) InvokeFuse presumes that the images are present where InvokeAI generally stores them relatively to the database, that is, in ../outputs/images relative to the directory where the database is.
See also the Options section for how to proceed if your files are someplace else, and for other important behavior-changing options.
On Linux, use the fusermount -u command to unmount the directory. Mac and WSL I think use the normal umount command (note spelling!) You won’t be able to unmount the directory if any process is still using it. This includes having files open there or even having a directory in it as its current working directory. The fuser -m command (may not exist on WSL) can tell you which processes are still using the mounted filesystem.
The basic directory structure as presented by InvokeFuse works like this (I started to draw a diagram but it just got ridiculous.) Note that the structure is pretty rigid, and you are expected not to skip levels. To see all the images in a board, for example, you need to look at something like
mnt/<boardname>/models/ALL/ALL/
See also below, though, about Unlisted Entries.
The top level, directly under the mount point, consists of directories named for each of the “boards” you have created in InvokeAI. There is also a special directory called UNSORTED/ for the “Uncategorized” section, where InvokeAI puts images that you haven’t assigned to any of the boards. (See the Caveats section for warnings about board names.)
The second level is fixed, and always contains the exact same two “subdirectories.” I was sort of hesitating between how to order things and wound up doing both. The basic idea in my mind when I was starting out was that I wanted to access images by the (positive) prompt used and by the model used to generate them, but I wasn’t sure which one of those was primary. So I ended up doing both, and you can choose to have paths that look like
<boardname>/models/<modelname>/<prompt>/003b9609-ce8d-4671-b59c-052859349a01.png
or ones that look like
<boardname>/prompts/<prompt>/<modelname>/003b9609-ce8d-4671-b59c-052859349a01.png
So an image in board myArt generated with juggernaut with a prompt “happy people” appears both as
myArt/models/juggernaut/'happy people-FS+YQH1pnb5W'/003b9609-ce8d-4671-b59c-052859349a01.png
as well as
myArt/prompts/'happy people-FS+YQH1pnb5W'/juggernaut/003b9609-ce8d-4671-b59c-052859349a01.png
At this level, one below choosing “models” or “prompts”, you have subdirectories named for each model used to generate images in the given board, or subdirectories named for each prompt used to generate images in the given board (depending on which you chose in the parent directory.)
In the “models” case, it is pretty straightforward, as models usually have short and “well-behaved” names. There will be a directory called stable-diffusion-v1-5/ and one called stable-diffusion-xl-base-1-0/ and one called juggernaut/ and one called realcartoonXL-v6/ and so on, for all the models that you’ve used to generate images in that board (so some of the examples I just gave might not be present.) There will also be a special subdirectory named NO MODEL/ which holds all images for which there is no model in the metadata (this happens for images generated by image-to-image processing, workflows, etc.) The NO MODEL/ subdirectory always appears, whether or not there are actually any images in it.
In the “prompts” case, there is similarly a NO PROMPT/ subdirectory for images which do not have a prompt in their metadata (this is usually caused by the same things that cause there not to be a model; most of the time, in fact, it seems that the set of “no model” images is the same as the set of “no prompt” images.)
But using prompts as directory names raises some awkwardness, because prompts can be very long and can contain funny characters and that can make them hard to read and type. So I just truncate the prompt and hope that the start of the prompt is meaningful to you. Actually, first I preprocess the prompt: I strip out anything that isn’t an ASCII letter, number, or underscore ([A-Za-z0-9_]), replacing any string of such characters with a single space, and then I strip off leading and trailing spaces. Then I use the first 20 characters of the resulting string, hoping that is meaningful enough to you. Because that preprocessing could easily result in distinct prompts boiling down to the same prefix, I also add a disambiguating suffix (after a - character), made of a 9-byte hash of the whole prompt (before processing) encoded into ASCII. I use a short hash so that the suffix isn’t too long, and because I figure even with a “small” hash space (272), it’s very unlikely that there will be collisions and it isn’t like we’re protecting valuable secrets that need foolproof cryptographic hashing. That’s why the examples above showed prompts with that string of nonsense characters at the end: those were the hash.
This level is (almost) exactly the same as the one above it, except that it does the other one of the two initial choices. So if you’re in the models/ branch and the level above this one was directories of model-names, this will be a directory of (truncated and hashed) prompts that were used in this board for that model. And if you’re in the prompts/ branch and the level above this one was directories of prompts, this level will be directories of model-names that were used for the given board with the given model. In the same way, there will be a NO PROMPT/ or NO MODEL/ subdirectory here.
There is one small difference, though. Only in the case where you’re in the prompts/ branch and so the parent directory chose the prompt, in its abbreviated form, there will also be a file here called PROMPT.TXT which contains the actual text of the prompt, unprocessed. This way you can check if you’re in the right place, in case the preprocessing obscured the distinctions between prompts. This is a normal file, you can read it with all normal tools.
Finally, this is the bottom layer. At this layer reside all the images you created in the given board, with the given model, with the given prompt. The images exist here as symbolic links to the actual images in the outputs/images/ directory of InvokeAI, to the extent that matters, but for most intents and purposes they’ll behave as the images, and you can display them etc.
If the PROMPT.TXT file was not in the parent of this directory (i.e., you were in models/ subtree and the parent of this directory was the abbreviated prompt), it will be down in this one.
Here’s where things get a little strange, if you’re used to navigating a normal filesystem. There are files and directories that you can access which are not listed when you do the listing of a directory. They aren’t hidden, just when you ask what’s in a directory, they simply aren’t there… and yet if you know their names you can access them anyway and things will happen. This might limit their usability in disk-browing apps, unless you can type in the name directly.
Down at the bottom level of the directory tree, where the symlinks to images reside, each image also has a not-shown “.META” file, which contains the image’s InvokeAI metadata (that is, the metadata in the InvokeAI database, which is distinct from any metadata that may be stored in the image file itself). The filename is the same as the image’s filename, except with .META (all caps) appended. So image art/models/juggernaut/folks-RT85jDq9YdSl/003b9609-ce8d-4671-b59c-052859349a01.png has an invisible companion file art/models/juggernaut/folks-RT85jDq9YdSl/003b9609-ce8d-4671-b59c-052859349a01.png.META which you can read to see its metadata (in JSON format).
The “bottom” of the tree structure described above isn’t really the bottom. There’s an invisible directory called dates/ down there which you can list. So you can
$ ls mnt/art/models/juggernaut/folks-RT85jDq9YdSl/dates/
and you’ll see a bunch of subdirectories with names like 2024-02-29/, for all the dates on which you generated images in the given board with the given model and prompt. And in each of those subdirectories you’ll find the symbolic links to the individual files (and their unlisted .META) files, as you might expect.
If you want to browse by date but not limit yourself to a particular board/model/prompt combination, you can use the ALL wildcard, below, to avoid limiting yourself at one or more of these levels.
This feature perhaps has a little more potential for blowing up than some others. It’s meant to help with the problem of the truncated prompts not including the stuff you want. At the <board>/models/<model>/ level, instead of going into one of the truncated prompt subdirs (or NO PROMPT/), you can list a directory called LIKE/%cute%/ or something like that. This will give you a listing of prompt subdirs for all prompts that are “like” the name you gave for the sub-directory of LIKE (i.e., the %cute% part), using the SQLite “LIKE” operator, thus using % and _ as wildcards, etc.
Unlike the usual prompt subdirectories, when LIKE/<pattern>/ is used, the names of the subdirectories are not truncated and hashed, but are presented straight-up, as they are, full length. So the directory names here can potentially be very long, or contain all sorts of punctuation or other characters. See below under Caveats for a potential bug that might come about.
Inside these subdirectories you’ll find the usual PROMPT.TXT file and the symbolic links to the relevant images, etc.
Currently, there does not seem to be a way to combine LIKE/ with dates/.
Using LIKE/ without giving a pattern after it essentially ignores the LIKE/ and acts like you’re just getting the usual truncated prompts.
See also under Caveats: using LIKE/* in other places in the directory tree may cause weird things to come about.
Any place in a path where you are normally expected to have some value that limits to a particular board or model or prompt, you can substitute the special word ALL (all-caps) to sort of “skip” that level and take all of the options. So ALL/ at the top level means all of the boards, ALL/models/ALL/ contains subdirectories for all the prompts you used for any board and any model, and so on. (Note that the second level, “models” or “prompts”, cannot be skipped with ALL/, because it would make no sense as the two trees are redundant to one another.) It is important to use ALL/ to skip levels (and not try just omitting them to put dates/ or LIKE/ someplace) because leaving it out can confuse InvokeFuse and you can wind up with image-names being listed as directories and so on. See under Caveats.
As with any mount type command, options for InvokeFuse go in the -o option:
python3 ./invokefuse.py -o dbfile="$PWD"/databases/invokeai.db,foreground,imagesdir=/otherdisk/images,real_files ./mnt
(not like most applications where options are given as --option or something). The options follow the -o option (after a space), and are separated by commas but not spaces, otherwise the shell will consider it a separate command-line argument and InvokeFuse will probably try to interpret it as the mount-point. Any spaces in the arguments (e.g., in pathnames) must be quoted, hence the use of double-quotes around $PWD in the example above. (There isn’t a good way to handle pathnames that contain commas; just avoid those.) Options are either standalone or take a value with an = sign.
dbfile=PATH- The one mandatory option. Provide the path to the
invokeai.dbfile that you are using. This must be an absolute path (not a relative path). Make sure you quote any spaces that might be in the pathname. foreground- Standalone FUSE option. If provided, the FUSE system runs in the foreground and does not fork off immediately to become a background process. This can be useful for debugging, since normal FUSE processes can’t write to a visible standard output or error stream, so debug statements aren’t helpful, and even exceptions don’t show on the console. But when running in “foreground” mode (even if you use
&on the command to run it in the background like you can with any command), at least you can see the exceptions and any print statements, etc. allow_other- Standalone FUSE option. Normally, a FUSE-mounted filesystem is accessible only by the user who mounted it (not even the root user can access it.) The
allow_otheroption allows other users to see and read the filesystem. This may be important especially if you are browsing over a network connection, etc. For security reasons, FUSE can only do this if it is explicitly allowed in the system FUSE configuration file,/etc/fuse.conf. You’ll need to make sure a lineusers_allow_otheris in that file. allow_root- Standalone FUSE option. This is like
allow_other, except it only allows the root (administrator) user access to the mounted file-system. rootdir=PATH- The directory that is the root of your InvokeAI installation. If not provided, it is presumed to be one level up from the directory where the
dbfileis. That is, if your dbfile is/mnt/d/invoke/databases/invokeai.db, then rootdir is/mnt/d/invoke, which is what you would expect. If you specify this, it should be an absolute path. imagesdir=PATH- The output directory where all your images are located. If not provided, it is presumed to be
<rootdir>/outputs/images, which is how InvokeAI organizes things normally. This is useful if you store your images on a different disk or something for space purposes. As with the other paths, this should be an absolute path. real_files- Standalone option. Operate in “real files” mode. Normally, the actual images are presented as symbolic links to the actual image files. But it seems that Windows systems don’t process symbolic links well, so the
real_filesoption makes InvokeFuse try to present the images as actual files and handle reading from them, etc, so as to make it usable with Windows. This may cause some small performance degradation, and there might also be other issues with it that we don’t know about yet.
This project was sort of thrown together, and there is a lot of error-checking and due diligence with inputs that is not being done. The user expected to use things correctly and not to provide opportunities for mishaps.
There are several exceptional names (and a format) which InvokeFuse treats specially, and I’m counting on the user not to have boards or models with those names. They are all defined as globals in the invokefuse.py file, so if you really need to you can just change their definition once.
UNSORTED- This is the reserved name for the “Uncategorized” board, of files not in a board otherwise. Don’t use this name (all-caps) for another board.
NO MODEL,NO PROMPT- These are reserved names for directories containing images with no listed model or prompt, respectively. Using
NO PROMPTas a prompt could probably only cause problems if it wound up being made into a directory underLIKE/*, and that is pretty unlikely. It’s also unlikely any model will be namedNO MODEL, but if it ever happens, be careful! Even so, be careful not to use these as board names. LIKE- This is reserved for the =LIKE/*= construction, in the prompts subdirectory. It’s unlikely to conflict with anything else, since even if you used
LIKEas a prompt, it would be replaced by its hash-appended replacement (LIKE-DpmyJsCLx1DM) in the directory, so it should not cause problems, except maybe if it occurs as a result of anotherLIKE/*construction. And similarly, don’t use this as a board name (you can use it if you don’t use all-caps.) ALL- This is the reserved word for “all” models or boards or prompts. Naming a board “ALL” (all-caps) would make it impossible to select just that board, since it would be confused for selecting all of them.
dates- This name is reserved for the =dates/= subdirectory, and using it as a board-name could be problematic. Note that unlike the other reserved names, this is lowercase.
/^\d{4}-\d{2}-\d{2}$/- This is what date subdirectories look like: four digits, a hyphen, two digits, a hyphen, and two more digits. If you use board-names that look like this, it will cause problems. (Because of the sloppy way in which things are done, the fact that it’s way up at the top level and not where a date would be expected does not help, and any of these “special forms” can be a problem as a board name.) This is actually a plausible way you might already be naming your boards; maybe this needs to be changed to have a prefix or suffix like
_or something. We’d have to change it both in the global variable and where the dates subdirs are generated. - The
/Character - Probably the most important remaining restriction is the use of the
/character. It’s the only character (apart from the NUL character) which Linux forbids in a filename or directory name. A board or a model with a/in the name will break things, and is not currently checked for.Ordinarily, prompts are stripped of bad characters, but if you use
LIKE/*and one of the prompts selected has a/, that could cause problems too. Right now, the/character is replaced by a space in all prompts, everywhere, so as to avoid the problem. Especially if you’re using auto-captioning, you might well have slashes in the prompts, and that causes I/O errors when reading the directory. This opens the door to a different bug, namely that you could have two distinct prompts which differ only in slash versus space, which InvokeFuse would treat as identical, and you’d never see the slash even in thePROMPT.TXTfile (though the images generated thereby should still be there.) This seems to be a pretty remote possibility, so for now choosing to live with that bug, as being better than the limitations caused by leaving the slash in place.
There are some other globally-defined fixed strings, but I don’t think they would cause problems even if you use the names elsewhere:
models,prompts- These are the fixed names for the second level, but since they only matter at the second level, there should be no issues with them even if you name a board like that, etc.
.META- This is recognized as the suffix for metadata files, but since that only matters when you’re actually trying to read a file, it shouldn’t be an issue elsewhere, where your names generally become directories.
The structure of a path to an image is <board>/models/<model>/<prompt>/<image.png> or <board>/prompts/<prompt>/<model>/<image.png>. The special dates/ subdir belongs at the bottom of either of those two structures, in place of the image-file, and it will have subdirs which will have images in them. LIKE/* should only be used as <board>/models/<model>/LIKE/<like_expr> (and probably not in the prompts/ subtree.) Metadata files should be accessed with the same path as an image, plus the .META suffix. PROMPT.TXT files should be accessed in the places explained above. Trying to put special things like dates/ anyplace else is not necessarily checked for or handled correctly and may result in nasal demons.
At the moment, the dating used by the dates/ subdirectory is not timezone-aware, so the images which it says were created on 2024-01-05 are those which were created then in UTC, not local time. So some of the ones you thought you made on that day might show up in the previous or next day’s directory, depending on your timezone.
InvokeFuse was developed on a Linux system, using FUSE capabilities which Windows doesn’t really support natively, so you might have some issues with using it. I have heard that it can be used with the WSL environment, which essentially provides you with a Linux-like environment in Windows. You’ll need to sudo apt install fuse in your WSL environment to make sure you have FUSE support installed. When the docs here call for an “absolute path”, you probably are going to need to use the WSL-style path (/mnt/d/invokeai/databases/invokeai.db or something). To access the filesystem from outside WSL, you’ll need to make sure that allow_others is specified, and that it’s permitted in your /etc/fuse.conf file, so that external tools can see it.