🚀 Feature Request: Support for stacks in nested directories

[calrock27]

⚠️ Please verify that this feature request has NOT been suggested before.

• I checked and didn't find similar feature request

🏷️ Feature Request Type

UI Feature

🔖 Feature description

As a user, I would like to be able to manage stacks that are nested inside other folders visible to Dockge... I currently organize a few of my compose files in sub folders of their use case. I'll provide an example below.

My existing folder structure -
/opt/stacks/game_services
->/satisfactory/docker-compose.yml
->/minecraft/docker-compose.yml

In it's current state, Dockge sees my /game_services directory as empty because it does not contain a compose file.

✔️ Solution

When clicking on the left hand side navigation for a folder stack, it would display a tree of directories and allow me to drill down to the compose file I'm looking to manage.

If a compose file does exist in the directory, it should display on the right hand side (current functionality) until I click into a sub directory, in which case the right hand side would then display the compose file from the subdirectory.

❓ Alternatives

As an alternative solution, I've considered consoplidating all of my compose files from the /game_services directory into a single compose file in the /game_services directory. This ends up being a bit messy though as I have a few other dependent services with each game server.

📝 Additional Context

No response

[Keyruu]

Please implement this. I have 20 stacks I want to manage and if I don't put those in a nested folder it well get messy really quick.

[s4msy]

On the other hand, sorting all compose files to a reliable order into a stack folder brought some clearness in my mess. Did this yesterday, and I am really exited about this project.

But I would agree, allowing one parent folder should be enough and dockge goes searching its way down though all subfolders for compose files is an enhancement.

[BluDood]

This would be great, either something like a configurable value for nesting limit, or just being able to specify multiple directories for compose files

[DaftBrit]

Plus one on this, I've got a few third party github repos that nest their compose files and dockge is fantastic for managing stacks. As an alternative to having dockge trawl subfolders is it possible to have symlinks work? We could then leave the git structure alone and just create a symlink in the stacks folder with the name for the stack.

[aliuq]

I really hope to be able to identify nested directories. Currently there are 70+ under my stacks

[gizmocuz]

I'm using folders to keep everything clean. One folder for each project. A project can have multiple folders with their docker containers.
Would be great if this was supported!

[ChrisonSimtian]

bump, also waiting for this :-)

[mkoo21]

I am going to piggyback on this. The docker compose api can already tell you the path to a project's config file. I see no reason for dockge to be searching the stacks directory, or for it to only check the first level.

[gizmocuz]

@mkoo21 , not that I am probably be able to make a PR for this, but could you point me/us to the documentation of this?
Is there a 'docker container ls' that displays the path to the configuration file?

At the moment the stacks are searched with the following call

let filenameList = await fsAsync.readdir(stacksDir);

There is of course the option to scan recursively, but, depending on some users data folders that could contain lots of files, this could take significantly longer than just scanning the root folders

[aliuq]

docker compose has version requirements, can use docker-compose ls

[gizmocuz]

docker compose has version requirements, can use docker-compose ls

I don't think the 'docker-compose' command exists anymore, that is for the old docker compose (v1).

You really should update:

https://docs.docker.com/compose/releases/migrate

[aliuq]

I don't think the 'docker-compose' command exists anymore, that is for the old docker compose (v1).

You really should update:

https://docs.docker.com/compose/releases/migrate

You are right, but the Docker version is always fixed for some NAS devices, e.g. Synology container docker with the latest version is 24.0.2

[mkoo21]

@gizmocuz There appear to be several places where dockge retrieves the list of active stacks

There is the part you pointed out, which just searches the top level of the configured stacks directory. And there are several other places where it does something like

        let res = await childProcessAsync.spawn("docker", [ "compose", "ls", "--format", "{{.Name}}" ], {
            encoding: "utf-8",
        });

Which is the smarter way to do it (also how dockge is able to show your other stacks and then say they're not "managed by dockge")

I'm not sure I understand what the difference is, much of the stacks.ts logic (that manages things like "managed by dockge") seems to be written around the idea that "managed by dockge" is defined by being exactly one level done in the stacks directory. And when I tried some very naïve modifications I broke much of that internal logic.

I can try to put up a draft this week but I am not familiar with this codebase and it doesn't look like a trivial refactor

[mkoo21]

There are some other considerations, too, like names not being unique (I think docker compose has its own philosophy for this though) or projects having more than one config file. Or some reason the authors decided to do it this way. So it is worth discussing.

[gizmocuz]

One thing that probably does not work with the 'docker compose ls' command is that it does not find your docker projects that are not running.
Yes, it displays docker containers that are not running, but when you never started the container before, it will not be detected.
A folder scan will 'detect' those.

@mkoo21 , thanks for the great looking PR!
As you are using a map, maybe you could also scan the folders like before, and if not found with the 'docker compose ls' command, add these as well?
Or maybe scan the folders to a maximum depth.
I have seen a PR here that also does this... found it, PR 670 (GitOps with multiple stacks)

[mkoo21]

Yes, I realized this when I was reviewing my PR. It seems to me like this "discovery" feature would mostly be useful for novices, while being able to nest arbitrarily (or just let docker compose be the source of truth) is really useful for people with many projects. But I will put a folder scan back for backwards-compatibility's sake. The current master version does both anyways.

The main issue is the fact that dockge assumes the name of a project is the name of the directory containing its config file. Which is only true most of the time, and becomes an issue when you nest (e.g. /opt/stacks/project and /opt/stacks/services/project would both be given the name "project"). I'm thinking I will just use the full path as the temporary name for "discovered' projects, but this isn't perfect either (there are a bunch of finicky things here, like you can also have multiple config files for a project or multiple projects in a folder, which is why I like letting docker compose be the main source of truth and having the folder scan just be a "nice to have"). I will think about it some more until I have a chance to write it.

[mkoo21]

PR for this issue is up. It should resolve project names better than the current version, and it should be able to "discover" projects in the stacks directory that have a config file with an "accepted name" that have never been run through the docker compose CLI.

I think this is now 100% backwards compatible. But let me know you notice anything working on the main branch and not in this one.