Revealing hidden processes

[halpomeranz]

I recently submitted a pull request to collect /proc/*/mounts to help detect when somebody is using a bind mount to hide processes (see https://dfir.ch/posts/slash-proc/). However, UAC could go farther and actively unmount and bind mounts found under /proc in order to reveal the hidden process information. Of course, this would actively change the state of the remote system and I wasn't sure if UAC wanted to go there.

Another idea I was considering was to implement something similar to chkproc from the chkrootkit distro (chkrootkit.org). Essentially write a small script that tries all /proc directories from {1..$(cat /proc/sys/kernel/pid_max)}. If it ends up in a directory that isn't visible under /proc, then you have a hidden process and probably a rootkit. Obviously, we would want to collect this process' info and probably call out the hidden process in some special way.

Thoughts on either of these ideas?

[tclahr]

I think the easiest way to identify them would be checking the mount command output for locations mounted in /proc:

$ sudo mount -o bind spoof /proc/19559

$ mount | grep -E "/proc/[0-9]+"
/dev/vda1 on /proc/19559 type ext4 (rw,relatime,errors=remount-ro)

Now the question is... how to get the info for 19559 without umouting /proc/19559 and "tampering" the system?

An option in uac.conf or in the command line (i.e.: --umount-proc-pid) to umount everything under /proc/[PID] could be a solution, but again, it would "tamper" the system.

Also, a helpful artifact would be comparing the ps output with /proc/[PIDs], the same way chkproc from chkrootkit does. This will not give us the details about the hidden process (such as cmdline, exe, etc...), but would give a quick insight about what is hidden.

[halpomeranz]

The bind mounting thing has been on my brain, so I blogged about it: https://righteousit.com/2024/07/24/hiding-linux-processes-with-bind-mounts/ As you can see, we ended up with similar regexes for detecting the bind mounts.

What if we started a new ".../artifacts/modifiers" directory in UAC for modules that can change the state of the system? We could create a new profile "ir_triage_with_mods" that would include these extra modules if people wanted to use them.

Your idea about comparing ps output with a listing of /proc doesn't work. If the directory is not visible under /proc then it won't show up in the ps output. The ps command gets all of its information by walking the /proc file system.

[tclahr]

I like the idea about the "modifiers" category.

[halpomeranz]

OK, let me work up some code and submit a pull request. Not sure when I will get a window to do this.

[tclahr]

I can try to write something and add you as a reviewer (I think).

[tclahr]

I was thinking about this... I think I will create a property that can be used in the yaml file to mark the artifact as a modifier. A property called "modifier".

Then, uac could provide a command line option to enable them during the execution. Something like --enable-mods.

This way we do not need to create different profiles for mods. We can add them to ir_triage and full and only run them when --enable-mods is provided.

version: 1.0
modifier: yes
artifacts:
  -
    description....
    ...
    ...

I would also add them to a dedicated artifacts/live_response/modifier directory (as proposed by you) for easier identification.

What do you think?