Run gprofiler without root/sudo

[pcarella1]

Description

This PR adds support for running gprofiler without root/sudo as discussed in issue 905. There are several assumptions and components that I will mention here.

This PR requires a change in the granulate-utils repo that defines the run_in_ns_wrapper function found in PR 265 on that repo intel/granulate-utils#265.

Assumptions when running without root:

1. When running without root, the user must use --pids to select user owned processes only.
2. The user must direct the log and pids files to a user owned directory (e.g. with --log-file and --pid-file parameters).
3. The user must set certain system parameters such as kernel.perf_even_paranoid as needed to allow gprofiler to run.
4. Some of the corner cases that require fallback rw exec directories for POSSIBLE_AP_DIRS may not be resolved.

Components:

1. Replaced exit/error when is_root check fails in verify_precondiftions and replaced it with this message: "Not running as root, and therefore functionality is limited. Profile is limted to only processes owned by this user that are passed with --pids. Some additional configuration (e.g. perf_event_paranoid) may be configured to operate without root."
2. Created run_in_ns_wrapper function which bypasses the code to enter name spaces when not root (as we assume we're always in the correct namespace for the processes being profiled)
3. Added a parameter to the pgrep_maps function to ignore permissions errors. Each time a profiler calls this function, it will check if root and if not, will pass "True".
4. Redirected the default value of TEMPORARY_STORAGE_PATH to the resources directory.
5. Added mkdir_owned_user function which is used in main where the TEMPORARY_STORAGE_PATH creates gprofiler_tmp, so that it doesn't throw an error when we aren't root, but still ensures the directory is owned by the current user.

Potential issues:

1. Is there anything I should add/change in the message that is displayed when the is_root check in verify_preconditions fails? Also, is print() to stdout correct here?
2. Is it fine to redirect TEMPORARY_STORAGE_PATH to the resources directory even in the default case, or should I add a check to only do this when not root?
3. Do we need to resolve the fallback rw exec directories for POSSIBLE_AP_DIRS?
4. I've tested this on two systems and it works on both, but on one of them I receive this warning (though gprofiler completes and produces valid results). I discuss this more in the corresponding granulate-utils PR since that is the source of the error.

[2024-12-02 19:59:55,557] WARNING: gprofiler.profilers.java: Failed to enable proc_events listener for exited Java processes (this does not prevent Java profiling)
Traceback (most recent call last):
File "granulate_utils/linux/proc_events.py", line 222, in start
PermissionError: [Errno 1] Operation not permitted

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "gprofiler/profilers/java.py", line 1395, in start
proc_events.register_exit_callback(self._proc_exit_callback)
File "granulate_utils/linux/proc_events.py", line 272, in wrapper
File "granulate_utils/linux/ns.py", line 305, in run_in_ns_wrapper
File "granulate_utils/linux/ns.py", line 299, in run_in_ns_wrapper
File "granulate_utils/linux/proc_events.py", line 260, in _start_listener
File "granulate_utils/linux/proc_events.py", line 225, in start
PermissionError: This process doesn't have permissions to bind/connect to the process events connector

Related Issue

#905

Motivation and Context

Users of this gprofiler have requested this feature as some cloud instances do not have root access, but still want to profile user owned processes.

How Has This Been Tested?

I ran stress-ng and targeted gprofiler to the stress-ng pids without sudo. It successfully produced flamegraphs
Sample command line: ./build/x86_64/gprofiler --pids 1421864 -o results/ -d 15 --log-file ./gprofiler.log --pid-file ./gprofiler.pid

I have tested this on x86 using scripts/build_x86_64_executable.sh script. Centos 9 Stream w/ kernel 6.6

Also tested using sudo targeting specific pid(s) and system-wide, and it still works.

Was not able to run tests/test.sh as it required apt-get/debian environment.

Screenshots

Checklist:

The code is linted.

I have not updated the README.md doc here. Might need some guidance.

• I have read the CONTRIBUTING document.
• I have updated the relevant documentation.
• I have added tests for new logic.

[Jongy]

Is there anything I should add/change in the message that is displayed when the is_root check in verify_preconditions fails? Also, is print() to stdout correct here?

I prefer that the rootless mode will be opt-in, and not a default in case of non-root user (which allows misconfigurations to continue misusing the profiler).
So a flag like --rootless that enabled this behavior & checks that we're not root. And the verify_preconditions() function will suggest using --rootless.

Is it fine to redirect TEMPORARY_STORAGE_PATH to the resources directory even in the default case, or should I add a check to only do this when not root?

I left some comments on the PR about this topic.

Do we need to resolve the fallback rw exec directories for POSSIBLE_AP_DIRS?

Not sure I got you here.

I've tested this on two systems and it works on both, but on one of them I receive this warning (though gprofiler completes and produces valid results). I discuss this more in the corresponding granulate-utils PR since that is the source of the error.

Yeah, it's fine - as I commented on granulate-utils, proc_events are expected to fail in rootless.

[pcarella1]

I added a commit that should address all of your comments, but let me know if there is anything I should change. In particular with the new --rootless option and in mkdir_owned_root_wrapper.

Replacing him in this PR

[sobolron]

Hi!

• I've merged the granulate-utils PR, now let's move the ref of the submodule to something on the master branch and I'll approve.
• About documentation - I think we need to create a table that contains the available capabilities of gProfiler, and which are supported and which are not in rootless mode. As this entire table of capabilities doesn't exist yet, I think that it's not really in the scope of this PR, but I highly recommend creating it in a separate one so that feature will be easier to use.
• About testing - The current testing "framework" (scripts 😅 ) doesn't currently support rootless run properly, this will require doing changes to it, and I highly recommend doing this as well, but again - pretty much out of scope IMO.

[pcarella1]

• I've merged the granulate-utils PR, now let's move the ref of the submodule to something on the master branch and I'll approve.

I updated the granulate-utils reference to dd8aa29 in the latest commit

[pcarella1]

Added a commit to address the test_not_root checklist failure.