Building Custom eBPF Programs to Catch Production Anomalies Before They Escalate

Standard observability tools aggregate metrics, smoothing over micro-bursts and anomalous syscall sequences until they become full-blown outages. Dropping custom eBPF probes directly into the kernel execution path allows you to intercept these exact anomalies in real time. We are replacing generic user-space dashboards with targeted, kernel-level tripwires.

TL;DR: Relying on user-space metrics means you miss sub-millisecond production anomalies like erratic disk I/O or unauthorized syscalls. Compiling targeted eBPF tracepoints allows you to detect these issues instantly at the kernel level. This post demonstrates writing, compiling, and loading a custom anomaly-detection program using Clang 18.0 and bpftool 7.2 on Linux kernel 7.2.

What you’ll walk away with:

• A raw C program that tracks sudden spikes in openat syscalls.
• The exact clang command to compile BPF bytecode with BTF (BPF Type Format) debug info.
• A deployment pipeline using bpftool prog load to inject the probe.
• A troubleshooting checklist to verify event ingestion and prevent verifier rejections.

Why Do Standard Monitoring Tools Miss Micro-Burst Anomalies?

Standard monitoring tools sample counters at one-second or ten-second intervals, entirely missing transient kernel events that execute in microseconds. Because user-space agents rely on polling /proc or /sys, they physically cannot catch micro-burst anomalies like sudden disk I/O storms or rapid file-open failures before the data aggregates away. You must intercept the event exactly when it occurs.

This requires eBPF (Extended Berkeley Packet Filter), a technology that runs sandboxed, event-driven programs in a privileged kernel context without requiring kernel source code changes or loading heavy modules. eBPF executes your custom logic synchronously with the kernel function. By bypassing user-space polling mechanisms, eBPF programs operate with near-zero overhead.

Imagine the core payment API at Aicademy. A rogue background job begins opening thousands of file descriptors per second, crushing the node’s disk IO. Prometheus scrapes the node 15 seconds later and sees a moderate CPU bump. If you had The bpftool perf Command That Reveals Hidden Network Latency in Kubernetes configured alongside a custom disk I/O eBPF probe, you would catch the exact PID triggering the spike at millisecond one.

Migrate critical high-frequency anomaly detection from user-space polling to kernel-space event triggers.

How Do You Write a Custom eBPF Program to Detect Syscall Spikes?

You write a custom eBPF program by defining a C function that attaches to a specific kernel tracepoint and increments a shared BPF map counter whenever triggered. By hooking sys_enter_openat, your code intercepts every file open attempt on the system, logging abnormal rates directly in kernel space.

The C code below sets up a BPF hash map to track file open operations per process ID (PID). We use libbpf macros to define the map and tracepoint structure. The code safely handles concurrency using __sync_fetch_and_add because multiple CPUs execute this tracepoint simultaneously.

#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>

struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, 10240);
    __type(key, u32);
    __type(value, u64);
} pid_open_count SEC(".maps");

SEC("tracepoint/syscalls/sys_enter_openat")
int detect_openat_spike(struct trace_event_raw_sys_enter *ctx) {
    u32 pid = bpf_get_current_pid_tgid() >> 32;
    u64 *count, initial_val = 1;

    count = bpf_map_lookup_elem(&pid_open_count, &pid);
    if (!count) {
        bpf_map_update_elem(&pid_open_count, &pid, &initial_val, BPF_ANY);
    } else {
        __sync_fetch_and_add(count, 1);
    }
    return 0;
}

char _license[] SEC("license") = "GPL";

View the libbpf header requirements for this code

bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h

The eBPF program requires strictly typed kernel structures to pass verifier checks. Generating vmlinux.h directly from your host kernel ensures Clang resolves trace_event_raw_sys_enter accurately.

Writing raw C gives you absolute control over memory boundaries and map allocations. Relying on higher-level abstractions often obscures memory limit errors until the kernel verifier outright rejects the program.

Always generate vmlinux.h directly from your target kernel version to prevent struct offset mismatches.

What Is the Compilation and Loading Pipeline for eBPF?

The eBPF pipeline compiles C source code into a constrained ELF object file using Clang, which a user-space loader then submits to the kernel. The kernel’s strict verifier checks the bytecode for safety before the JIT compiler translates it into native machine code and attaches it to the hook.

This architecture guarantees that custom anomaly detection cannot crash the host system or cause kernel panics. The verifier in Linux kernel 7.2 analyzes all possible execution paths, immediately rejecting programs containing unbounded loops, uninitialized variables, or invalid memory access. The pipeline enforces stability by design.

flowchart TD
    A["C Source Code"] -->|"clang 18.0"| B["ELF BPF Object"]
    B -->|"bpftool prog load"| C{"Kernel Verifier"}
    C -->|"Safe"| D["JIT Compiler"]
    C -->|"Unsafe"| E["Rejected (Error)"]
    D --> F["Kernel Hook (Tracepoint)"]

To compile the C code we just wrote, enforce the exact target architecture and BTF generation flags. Using Clang 18.0 is strictly required here to ensure modern CO-RE (Compile Once – Run Everywhere) relocations compile without stripping types.

clang -target bpf -D__TARGET_ARCH_x86 -O2 -g -c detect_openat.c -o detect_openat.o

Next, load the resulting object into the kernel and pin the map to the BPF virtual filesystem using bpftool 7.2. Pinning prevents the program from detaching the moment the loader process exits.

sudo bpftool prog load detect_openat.o /sys/fs/bpf/detect_openat type tracepoint

For teams architecting cluster-wide security tools, understanding this manual lifecycle is a prerequisite before automating deployments. You can see how this scales in eBPF for the Impatient: Architecting Runtime Security at the Kernel Edge.

Compile with the -g flag to embed BTF debug information, which is mandatory for CO-RE portability across different kernel versions.

Which Toolchain Should You Use for eBPF Deployment?

You should use libbpf with CO-RE for all modern eBPF deployment pipelines rather than older wrapper frameworks. Libbpf produces standalone, portable binaries that do not require installing bulky compiler toolchains or kernel headers on every single production node operating in your fleet.

Historically, practitioners relied on BCC (BPF Compiler Collection) and Python scripts to run custom tools. BCC requires compiling the C code at runtime on the target machine. This approach wastes CPU cycles on production nodes and introduces significant security risks by demanding clang on application servers.

Framework	Portability	Target Node Requirements	Best For
libbpf + CO-RE	High (Compile Once)	None (pre-compiled binary)	Winner: Production deployments
BCC (Python/C)	Low (Compile on target)	Clang, LLVM, Kernel Headers	Local debugging and prototyping
Cilium / Tetragon	Abstracted	Agent installation	Out-of-the-box Kubernetes security

Default to libbpf unless you are explicitly deploying a third-party managed agent. For custom anomaly detection tools tailored to specific application logic, libbpf is the industry standard. Linux kernel 7.2 increased the verifier complexity limit to 1 million instructions per program, allowing libbpf binaries to execute highly sophisticated inspection logic.

Use libbpf and CO-RE to keep production nodes stripped of compiler dependencies.

How Do You Verify an eBPF Probe is Actually Processing Events?

You verify probe execution by checking the kernel’s registered programs and map statistics using bpftool, then confirming hit counts on the specific BPF map. If your anomaly detector is loaded but silent, you must inspect the map elements to ensure the tracepoint is successfully triggering and passing verifier constraints.

Run the following command to list all currently loaded BPF programs and locate yours. We are executing this via bpftool 7.2 to read the JIT allocation statistics.

sudo bpftool prog show

245: tracepoint  name detect_openat_s  tag a1b2c3d4e5f6g7h8  gpl
    loaded_at 2023-10-25T10:14:32+0000  uid 0
    xlated 152B  jited 104B  memlock 4096B  map_ids 89
    btf_id 112

If you see your program, retrieve the map contents using the map_ids value (89 in this example). This will dump the hex values of the PIDs and their corresponding openat event counts. We regularly see teams unmask hidden kernel backdoors by dumping map contents left behind by hidden rootkits.

To fix a map dropping events under heavy load, adjust the max_entries configuration in your C code before compilation. If the hash map fills up, bpf_map_update_elem will silently fail.

  struct {
      __uint(type, BPF_MAP_TYPE_HASH);
-     __uint(max_entries, 1024);
+     __uint(max_entries, 102400);
      __type(key, u32);
      __type(value, u64);
  } pid_open_count SEC(".maps");

Use this validation checklist before assuming your probe is fully functional in a staging environment:

• Verify the target kernel supports the tracepoint by grepping /sys/kernel/tracing/available_events.
• Confirm bpftool prog load returned a zero exit status.
• Run bpftool map dump id <ID> and verify the raw byte keys are actively populating.
• Check dmesg logs for verifier warnings, map allocation limits, or JIT compiler failures.

Validate map contents using bpftool map dump rather than blindly trusting the initial load command’s success status.

Bottom Line

Custom eBPF programs eliminate the blind spots created by user-space polling intervals, giving you absolute visibility into kernel execution. By writing raw C tracepoints and compiling them with Clang 18.0, you intercept production anomalies exactly when they occur. Stop waiting for Prometheus to scrape a spiked CPU metric and start dropping kernel-level tripwires today.

In the next part of this series, we construct an automated CI/CD pipeline for testing CO-RE payloads against multiple kernel versions.

FAQ

Why does the compiler throw an error about missing vmlinux.h?

The vmlinux.h header contains all internal kernel structures required for eBPF to read memory offsets correctly. You must generate it directly from your host system using bpftool btf dump file /sys/kernel/btf/vmlinux format c > vmlinux.h.

What is the difference between BCC and libbpf for eBPF deployments?

BCC requires a full compiler toolchain and Python installed on the target host to compile C code at runtime. Libbpf utilizes CO-RE to compile a standalone binary offline, making it strictly superior for secure production deployments.

How do I detach an eBPF program that I loaded with bpftool?

eBPF programs loaded and pinned via bpftool remain active until the pinned file is removed. Run sudo rm /sys/fs/bpf/detect_openat to unpin it, which allows the kernel to safely detach and garbage-collect the program.

Can eBPF programs crash my production server?

No. The Linux kernel verifier strictly analyzes the BPF bytecode before it is loaded, completely rejecting any program containing unbounded loops or invalid memory access. This isolation ensures your code cannot trigger a kernel panic.

Part of the series: advanced-ebpf-ops

1. The `bpftool perf` Command That Reveals Hidden Network Latency in Kubernetes
2. Building Custom eBPF Programs to Catch Production Anomalies Before They Escalate (you are here)
3. The 'Invisible Backdoor' Incident: How eBPF Caught a Kernel-Level Supply Chain Attack
4. Is Your eBPF Deployment Actually Secure? A 5-Minute Audit for Production Environments

Further Reading

• https://github.com/iovisor/bcc
• https://cilium.io/

---

🚀 Ready to get hands-on? Spin up an interactive AI or Kubernetes Sandbox at Aicademy Labs for free.