How a Deep Dive into AI Networking Made Me Re-think Network Security

Security Sonar

You can’t secure or optimize LLMs, deep learning pipelines, or AI agents if you don’t understand the fabric they run on.

For the past several months I’ve been studying NVIDIA’s AI factory architecture from a platform security lens. That meant going deep down the stack into AI networking to understand how modern AI workloads are actually secured on the wire.

Running your own AI factory — custom, self-hosted, or fine-tuned proprietary models — is the recommended path when you need maximum data sovereignty, deep customization, or long-term cost efficiency at high volume. So as a platform security engineer, there’s a good chance you’ll eventually have to secure AI factory infrastructure. This post is about one specific way that infrastructure breaks the assumptions network security has relied on for two decades, using Network Security Monitoring (NSM) as the worked example.

What NSM is, and why it matters

The SANS Institute defines NSM as “a continuous process to collect and analyze indications of threats for the purpose of responding.”

Its core advantage is that it’s threat-centric rather than vulnerability-centric. Vulnerability management tries to patch the holes in the ship. NSM is the sonar listening for the torpedoes already in the water.

  • Assumption of compromise. NSM operates on an assume-breach mentality. Preventive controls — firewalls, EDR, access lists — will eventually fail or be bypassed, and the monitoring has to be there when they do.
  • Operational visibility. By focusing on the wire, NSM provides an objective record that host logs can miss. It captures a threat actor moving laterally whether or not they’ve cleared the event logs on a compromised host.
  • Continuous resilience. Unlike a point-in-time audit or scan, NSM is persistent. Session data and transaction logs give you the historical context needed for deep forensics and for scoping the blast radius of an incident.
  • High-fidelity detection. Run by analysts using tools like Zeek or Cisco Secure Network Analytics, NSM turns raw traffic into intelligence — surfacing low-and-slow exfiltration and living-off-the-land techniques that signature-based alerting tends to miss.

Modern NSM has already evolved

SANS supplies the foundational definition — Zeek for traffic analysis, Suricata for IDS/IPS — but enterprise NSM has moved beyond connection logs and packet captures.

Cisco Secure Network Analytics (formerly Stealthwatch) is a good example of the agentless approach. Instead of relying on TAPs, it ingests NetFlow and telemetry from existing switches, routers, and firewalls to build a digital baseline of normal, then uses behavioral modeling to flag ransomware or insider activity in real time without decrypting a single packet. Organizations spend heavily on this capability because detecting lateral movement and anomalous behavior is often the only thing between a contained incident and a full breach.

Why AI networking is different

Traditional security architecture leans on the operating system kernel as its primary witness. Whether it’s an EDR agent using kernel-level hooks to monitor system calls, or an NSM sensor like Zeek, the assumption is the same: nothing happens on the system without the kernel knowing about it. Even telemetry-driven platforms like Secure Network Analytics depend on this — the NetFlow they ingest is generated by the kernel’s networking stack.

AI networking breaks that assumption on purpose.

The paradigm shift: kernel bypass

To hit the ultra-low latency that GPU clusters need, AI fabrics use RDMA (Remote Direct Memory Access), which intentionally bypasses the CPU. Data is offloaded and moved directly between network hardware and GPU memory, never passing through the CPU or the Linux kernel.

Because the kernel never touches these packets:

  • Zeek has nothing to analyze — the kernel never hands it a packet copy.
  • EDR has nothing to log — no system call was ever made.
  • NetFlow-based analytics lose visibility — traditional telemetry points don’t see hardware-to-hardware communication.

It’s like monitoring a highway from a toll booth and then realizing the most important traffic is moving through a private high-speed tunnel built directly beneath the road — invisible and unreachable from your checkpoints.

Three disruptions for network security

  1. The kernel-bypass blind spot. RDMA moves data straight from the wire into GPU memory. If your detection strategy depends on host-based agents or kernel-level inspection, you are blind to the most critical data flows in the environment.
  2. From best-effort to lossless fabrics. Standard Ethernet is best-effort — drop a packet, TCP retransmits. An AI training or inference cluster running InfiniBand or RoCE (RDMA over Converged Ethernet) has to be lossless. A microsecond of jitter or one dropped packet can create tail latency that desynchronizes thousands of GPUs and stalls the whole job. You cannot drop a heavy, latent security appliance inline and hope it holds.
  3. The DPU as the new choke point. Since the fabric can’t be slowed down, security has to move into the hardware. Offloading encryption, firewalling, and telemetry to a DPU (Data Processing Unit) such as NVIDIA BlueField keeps line-rate performance while restoring visibility. The DPU becomes the sensor, sitting at the edge of the memory-to-memory corridor — the same architectural pattern the major security vendors are shipping to AI factory customers.

Where network security goes next

The high-speed RDMA and RoCE corridors have routed around tools like Zeek. Reclaiming visibility without melting CPUs means building security into the infrastructure rather than layering it on top. The pieces to watch:

  • DPUs as the new source of truth — hardware acting as a distributed sensor that sees, and can act on, what the OS cannot.
  • DOCA — NVIDIA’s framework for programming security logic directly onto the DPU.
  • NVIDIA Morpheus — a GPU- and DPU-accelerated framework for analyzing fabric telemetry at a scale traditional NSM tooling can’t reach.

As the World Economic Forum’s Cybersecurity Outlook has noted, AI introduces risk at roughly the same rate it introduces efficiency. Protecting the progress means a proactive posture that operates at the speed of the fabric — not one bolted on at a toll booth the traffic no longer passes.

I’m NVIDIA-Certified in AI Networking (NCP-AIN) and will be speaking at ONUG’s AI Networking Summit in Dallas, May 13–14, 2026.