How Transit AI Handles Nondeterminism in Network Operations

Transit AI takes a different approach to AI guardrails — it makes the model’s judgment structurally irrelevant to whether anything unsafe can happen.

LLMs are probabilistic: ask the same question twice, get two different answers. That’s a shrug for a chatbot and a real problem when the model in question is proposing commands to a production router or firewall. Most AI guardrails treat nondeterminism as a content-filtering problem — score the output, then allow or block it. A hands-on review of Transit AI, an SSH client with a built-in network-ops AI agent, is worth reading closely for the opposite bet: guardrail the architecture, not the output, so an unpredictable model has nothing unsafe to reach for in the first place.

Behavioral guardrails are a bet on the model. Structural guardrails are a bet on the code.

A behavioral guardrail trains or prompts a model not to do something, then screens its output for signs it did it anyway. That approach inherits the model’s own nondeterminism — the same jailbreak attempt can fail nine times and succeed on the tenth, and a filter tuned against last month’s failure mode has no guarantee against next month’s. CSO Online’s reporting on the AI harness as an attack surface documents this failure pattern independent of any single vendor: across Anthropic’s, Google’s, and OpenAI’s own automation repositories, researcher Elad Meged of Novee Security found the same recurring architecture flaw — “decide in one place, consumed in another with more power” — where a downstream component trusted an upstream security decision without re-validating it. Nondeterminism doesn’t cause that flaw, but it guarantees you can’t fully test your way around it: you can’t enumerate every input a probabilistic model might produce.

A companion argument, made independently in a self-published from-scratch build of a coding agent (“Agentic Coding: Build the Harness”), makes the structural case directly: “the real attack surface is the tool-call manifest,” not the prompt — a read-only manifest cannot delete a repository no matter how creatively it’s prompted, because the capability to delete simply isn’t wired in. That’s the same logic Transit AI applies to network operations. Per its own security-model documentation, the agent’s toolset is fixed at six items — list sessions, read redacted scrollback, propose a command, propose a read-only API call, query a packet capture summary, ask a clarifying question — and nothing else exists to invoke, whatever the model outputs. Browsing local files, opening new connections, editing device inventory: the docs describe the agent as “structurally blind to” these, not merely instructed to avoid them. Expanding the menu requires a shipped code change, not a better prompt.

Two gates, not one, and neither is optional

The mechanism that matters most for network operations specifically is the double gate on any proposed command: a per-vendor allow-list of read-only commands (Junos, Cisco IOS/IOS-XE/NX-OS, Arista EOS, Palo Alto PAN-OS), and explicit human approval showing the literal command and target device. An “always approve show commands” convenience shortcut only removes a click — it cannot reach past the permit list to something the list doesn’t cover. Generic Linux shells get the opposite treatment: since no clean read-only command subset exists for a general shell, Transit AI drops the permit list entirely and requires individual approval with a persistent warning banner on every command. That’s a tell that the two-gate model isn’t theater — where the structural guarantee can’t be made, the docs say so and fall back to friction instead of a false sense of safety.

Credential handling gets the same treatment. Transit AI names the isolation between the AI and the credential store the “agent firewall”: the AI has no programmatic read path to credentials, which live only in the OS keychain, and an automated build check fails if that isolation is ever wired around. Device output is redacted (PEM blocks, encrypted secrets, cloud credentials, JWTs) before the model ever sees it — an explicit defense against a compromised device injecting instructions back through its own output. Worth noting: even a successful injection has nowhere to go, because the permit list applies regardless of what triggered the proposed command.

What it looked like against a real firewall

The review’s most useful evidence is a single unscoped prompt against a lab PA-440: “please review all security policies and make sure they align with CIS.” No rule names, no benchmark version. The agent enumerated all five configured policies, pulled each rule’s config, and produced six findings mapped to the CIS Palo Alto Firewall 10 Benchmark — no logging on any rule, no security profiles on the allow rules, two overly broad any/any/any rules (one of them, Netgear-allow, permitting untrusted inbound traffic with no restriction), and missing logging on both default rules. Every finding came from read-only queries; the agent made zero configuration changes. Fixing any of it required a human in configure mode — which is the entire point of the design, not an incidental limitation.

That last part deserves a second look next to the Cloud Security Alliance’s writeup on AI incident-response playbooks, which frames AI forensics as unusually hard precisely because non-deterministic systems don’t reliably reproduce their own behavior — the same malicious prompt can succeed in one session and fail in the next, so incident response has to rely on replaying interaction logs rather than re-running the system to see what happens. Transit AI’s structural model doesn’t make that forensics problem disappear — you’d still want full logging of every proposed command and every approval decision — but it does shrink what a bad outcome can look like. If the worst a compromised or simply erratic model session can do is propose a command that a permit list and a human have to separately approve, “what happened” and “what could have happened” converge much closer together than in a system where the guardrail is a post-hoc content filter.

Where to stay skeptical

This is a first-party hands-on review, not an independent audit or penetration test of Transit AI itself. The architectural claims — the agent firewall, the redaction layer, the build-time check that blocks credential access — come from Transit AI’s own documentation plus one practitioner’s lab testing against a single device family. The review discloses a paid personal subscription and no vendor compensation, which is worth noting but doesn’t substitute for third-party verification. The source material doesn’t say how permit lists are authored or versioned, doesn’t address vendors beyond the five named, and doesn’t say what happens when a network engineer needs a command family the permit list hasn’t caught up to yet. None of that undercuts the architectural argument — structural over behavioral guardrails is sound reasoning independent of any one vendor’s implementation — but it does mean the specific claims about this product should be treated as one well-documented data point, not a settled verdict.

Takeaways

• Nondeterminism is a different risk for a network-ops agent than for a chatbot: the same unpredictability that produces a mildly wrong answer in a chat window can produce a proposed command against production infrastructure.
• Behavioral guardrails (score and filter model output) inherit the model’s own unpredictability. Structural guardrails (restrict what code paths exist to invoke) don’t — a capability that isn’t wired in can’t be reached by any output, however the model produces it.
• Transit AI’s version of this: a fixed six-capability menu, a per-vendor read-only permit list, mandatory human approval on top of the permit list (not instead of it), and credential/output isolation layers — evaluated hands-on against a lab PA-440, where a single unscoped prompt produced a legitimate, CIS-benchmark-aligned firewall audit with zero configuration changes made.
• Structural guardrails narrow what “review the logs” has to reconstruct after the fact, which matters given how much AI incident forensics already depends on replaying interaction logs rather than re-running a non-reproducible system.
• The evidence here is one practitioner’s hands-on test plus a vendor’s own docs — treat the specific product claims as a data point, evaluate the architectural pattern (structural over behavioral) on its own logic, and verify independently before trusting either in a production network.

Security Sonar is Peter Campbell’s cybersecurity advisory practice, focused on security standards and guardrails for agentic AI. If your team is evaluating an AI agent with write access — or even read-only proposal access — to production network infrastructure, that evaluation should start with the same question this piece asks: is the safety property behavioral or structural? See securitysonar.com.