Publications / ARGUS
ARGUS · NDSS 2027
Hardware-Rooted Inference-Time Verification for LLM Execution on Untrusted GPUs
Network and Distributed System Security Symposium (NDSS 2027). To appear.
A model response does not, by itself, show which model produced it or whether its computation was carried out faithfully. ARGUS uses a CPU confidential VM to check LLM execution on an untrusted GPU and attach verifiable execution evidence to each response.
What needs to be verified
An application may receive a response from a cloud API, rented GPU, or external agent. Before using that response, it may need evidence that the intended model ran on the stated request under an expected runtime and verification policy.
ARGUS puts the root of trust in a CPU confidential virtual machine (CVM), using AMD SEV-SNP in the prototype. The GPU performs the full LLM forward pass but remains outside the trusted computing base. This avoids requiring a GPU TEE. The CPU verifier checks the computation and signs a receipt binding the model, request, response, measured runtime, and verification policy.
How ARGUS works
Matrix multiplications are checked with Freivalds-style randomized projections whose secret randomness stays inside the CVM. Fixed-weight projections can be precomputed. Nonlinear operations—including normalization, positional encoding, masking, softmax, residual additions, and activations—are recomputed on the CPU. Cross-layer checks connect these local checks into an execution chain.
Compute on the GPU
The untrusted accelerator runs the model and supplies the intermediate values needed for verification.
Check inside the CVM
An attestable verifier checks matrix products, recomputes nonlinear operations, and validates the chain between layers and the token-selection policy.
Release with a receipt
After verification completes, release the response and a signed receipt binding it to the request, model, runtime, and policy.
- Overlap work across layers. A pipeline overlaps GPU computation, transfers, copying into trusted memory, and CPU verification.
- Transfer only what the verifier needs. Layer inputs and matrix-product outputs form a minimal transfer set; the verifier reconstructs nonlinear intermediates. Packed transfers reduce crossings through shared bounce buffers. Each frame is copied into protected CVM memory before it is checked, preventing a later GPU write from changing the data being verified.
- Separate generation from verification scheduling. Prefill verification overlaps decoding. After generation, ARGUS checks the suffix in a batch using the already verified prefix KV cache, reducing token-by-token synchronization.
The CPU CVM is trusted; the GPU and shared transfer buffers are not. The prototype verifies the generated suffix before releasing the response, rather than streaming individually verified tokens.
What the experiments show
The testbed combines an AMD EPYC 9115 CPU, an AMD SEV-SNP confidential VM, and an NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB of memory. Experiments use FP32, eager attention, batch size one, and a single GPU, across six open-source models in the 1B–8B range. Reported serving timings exclude one-time initialization.
- 18–74% end-to-end overhead on balanced workloads. This is relative to unverified GPU inference under the same configuration, when the prompt-to-generation length ratio is at most two (T/G ≤ 2). Prompt-heavy workloads have higher overhead: 45–114% in the reported evaluation (§V and Appendix B).
- Every token-changing corruption was detected in the tested arithmetic attacks. Tests on Qwen3-1.7B cover precision reduction, single-element perturbations, row noise, weight quantization, selectively skipped matrix multiplications, and skipped layers. The honest evaluation reports zero false positives (§VI). This is an experimental result over those attacks and settings, not a guarantee that every possible modification is detectable.
- Verification is sensitive to floating-point tolerances. Small deviations can fall within the calibrated tolerance without changing output tokens. The security analysis distinguishes these numerical effects from the exact-arithmetic guarantees of randomized matrix checks.
What the evidence means
The receipt concerns faithful execution under the stated model and verification policy. It does not establish that an answer is factually correct, safe, or useful. ARGUS also does not hide model weights, prompts, or intermediate values from the GPU or inference provider.
The design trusts the CPU hardware and firmware, the CVM guest kernel, and the measured verifier and inference software stack. Denial of service and side channels are outside its scope. Default greedy decoding checks the selected argmax; optional top-k membership checks show that a token belongs to the allowed candidate set, but do not prove a sampling distribution.
The current prototype covers dense models with eager attention in the evaluated single-GPU, batch-one configuration. Fused attention kernels, mixture-of-experts models, speculative decoding, and multi-GPU serving need additional work. Lower-precision inference requires new numerical calibration.
Paper and artifact
The full paper explains the design (§IV), performance evaluation (§V), security evaluation (§VI), and limitations (§VII). The archived artifact provides the implementation and reproduction instructions. Functional and performance experiments can run in a normal VM; the CVM and attestation experiments require SEV-SNP support.
Related: FLASH reduces checkpoint interruptions when both the CPU and GPU provide confidential computing. ARGUS instead verifies execution on an untrusted GPU. Explore the lab’s broader work on verifiable AI.
Abstract
When LLM inference is served across trust boundaries, the receiver has no technical means to verify that a response came from the intended model under a trustworthy runtime. This creates a need for per-inference execution evidence that can be checked before a downstream client consumes the result. Existing approaches do not yet provide such evidence for practical online LLM serving. Cryptographic proof systems remain too slow for online serving; GPU Trusted Execution Environment (TEE)-based systems require specialized hardware unavailable on most deployments; and prior CPU TEE-based systems lack the inference-specific optimizations required for practical evidence generation at serving time.
We present ARGUS, a system that uses a CPU confidential VM (CVM) as a hardware root of trust to generate per-inference execution evidence for GPU-accelerated LLM serving. The GPU remains an untrusted accelerator and runs the full LLM forward pass, while an attestable verifier inside the CVM checks the execution and signs a receipt binding the response to the model, request, runtime, and verification policy. ARGUS is built around system-level optimizations that reduce CPU-GPU data movement, trust-boundary crossings, and token-level synchronization. Together, these optimizations make it possible to generate evidence at inference time while preserving most of the GPU serving throughput. On six open-source models (1B–8B parameters), ARGUS adds only 18–74% end-to-end overhead on balanced workloads, relative to unverified GPU inference, compared to 45× GPU-prefill latency for a reimplemented Slalom-style synchronous verifier. Across six arithmetic-attack classes, ARGUS detects every token-changing corruption with zero false positives across our honest evaluation.
Based on the linked paper. Read the full paper for the design, threat model, and evaluation. Lab members and advisees are shown in bold.