CLIP Image Filtering on AMD Ryzen AI Max

A feasibility & capacity report — running ONNX-optimized OpenAI CLIP as a real-time video-stream filter on Ryzen AI Max+ 395 (Strix Halo), versus an Nvidia setup.

TL;DR — Porting an ONNX CLIP filter from Nvidia to Ryzen AI Max is an ONNX Runtime execution-provider swap, not a rewrite. For a low-fps continuous filter the chip is arguably a better fit than a big discrete GPU on power and always-on efficiency. At scale (50 streams), the binding constraint is not CLIP inference — it's whether 50 concurrent video decodes stay on the hardware decoder or spill to the CPU.

Note on figures: throughput numbers below are engineering budget estimates derived from published model FLOP counts and chip specs — there is no public CLIP-on-Strix-Halo benchmark to cite. Treat them as ballpark sizing, not measured results.

The workload

OpenAI CLIP used as an image-embedding filter over a live video stream: decode frames, run each through CLIP's image encoder, compare the embedding against target prompt embeddings, keep/drop by a similarity threshold. Pipeline is multithreaded OpenCV feeding ONNX Runtime. Currently running well on Nvidia (CUDA / TensorRT via ONNX Runtime).

How CLIP runs on Ryzen AI Max

The AI Max+ 395 (Strix Halo) has three places to run inference, and for CLIP the choice matters:

Target	ONNX Runtime path	Best for
XDNA 2 NPU ~50 TOPS INT8	Vitis AI execution provider (AMD Ryzen AI SW stack)	Low-power, always-on lightweight vision — purpose-built for exactly this
iGPU 40 CU RDNA 3.5 (Radeon 8060S)	DirectML (Windows) or ROCm/MIGraphX (Linux)	Highest raw throughput; heavier models / larger batches
CPU 16× Zen 5 / 32T	Default CPU EP	Fallback / unsupported ops

The port is a matter of swapping the ONNX Runtime execution provider from CUDA to Vitis-AI (NPU) or DirectML/ROCm (iGPU). Same ONNX model file, different EP. That's essentially the whole job.

⚠️ Two gotchas

NPU wants INT8. To run on the XDNA NPU you quantize the CLIP encoder to INT8 (AMD's Quark / Vitis quantizer). Tiny accuracy hit — usually fine for a filter (it's a similarity threshold, not pixel-perfect). Want FP16 with zero fuss? Target the iGPU via DirectML/ROCm instead — easier port, still plenty fast.
First-run compile. Vitis AI compiles the ONNX graph when the ORT session starts (can take a couple of minutes). One-time, cached — don't panic at cold-start.

Scaling to 50 streams × 2 fps × 640

That's 100 CLIP inferences/sec plus 50 concurrent video decodes. The single most important fact:

CLIP fixes its input resolution CLIP always resizes each frame to the model's trained size (e.g. 224×224) before inference. So "640 res" only affects your OpenCV decode/resize cost — it does not change the inference cost at all. Source resolution is a decode-pipeline concern, not a model concern.

The model variant swings the compute ~15–20×

This is the fact that decides everything on the inference side — pin it down before sizing anything:

Variant	~GFLOPs / image	Compute @ 100 inf/s	On Ryzen AI Max
ViT-B/32	~4.4	~0.44 TFLOPs/s	Trivial — a rounding error on this chip
ViT-L/14	~80	~8 TFLOPs/s	Real load, but comfortably in iGPU territory — size it properly

B/32 laughs it off. L/14 is fine on the iGPU but no longer "free." Compute is not the limiting factor either way — but which variant sets how much headroom you keep.

The actual ceiling: 50 simultaneous decodes

The scaling stress isn't CLIP — it's the front of the pipeline:

Hardware video decode (VCN). Strix Halo's media engine has a concurrent-session limit. If 50× H.264/H.265 streams fit on the VCN hardware decoder, decode is nearly free. If it overflows to software decode, that lands on the 16 CPU cores — and that is what falls over first, long before the NPU/iGPU breaks a sweat. Verify the concurrent-decode limit for your codec/resolution.
50 OpenCV pipelines. Even at 2 fps, 50 threads doing decode → colour-convert → resize → tensor is real CPU work. This is the likely practical bottleneck — keep it multithreaded, pin threads, avoid per-frame allocations.
Batch across streams. Don't run 50 independent ORT sessions. The 2 fps cadence gives a ~500 ms window per frame — huge room to gather frames across streams into a single batched EP call (~100 at a time) and fire once. Naive 50-sessions is the slow path; batching is what makes 100 inf/s effortless.

Memory bandwidth (256 GB/s unified) is a non-issue at this frame budget.

Verdict

On paper this is very doable on a single Ryzen AI Max. The inference budget is small (trivially so for B/32), and the chip's whole pitch is efficient always-on vision. The two things that actually decide it:

Which CLIP variant (B/32 vs L/14) — sets your inference headroom.
Whether 50 concurrent decodes stay on the VCN hardware decoder or spill to CPU — sets your true ceiling.

Where Nvidia still pulls ahead: NVDEC offers very high concurrent-decode limits, and TensorRT gives mature batched inference with day-one tooling. If decode concurrency turns out to be the wall, that's the axis where a discrete card wins. If it's about efficient, quiet, always-on lightweight vision at modest fps, Ryzen AI Max is the better-fitting box.