The Hidden Network: Ray Control-Plane Exposure in Distributed LLM Inference on Kubernetes (Empirical Study, EKS)

We evaluated the runtime network surface of distributed LLM serving (KubeRay + vLLM pipeline parallelism) on an Amazon EKS 1.35 cluster from the perspective of an unprivileged neighbour pod in an unrelated namespace. Summary of observations: – 15 of 17 observed Ray listening sockets were omitted from pod declared containerPort metadata. – An unprivileged neighbour pod in an unrelated namespace reached Ray GCS (6379) and raylet RPC (10002 to 10006) over unauthenticated cleartext gRPC. – Four default security scanners (Trivy, Checkov, Kubescape, kube-linter) did not inspect the RayCluster custom resource, failing to represent runtime exposure. – An ingress default-deny NetworkPolicy blocked all probed Ray ports from the neighbour pod in steady state. – WireGuard encryption via Cilium chained with AWS VPC CNI carried the workload with an observed 3.4% to 6.5% throughput drop across a bracketed single-run test. – The Ray Job API (8265) answered unauthenticated requests on default CPU Ray images, but was not listening in the GPU vLLM deployment (RCE against the GPU inference stack was not demonstrated). All test manifests, scanner outputs, network logs, and repro scripts are open source: https://github.com/Sorami-Consulting-AU/distributed-llm-inference-hidden-network submitted by /u/No-Peanut-6988 [link] [comments]Technical Information Security Content & DiscussionRead More