Lead AI/ML Inference Optimization Engineer (PyTorch / CUDA) at NeuralFoundry

Job Description

NeuralFoundry AI serves low-latency frontier language and multimodal models for enterprise code generation. We are looking for a Lead AI/ML Inference Optimization Engineer to slash time-to-first-token and maximize tokens-per-second throughput.

Technical Scope You will benchmark FP8 and INT4 weight quantization kernels, customize continuous batching schedulers in vLLM and TensorRT-LLM, and orchestrate InfiniBand RDMA multi-node inference pods on Kubernetes. Your optimizations directly govern GPU cluster utilization and customer latency SLAs.

What We Look For Hands-on expertise with PyTorch, Python, Triton kernels, or custom CUDA C++ operators. Deep understanding of transformer attention mechanisms, PagedAttention, and speculative decoding. Experience debugging distributed NCCL communication collectives across multi-GPU nodes.