Senior Software Engineer – TensorRT Edge-LLM
Location
Santa Clara, CA
Job Type
Full-time
Category
other-general
Posted
June 03, 2026
Are you passionate about pushing the limits of real-time large language model inference? Join NVIDIA’s TensorRT Edge-LLM team and help shape the next generation of edge AI for automotive and robotics. We build the software stack that enables Large Language, Vision-Language, and Multimodal (LLM/VLM/VLA) models to run efficiently on embedded and edge platforms — delivering cutting-edge generative AI experiences directly on-device.
What you’ll be doing:
+ Develop and evolve a state-of-the-art inference framework in modern C++ that extends TensorRT with autoregressive model serving capabilities, including speculative decoding, LoRA, MoE, and KV cache management.
+ Design and implement compiler and runtime optimizations tailored for transformer-based models running on constrained, real-time platforms.
+ Collaborate with teams across CUDA, kernel libraries, compilers, and robotics to deliver high-performance, production-ready solutions.
+ Contribute to CUDA kernel...
What you’ll be doing:
+ Develop and evolve a state-of-the-art inference framework in modern C++ that extends TensorRT with autoregressive model serving capabilities, including speculative decoding, LoRA, MoE, and KV cache management.
+ Design and implement compiler and runtime optimizations tailored for transformer-based models running on constrained, real-time platforms.
+ Collaborate with teams across CUDA, kernel libraries, compilers, and robotics to deliver high-performance, production-ready solutions.
+ Contribute to CUDA kernel...