Principal Engineer - Data Retrieval

NVIDIA is seeking an engineering lead to architect and scale the data acquisition infrastructure that powers our foundation models. Starting with code and STEM data for Nemotron, you will build the systems that discover valuable sources, crawl the open web at scale, and deliver reliable, high-quality data to training pipelines. This is a hands-on technical leadership role spanning distributed systems, crawl infrastructure, and third-party data ingestion. You will own the architecture of a unified platform covering source discovery, URL indexing, crawl frontiers, predictive ranking, scheduling, distributed fetching, and ingestion. We have existing infrastructure to build on, and the goal is to bring it to pretraining scale and extend it to additional model families, modalities, and acquisition workflows over time. Along the way, you will lead architecture reviews and critical implementation work, mentor engineers, and help build the team. What you’ll be doing: Work with storage and scaling teams to design scalable frontier storage and queueing systems with high-throughput writes, efficient lookups, deduplication, and scheduling based on data quality, diversity, and freshness. Build a continuous discovery loop that extracts links and feeds new candidates back into the crawl frontier. Develop a shared crawl execution layer that fetches reliably and politely at scale, handling dynamic websites while keeping resource use under control. Build extensible pipelines for open-web, lice