In this role, you will secure fal.ai’s compute platform beneath the application layer, spanning Linux kernel and virtualization components through GPU drivers, firmware, and host isolation. The focus is on building robust isolation for shared GPU infrastructure, reducing real-world risk, and driving security improvements across the stack.
Key Responsibilities
- Develop and maintain a threat model for executing code on shared GPU hosts, covering containers and microVMs down to PCIe, IOMMU, and GPU passthrough.
- Design and validate GPU isolation controls, including memory scrubbing and device-state scrubbing between tenants.
- Support deployment of hardware-backed isolation where appropriate, including confidential computing and attestation.
- Harden kernel builds and host images using config minimization, LSMs, seccomp, and namespace and cgroup policy, with secure and measured boot.
- Reduce attack surface across GPU-related components, including GPU drivers, CUDA runtimes, NVLink/NVSwitch fabrics, BMC, and firmware.
- Assess and triage security vulnerabilities in the kernel, hypervisors, GPU drivers, and firmware, including judging exploitability and real impact, setting severity, and driving fixes or mitigations.
- Monitor vendor bulletins and upstream disclosures, and run kernel and driver patching that keeps pace without disrupting the fleet.
- Perform vulnerability research on key dependencies.
- Contribute to the host and isolation security roadmap and evaluate relevant tooling.
- Coordinate with Detection & Response on telemetry and serve as the escalation point for host, kernel, and driver incidents.
- Review security-relevant designs from Infra, Platform, and ML teams, helping make the secure approach the default.
- Collaborate with GPU vendors, cloud providers, and upstream maintainers on disclosure processes and fixes.
Required Qualifications
- 5+ years in systems or security engineering with deep, hands-on experience in the Linux kernel, hypervisors, or device drivers.
- Fluency in kernel C, with comfort using Rust, Go, or Python for tooling.
- Strong understanding of virtualization internals, including KVM, VFIO, and IOMMU/DMA protection, plus tradeoffs across containers, sandboxes, and microVMs.
- Experience shipping hardening measures that run in production at fleet scale.
- Proven ability to find, triage, or fix real vulnerabilities in low-level code and explain the exploit path.
- Ability to communicate risk clearly to both engineering teams and executives.
Technologies You Will Work With
Linux kernel, hypervisors, microVM, PCIe, IOMMU, GPU passthrough, LSMs, seccomp, namespace and cgroup policy, secure and measured boot, GPU drivers, CUDA runtimes, NVLink, NVSwitch, BMC, kernel C, Rust, Go, Python, KVM, VFIO, IOMMU/DMA protection, containers, sandboxes, LLMs, agents, AI coding tools, NVIDIA GPU stack, open kernel modules, GSP firmware, MIG, vGPU, GPU confidential computing, AMD SEV-SNP, Intel TDX, attestation, confidential computing.
Location and Employment
- Location: San Francisco, CA (onsite)
- Employment type: Full time
- Department: Engineering
Compensation
- Salary range: USD 180,000 - 250,000 per year
- Equity: Offers Equity
- Additional compensation statement: $180,000 - $250,000 + equity + comprehensive benefits package
Nice-to-Haves / Qualifications
- AI fluency: daily use of LLMs, agents, and AI coding tools for code review, tooling, and vulnerability research.
- NVIDIA GPU stack experience: open kernel modules, GSP firmware, MIG, vGPU, or GPU confidential computing.
- Confidential computing experience: AMD SEV-SNP, Intel TDX, and attestation.
- Experience with published CVEs, talks, or write-ups on kernel or hypervisor security.
- Prior work at a GPU cloud, hyperscaler, or chip vendor.
Benefits
- Health, dental, and vision insurance (US)
- Visa sponsorship and relocation support to San Francisco
- Regular team events and offsites
- Interesting and challenging work
- Significant learning and growth opportunities
