The AVP Penetration Tester (LLM) will lead internal offensive security work with a specific focus on AI and agentic built applications, including testing of associated APIs and infrastructure. The role emphasizes end-to-end assessment execution, stakeholder communication, and repeatable reporting with retesting to confirm remediation closure.
Location and Work Model
Austin, TX (hybrid)
Compensation
USD 128,647 - 214,343 per year
Role Responsibilities
- Coordinate end-to-end internal penetration testing activities by partnering with product stakeholders and working with Security Architects during SDLC processes for LLM and AI initiatives, including early recommendations prior to production deployment.
- Execute tactical security penetration testing to validate the security of company applications, including LLMs, agentic built applications, and supporting APIs, aligned to OWASP Top 10 threats; provide feedback and recommendations with the Application Security team to increase automated capabilities, including AI-assisted tooling.
- Identify vulnerabilities and devise mitigation strategies for AI systems, including creative approaches to test and circumvent security guardrails.
- Stay current on emerging TTPs, zero-days, and remediation strategies relevant to agentic developed applications, systems, and their supporting infrastructure.
- Develop or modify custom tooling to address new testing needs using models such as Claude.
- Document and formally report testing initiatives, findings, justified risk ratings, remediation recommendations, and validation results in a clear and concise format.
- Partner with technology teams to present results, communicate the threat posed, and provide remediation guidance that is understandable to IT stakeholders.
- Perform security assessments across internal and external networks, infrastructure, cloud environments, and a range of internally developed and commercial products.
- Support development and maintenance of tools or scripts used in agentic penetration testing.
- Lead assigned AI/LLM penetration testing initiatives and communicate program status to leadership.
- Oversee communication and reporting for testing outcomes and conduct retesting to validate successful closure of previously identified findings.
- Develop tooling for agentic assisted LLM penetration testing.
Required Experience
- 5+ years conducting application/API and network-based penetration testing engagements.
- 5+ years troubleshooting tools, manually finding issues in code, and rewriting code to remove bugs.
- 3+ years leading penetration testing engagements end-to-end.
- 2+ years experience pentesting AI/LLM/GenAI applications.
- 2+ years experience using AI models such as Claude to rapidly develop tooling.
Technologies and Tools
OWASP Top 10 threats, Burp Suite, Promptfoo, HexStrike, Claude, CAI, Garak, Pyrit, Kali Linux, Nessus, Metasploit, Cobalt Strike, Mitre Atlas, OWASP Top 10 for LLM, OSCP, OSAI, OSCE, OSWE, GPEN, GCIH, GWAPT, GXPN, MITRE ATT&CK framework, Atlas, SDLC, MCP servers, Models, Chatbots, Claude Code, .NET, JavaScript, Python, Java, PowerShell, Perl, Ruby, Bash, C#, Golang, Linux, Mac, Windows, AWS, Azure, Containers, Kubernetes, microservices, serverless functions
Benefits
- 401K matching
- Health benefits
- Employee stock options
- Paid time off
- Volunteer time off
Core Competencies
- Advanced knowledge of security assessment tools and frameworks, including Burp Suite, Promptfoo, HexStrike, Claude, CAI, Garak, Pyrit, Kali Linux, Nessus, Metasploit, Cobalt Strike, Mitre Atlas, OWASP Top 10 for LLM, and similar.
- At least one industry certification such as OSCP, OSAI, OSCE, OSWE, GPEN, GCIH, GWAPT, or GXPN.
- Excellent communication skills and the ability to collaborate with internal and external stakeholders while finding issues, advising on security, and implementing solutions.
- Strong organizational skills.
- High curiosity for experimenting with and testing security features and controls.
Preferred Qualifications
- Bachelor’s Degree or equivalent in Information Security, Engineering, or Computer Science.
- Advanced understanding of OWASP, the MITRE ATT&CK framework, Atlas, and the software development lifecycle (SDLC).
- Advanced knowledge and experience penetration testing AI/LLM systems including MCP servers, Models, and Chatbots.
- Advanced understanding of tool development with AI such as Claude Code.
- Advanced knowledge in programming languages including .NET, JavaScript, Python, Java, PowerShell, Perl, Ruby, Bash, C#, Golang, or similar.
- Advanced level knowledge of Linux/Mac/Windows operating systems and AWS/Azure cloud environments, including cloud-native resources such as Containers, Kubernetes, microservices, serverless functions.
- Strong breadth and depth of knowledge in security of LLM systems, including MCP servers, models, tooling, and infrastructure.
Work Authorization Notice
This position does not offer work authorization sponsorship now or in the future. Applicants must have valid U.S. work authorization that does not require employer sponsorship.
