Principal Engineer -Testing
Invisible Technologies
About Invisible
Invisible Technologies is the AI training and scaling partner for the leading foundation model providers, enterprises, and governments, bridging the gap between AI potential and production. Invisible’s unique AI Process Platform combines elite global human expertise, cutting edge technology, and deep institutional knowledge gained by training 80% of the world’s leading AI models. Trusted by AWS, Microsoft, and Cohere, we have an unparalleled ability to operationalize AI for real-world applications. Our explosive growth landed us the #2 spot on the Inc. 5000 in 2024, closing the year on $134m revenue.
About The Role
We are seeking a highly experienced Principal Engineer – Testing to join our dynamic engineering team. The ideal candidate will bring deep expertise in quality engineering, automation, and test strategy for complex, distributed systems. As part of the core team building our AI platform, you’ll design and lead the testing architecture for autonomous workflows, intelligent agents, and high-reliability infrastructure.
What You’ll Do
- Design, implement, and own the testing architecture across the AI platform, spanning backend, APIs, data pipelines, and AI models
- Build and scale automation frameworks to support unit, integration, regression, performance, and AI-specific testing
- Develop test strategies that balance reliability, speed, and coverage for complex, dynamic, and data-heavy systems
- Create simulation and sandbox environments to test AI workflows and orchestration logic
- Integrate automated testing into CI/CD pipelines, enabling confident, rapid deployments with safety guarantees
- Lead root cause analysis and implement practices that prevent regressions across systems and features
- Champion a quality-balanced culture and mentor engineers in best practices for building robust, testable systems
What We Need
- 10+ years of experience in software quality engineering, test automation, or systems testing roles
- Expertise in test automation frameworks (e.g., Pytest, Playwright, etc.) and infrastructure-as-code pipelines
- Strong programming skills in Python, JavaScript/TypeScript
- Experience designing and validating tests in distributed, event-driven, and microservices architectures
- Familiarity with testing in AI/ML systems, including model output validation and behavior-driven testing.
- Experience with containerized environments (Docker), and cloud platforms (AWS/GCP/Azure)
- Strong understanding of CI/CD systems (particularly Git,) and modern observability tooling (e.g., Grafana, Prometheus)
- Excellent problem-solving, debugging, and documentation skills
- A track record of technical leadership and mentoring in cross-functional environments
Nice to have
- Experience testing large-scale AI frameworks, LLM integrations, or decision systems
- Exposure to tools like dbt, Spark, or DataDog
- Contributions to open-source testing tools or quality initiatives
- Deep curiosity about LLM behavior, autonomous systems, and human-in-the-loop testing strategies
- A desire to develop the most consequential AI software system of the future!
What’s In It For You
Compensation:
Invisible is committed to fair and competitive pay, ensuring that compensation reflects both market conditions and the value each team member brings. Our salary structure accounts for regional differences in cost of living while maintaining internal equity.
An Invisible Talent Acquisition Partner can provide more information on which locations are included in each of our geographic pay tiers during the interview process. For candidates outside the U.S., compensation will be adjusted to reflect local market conditions and cost-of-living differentials.
Bonuses and equity are included in offers above entry level. Final compensation is determined by a combination of factors, including location, job-related experience, skills, knowledge, internal pay equity, and overall market conditions. Because of this, every offer is unique. Additional details on total compensation and benefits will be discussed during the hiring process.
What It’s Like to Work at Invisible:
At Invisible, we’re not just redefining work—we’re reinventing it. We operate at the intersection of advanced AI and human ingenuity, pushing the boundaries of what’s possible to unlock productivity and scale. Ownership is at the core of everything we do. Here, you won’t just execute tasks—you’ll build, innovate, and shape the future alongside world-class clients pushing the boundaries of AI.
We expect bold ideas, relentless drive, and the ability to turn ambiguity into opportunity. The pace is fast, the challenges are big, and the growth is unmatched. We’re not for everyone, and we’re okay with that. If you’re looking for predictable routines, this isn’t the place for you. But if you’re driven to create, thrive in dynamic environments, and want a front-row seat to the AI revolution, you’ll fit right in.
Country Hiring Guidelines:
Invisible is a remote-first organization and hires new team members in countries around the world. Although many of our roles are fully remote, some roles may carry specific location-based eligibility requirements. Our Talent Acquisition team can help answer any questions about location after starting the recruiting process.
AI Interviewing Guidelines:
Our hiring team thoughtfully uses AI to support an efficient, engaging, and inclusive interview process. Since AI can also be a helpful tool for candidates, we've outlined expectations for using it ethically throughout your interview journey. Click here to learn more about how we use AI and our guidelines for candidates.
Accessibility Statement:
We’re committed to providing reasonable accommodations for individuals with disabilities. If you need assistance or accommodation due to a disability, please contact our Talent Acquisition team during the recruitment process at accommodation@invisible.email.
Equal Opportunity Statement:
We’re an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, or veteran status, or any other basis protected by law.
Due to a high volume of candidates, Invisible may use automated decision-maker technologies to filter candidates based on response to our application questions and other provided information. Our use of automated decision-making enables us to be efficient by providing a manageable list of possible candidates that meet our mandatory hiring criteria. If you object to our use of automated decision-making please contact us.