TWG Global AI

TWG Global AI

Senior / Staff Software AI Test Engineer, AI Engineering

Santa Monica, California, United States · Staff+

Sponsorship not specified$190k-$250kDetected 69 days ago
PythonJavaiOSObject-Oriented ProgrammingCode ReviewGitAzureDockerCI/CDGitHub ActionsJenkinsCircleCILLMsRAGAgentic AILangGraphSeleniumCypressPlaywrightpytestTest Automation

About the role

  • This is a software engineering role focused on test automation.
  • You'll work shoulder-to-shoulder with AI engineers and data scientists, contributing production-quality code to shared repositories.

Responsibilities

  • Framework and harness engineering Design and build scalable, reusable test automation frameworks for AI agents, LLM-powered applications, and underlying APIs.
  • Treat test code as production code: code review, type hints, documentation, library design.
  • Evaluation infrastructure Build evaluation infrastructure for benchmarking agent performance against SOTA LLMs, competitors, and internal baselines.
  • Own regression suites, golden datasets, rubric-based evals, and metric dashboards.
  • Build tooling for synthetic test data generation, edge-case discovery, and adversarial testing.
  • Resilience and load Design and run release, system, performance, and load tests against streaming, stateful, and async systems.
  • Build chaos and fault injection tooling for token expiry, connection pool exhaustion, provider failover, and cache pressure scenarios.
  • Drive contract testing across LLM providers (Bedrock, Anthropic, OpenAI) to catch parity drift.
  • Build trace-based assertions on LangGraph state, tool calls, and agent decisions - debugging an agent failure means replaying graph state, not re-running a prompt.
  • Human-in-the-loop and partnership Implement HIL review workflows where automation alone cannot validate quality, then push the automation boundary outward.

Requirements

  • 3-7 years of software engineering experience, with a meaningful portion focused on test automation, SDET, or software engineering in test roles.
  • Experience testing across multiple client surfaces: iOS apps, plugins, and Chrome extensions.
  • Proven experience integrating test automation into CI/CD systems (GitHub Actions, Jenkins, CircleCI, GitLab CI, or similar).
  • Bachelor's degree or higher in Computer Science, Engineering, or a related field.

Nice to have

  • Experience with Azure (our primary cloud) and containerization (Docker).
  • Experience testing RAG pipelines, agentic workflows, or multi-step tool-calling systems.
  • Compensation: The base pay for this position is $190,000-250,000.

Skills

  • Our agents are written in LangGraph and run on Azure on the TWG side, with a parallel Vercel-based stack on the Palantir side.

Compensation

  • The base pay for this position is $190,000-250,000.
  • A bonus will be provided as part of the compensation package, in addition to a full range of medical, financial, and/or other benefits.

Benefits

  • Champion quality engineering practices across the team: code review, coverage standards, observability, reproducibility.

Company info

  • code review, coverage standards, observability, reproducibility.
  • Ensure user-centric validation so AI outputs are accurate, reliable, and meet real-world application needs.
Apply directly at TWG Global AI →Create a free account for alerts like thisView TWG Global AI immigration profile

This listing is sourced directly from TWG Global AI's careers page and normalized into a canonical job model.