AI Engineer, Evaluation

<div class="content-intro"><p><strong>About Mixpanel</strong></p> <p>Mixpanel turns data clarity into innovation. Trusted by more than 29,000 companies, including Workday, Pinterest, LG, and Rakuten Viber, Mixpanel’s AI-first digital analytics help teams accelerate adoption, improve retention, and ship with confidence. Powering this is an industry-leading platform that combines product and web analytics, session replay, experimentation, feature flags, and metric trees. Mixpanel delivers insights that customers trust. Visit <a href="mixpanel.com" target="_blank">mixpanel.com</a> to learn more.</p></div><p><strong>About the Team</strong></p> <p>The AI Platform team is a newly formed team at the center of Mixpanel's AI-first analytics vision. With a greenfield charter, we're building the infrastructure that accelerates and transforms AI product development at Mixpanel, both internally and for our customers.&nbsp;</p> <p>We provide the tools to improve AI products, enabling internal teams and customer agents to build things that were previously unimaginable.&nbsp; We build shared infrastructure that is essential for developing AI features with confidence and speed, and that gives every agent the tools, context, and quality guarantees to act autonomously on behalf of users.</p> <p>Some examples of what we are building:</p> <ul> <li><strong>Agent Optimization Framework:</strong> A system that optimizes AI products for quality, speed, and cost, using metrics from production and evals, given the context of the invocation.</li> <li><strong>AI Agent Integrations and Accessibility:</strong> Products and tools that bring the power of Mixpanel to wherever it is most effective for our customers, including a Mixpanel slackbot, MCP, and a public skills library.</li> <li><strong>AI Engine:</strong> Services that centralize core AI and LLM functionality at Mixpanel in order to accelerate AI development, compound the value of AI efforts, and provide customers with a consistent experience.</li> </ul> <p><strong>Role Overview</strong></p> <p>We are looking for an experienced and driven Senior Software Engineer to join our AI Platform team. You will be responsible for building the scalable, secure, and reliable infrastructure that accelerates AI agent development.&nbsp; You will be a leader and key contributor in a small, fast paced, newly formed team with a mandate to empower AI development across the company.</p> <p><strong>Responsibilities</strong></p> <ul> <li>Platform Architecture: Design and develop the core backend services, APIs, and microservices that enable product teams to easily and securely leverage AI models.</li> <li>Agen