Python Engineer, AI Coding Agent Evaluator

G2i is seeking a highly experienced Senior or Staff-level Software Engineer to join as an AI Interaction Evaluator. This unique role focuses on assessing the quality of interactions with modern AI coding agents, such as OpenAI Codex and Claude Code, to determine if they demonstrate the reasoning and judgment expected of a top-tier developer.

Key responsibilities

• Evaluate AI-generated coding interactions end-to-end to ensure they align with strong engineering standards.
• Assess the quality of explanations and reasoning provided by AI models rather than just focusing on syntax.
• Provide clear, opinionated feedback on what works, what fails, and what feels misleading in AI-assisted workflows.
• Help define the benchmarks for what constitutes high-quality interaction when using tools like Cursor.
• Distinguish between varying levels of response quality to improve model performance.

Requirements

• Staff or Principal-level engineering experience or equivalent deep technical background.
• Strong proficiency in Python, TypeScript, or JavaScript.
• Hands-on experience with AI-assisted development tools such as OpenAI Codex, Claude Code, or Cursor.
• Ability to make rigorous, subjective judgments on engineering taste and developer experience.
• Comfortable providing direct, critical feedback on complex technical outputs.

What we offer

• Opportunity to shape the future of AI-assisted software development workflows.
• Flexible engagement model with 10–20 hours of work per week.
• Collaborative environment focused on defining high-level engineering standards.
• Direct involvement in evaluating cutting-edge AI coding technologies.