Senior Software Engineer

This is a 6-month contract role under Optimum Solutions.This role supports our client, a leading Southeast Asian superapp that offers a wide range of services, including mobility, food delivery, and digital payments.Work Schedule and Hours: Monday - Friday, 10am - 7pm (inclusive of a 1-hour lunch break)Key Responsibilities:• Take ownership of the ACS embedded-evaluation workstream.• Establish reliable regression evaluation for at least one additional agent team.• Make its distributed traces consistently usable for debugging and automated evaluation.• Ship at least one reusable improvement to EvalsHub or its SDK/tooling.• Reduce manual evaluation effort through automated datasets, evaluators, and CI integration.• Document ownership, operational procedures, and onboarding guidance.• Embed with AI product teams and own measurable agent-quality outcomes.• Build and maintain golden datasets, regression suites, evaluation pipelines, and quality dashboards.• Convert product rules and human review procedures into deterministic, numeric, LLM-as-judge, trajectory, tool-selection, SOP-adherence, and multi-turn evaluators.• Diagnose evaluation and production-trace failures, then convert findings into dataset improvements, evaluator changes, or fixes to prompts, tools, skills, and agent workflows.• Instrument distributed agent systems using OpenTelemetry, including Temporal workflows and Go/Python services.• Ensure traces contain consistent inputs, outputs, tool calls, metadata, feedback, and root-agent results.• Develop reusable capabilities across EvalsHub backend, SDK, CLI/plugins, and supporting frontend interfaces.• Integrate evaluations into CI/CD and model-release workflows.• Run model, retrieval, embedding, and agent-architecture experiments; balance accuracy, latency, and cost.• Drive migrations from legacy evaluation and observability systems.• Write technical designs and documentation, run demonstrations, and enable engineering teams to use evaluation tooling independently.• Participate in the LLMOps operational rotation and support production-quality integrations.Qualifications:• Minimum 2 years of relevant experience in one or more of the following areas:> EvalsHub, LangSmith, LangGraph/LangChain, FastAPI, Temporal, Grafana, Redis, or similar technologies.> React/TypeScript and internal developer-tooling interfaces.> LLM-as-judge, multi-turn evaluation, tool/MCP evaluation, and agent-trajectory analysis.> Text-to-SQL/Text-to-DSL, RAG, semantic retrieval, embedding evaluation, or model benchmarking.> Building platform capabilities that can be reused across several AI products.• Strong analytical and problem-solving skills related to AI systems, evaluation frameworks, and data-driven quality improvements.• Excellent project management and cross-department communication abilities.• A detail-oriented and metrics-driven approach to continuous improvement