About the Role
Description
Hybrid role requiring onsite attendance in Cupertino, CA three (3) days per week. Contract opportunity. This role focuses on AI/LLM quality evaluation and measurement, not traditional model development. Candidates should have experience owning or leading evaluation strategies for production AI/LLM systems. We are seeking an experienced Machine Learning Engineer to help define, measure, and improve the quality of AI systems powered by Large Language Models (LLMs). In this role, you will design and scale evaluation frameworks that ensure AI-generated outputs are accurate, reliable, and consistent across a variety of use cases. You will partner closely with Product, Engineering, Research, and Data teams to establish evaluation methodologies, automate quality assessment workflows, and develop the metrics that drive model improvements. This is an ideal opportunity for someone who enjoys solving complex problems at the intersection of AI quality, evaluation strategy, automation, and data-driven decision making.
Top Skills
AI/LLM Evaluation & Quality MeasurementProblem Solving & System DesignCommunication & OwnershipPython, SQL & Automation
Top Responsibilities
Design and build LLM evaluation frameworks, including automated scoring systems, rubric-based grading, and LLM-as-a-Judge solutions.Define and track evaluation metrics, KPIs, and dashboards that measure model quality and performance at scale.Perform error analysis, root-cause investigations, and quality deep dives to identify opportunities for model improvement.Partner with cross-functional teams to integrate evaluation standards, workflows, and quality requirements into production environments.
What We're Looking For
Experience designing and maintaining LLM evaluation frameworks and measurement methodologies.Hands-on experience with autograders, LLM-as-a-Judge systems, and automated quality validation.Strong prompt engineering skills, including the development of evaluation prompts and grading rubrics.Proficiency with Python, SQL, and large-scale data analysis.Experience creating KPIs, dashboards, and executive-ready reporting on AI quality.Ability to evaluate model performance through error analysis, human-vs-model calibration, and quality benchmarking.Experience working with production AI/LLM systems and large evaluation datasets.
What Will Make You Stand Out
You've personally owned or led an AI/LLM evaluation strategy.You've built scalable systems to measure AI quality automatically.You can clearly explain how AI quality is defined, measured, and improved.You have experience influencing product and engineering decisions through evaluation insights and data.
Experience Level: Expert
Job Type & Location
This is a Contract position based out of Cupertino, CA.
Pay And Benefits
The pay range for this position is $55.00 - $65.00/hr.
Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors.
Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave)
Job Type & Location
This is a Contract position based out of Cupertino, CA.
Application Deadline
This position is anticipated to close on Nov 20, 2026.
Workplace Type
This is a fully remote position.
About TEKsystems
We're partners in transformation. We help clients activate ideas and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across North America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and real-world application, we work with progressive leaders to drive change. That's the power of true partnership. TEKsystems is an Allegis Group company.
The company is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
About TEKsystems And TEKsystems Global Services
Were a leading provider of business and technology services. We accelerate business transformation for our customers. Our expertise in strategy, de