T

Machine Learning Engineer - LLM Evaluator - Hybrid Opportunity

TEKsystems · Anywhere

Full-timeLeadPython

🔥14 people viewed this job

About the Role

Description Hybrid role requiring onsite attendance in Cupertino, CA three (3) days per week. Contract opportunity. This role focuses on AI/LLM quality evaluation and measurement, not traditional model development. Candidates should have experience owning or leading evaluation strategies for production AI/LLM systems. We are seeking an experienced Machine Learning Engineer to help define, measure, and improve the quality of AI systems powered by Large Language Models (LLMs). In this role, you will design and scale evaluation frameworks that ensure AI-generated outputs are accurate, reliable, and consistent across a variety of use cases. You will partner closely with Product, Engineering, Research, and Data teams to establish evaluation methodologies, automate quality assessment workflows, and develop the metrics that drive model improvements. This is an ideal opportunity for someone who enjoys solving complex problems at the intersection of AI quality, evaluation strategy, automation, and data-driven decision making. Top Skills AI/LLM Evaluation & Quality MeasurementProblem Solving & System DesignCommunication & OwnershipPython, SQL & Automation Top Responsibilities Design and build LLM evaluation frameworks, including automated scoring systems, rubric-based grading, and LLM-as-a-Judge solutions.Define and track evaluation metrics, KPIs, and dashboards that measure model quality and performance at scale.Perform error analysis, root-cause investigations, and quality deep dives to identify opportunities for model improvement.Partner with cross-functional teams to integrate evaluation standards, workflows, and quality requirements into production environments. What We're Looking For Experience designing and maintaining LLM evaluation frameworks and measurement methodologies.Hands-on experience with autograders, LLM-as-a-Judge systems, and automated quality validation.Strong prompt engineering skills, including the development of evaluation prompts and grading rubrics.Proficiency with Python, SQL, and large-scale data analysis.Experience creating KPIs, dashboards, and executive-ready reporting on AI quality.Ability to evaluate model performance through error analysis, human-vs-model calibration, and quality benchmarking.Experience working with production AI/LLM systems and large evaluation datasets. What Will Make You Stand Out You've personally owned or led an AI/LLM evaluation strategy.You've built scalable systems to measure AI quality automatically.You can clearly explain how AI quality is defined, measured, and improved.You have experience influencing product and engineering decisions through evaluation insights and data. Experience Level: Expert Job Type & Location This is a Contract position based out of Cupertino, CA. Pay And Benefits The pay range for this position is $55.00 - $65.00/hr. Individual compensation offered for this position within this range will depend on many factors, including qualifications, skills, relevant experience, job knowledge, geographic location, internal equity, and other pertinent job-related factors. Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave) Job Type & Location This is a Contract position based out of Cupertino, CA. Application Deadline This position is anticipated to close on Nov 20, 2026. Workplace Type This is a fully remote position. About TEKsystems We're partners in transformation. We help clients activate ideas and solutions to take advantage of a new world of opportunity. We are a team of 80,000 strong, working with over 6,000 clients, including 80% of the Fortune 500, across North America, Europe and Asia. As an industry leader in Full-Stack Technology Services, Talent Services, and real-world application, we work with progressive leaders to drive change. That's the power of true partnership. TEKsystems is an Allegis Group company. The company is an equal opportunity employer and will consider all applications without regards to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law. About TEKsystems And TEKsystems Global Services Were a leading provider of business and technology services. We accelerate business transformation for our customers. Our expertise in strategy, de

Allegis Group, Inc. is a multinational talent management firm headquartered in Hanover, Maryland, United States. As of 2018, it had US$13.4 billion in revenue, and 19,000 employees. It ranks fourth in the world after Adecco, Randstad and ManpowerGroup.

💬 Developer Questions

Ask the team a question — answers show up here

🎯

What does the interview process look like?

🤖

What AI/vibe coding tools does the team use daily?

👥

How big is the engineering team?

⏰

Is the team fully async or are there required meetings?

🚀

What does onboarding look like for remote hires?

🔧

Can you share more about the tech stack and architecture?

📈

What does career growth look like in this role?

📅

What does a typical day look like?

💰

Is there a salary range you can share?

📊

Is equity or stock options part of the package?

🌍

Are there timezone requirements or preferences?

🛂

Do you sponsor work visas?

🏢 Is this your listing? Claim it to answer questions

Similar Jobs

Helpful resources

Hiring for a similar role? Post your job here — it's free →