Remote | Senior Software Engineer – LLM Evaluation

This a Full Remote job, the offer is available from: United States, New York (USA) We are sharing a specialised part-time consulting opportunity for experienced Senior Software Engineers to contribute to advanced large language model evaluation, coding benchmark development, and AI-assisted software-engineering research. Selected professionals will curate and evaluate code, develop verification mechanisms, assess AI-generated software across multiple programming languages, and help researchers understand where advanced models succeed or fail throughout the software-development lifecycle. The work combines senior-level software engineering judgement with structured evaluation and benchmark design. Key Responsibilities Code Curation & Solution Development Curate high-quality code examples for model training and benchmarkingDevelop precise reference solutions to software-engineering tasksCorrect and improve code across multiple programming languagesWork with Python, JavaScript, ReactJS, C/C++, Java, Rust, and GoMaintain strong standards for correctness, clarity, and maintainability AI-Generated Code Evaluation Evaluate AI-generated code for technical correctnessAssess solutions for efficiency, scalability, and reliabilityIdentify implementation weaknesses and recurring error patternsReview code quality against professional engineering standardsProvide structured rationales supporting evaluation decisions Verification & Automated Assessment Design mechanisms that automatically verify software-engineering solutionsBuild agents capable of assessing code qualityDevelop approaches for identifying common model failure patternsCreate deterministic or structured checks where appropriateSupport reliable evaluation across repeated coding tasks Software Engineering Lifecycle Evaluation Evaluate model capabilities across the software-development lifecycleAssess reasoning around prototyping and architecture designReview API design and production implementation decisionsEvaluate launch, experimentation, monitoring, and operational scenariosIdentify where models struggle with real-world engineering workflows Research & Benchmark Collaboration Collaborate with researchers and cross-functional technical teamsContribute to datasets used for training and benchmarkingHelp define engineering evaluation strategies and quality standardsCompare model performance against professional engineering expectationsSupport iterative improvements to coding-focused evaluation systems Ideal Profile Several years of professional software-engineering experienceAt least 2 years of continuous full-time experience at a top-tier product companyStrong full-stack development capabilitiesExperience building scalable, production-grade softwareDeep understanding of software architecture and system designStrong knowledge of debugging and code-quality assessmentExperience reviewing and improving complex production codeProficiency in one or more of Python, JavaScript, C/C++, Java, Rust, or GoReactJS experience is valuable for frontend-oriented assignmentsStrong understanding of API design and production implementationFamiliarity with software monitoring and operational maintenanceAbility to reason across the complete software-engineering lifecycleStrong analytical and problem-solving capabilitiesExcellent written and verbal communication skillsAbility to produce clear and structured evaluation rationalesComfortable collaborating with research and technical teams in a remote environment Engagement Details Part-time independent contractor engagementFully remoteMinimum commitment: 10 hours per weekFlexible workload of up to 40 hours per weekPartial overlap with PST working hours is requiredInitial project duration is approximately 1 monthProject is expected to begin the following weekExtension may be available depending on performance and project fitNo medical or paid-leave benefits are included under the contractor arrangementWork will involve coding evaluation, benchmark development, verification systems, and software-engineering researchAssignments may span Python, JavaScript, ReactJS, C/C++, Java, Rust, Go, and related technologiesWork must be completed without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any employer, client, institution, or other third party About the Platform This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy This offer from "24-MAG" has been enriched by Jobgether.com and got a 0% flex score.