Remote | Senior Software Engineer – LLM Evaluation (US/Canada/WEU based)

This a Full Remote job, the offer is available from: North America, United States, Canada, New York (USA) We are sharing a specialised part-time consulting opportunity for experienced software engineers to contribute to advanced large language model evaluation, coding benchmark development, and AI-assisted software-engineering research. Selected professionals will curate and evaluate code, develop verification mechanisms, assess AI-generated software across multiple programming languages, and help research teams understand how advanced models perform throughout realistic software-development workflows. Key Responsibilities Code Curation & Solution Development Curate high-quality code examples for model training and benchmarkingDevelop precise solutions to software-engineering tasksCorrect and improve code across multiple programming languagesWork with Python, JavaScript, ReactJS, C/C++, Java, Rust, and GoMaintain strong standards for correctness and maintainability AI-Generated Code Evaluation Evaluate AI-generated code for technical correctnessAssess solutions for efficiency, scalability, and reliabilityIdentify implementation weaknesses and recurring error patternsReview code quality against professional engineering standardsProvide structured rationales supporting evaluation decisions Verification & Automated Assessment Build agents that assess code qualityDesign mechanisms for automatically verifying software solutionsIdentify recurring model-generated coding errorsDevelop reliable checks for engineering tasksSupport reproducible evaluation across repeated assignments Software Engineering Lifecycle Evaluation Evaluate model capabilities across the software-development lifecycleAssess reasoning around prototyping and architecture designReview API design and production implementation decisionsEvaluate launch, experimentation, monitoring, and maintenance scenariosIdentify areas where models struggle with real-world engineering workflows Research & Benchmark Collaboration Collaborate with research and cross-functional technical teamsContribute to datasets used for training and benchmarkingHelp define engineering evaluation strategiesCompare model performance against professional engineering expectationsSupport iterative improvements to coding-focused evaluation systems Ideal Profile 3+ years of professional software-engineering experienceStrong full-stack development capabilitiesExperience building scalable, production-grade softwareStrong understanding of software architecture and system designDeep knowledge of development, debugging, and code-quality assessmentExperience reviewing and improving complex software implementationsProficiency in one or more of Python, JavaScript, Java, C++, Rust, or related languagesReactJS, C, or Go experience may also be relevant to project assignmentsStrong understanding of API design and production implementationFamiliarity with software monitoring and operational maintenanceAbility to reason across the complete software-engineering lifecycleStrong analytical and problem-solving capabilitiesExcellent written and verbal communication skillsAbility to provide clear, structured evaluation rationalesComfortable collaborating remotely with research and technical teams Engagement Details Part-time independent contractor engagementFully remoteCandidates must be based in the United States, Canada, or eligible Western European (WEU) countriesSource examples of WEU locations include Austria, Belgium, France, and GermanyMinimum commitment: 10 hours per weekFlexible workload of up to 40 hours per weekInitial project duration is approximately 1 monthExtension may be available depending on performance and project fitNo medical or paid-leave benefits are included under the contractor arrangementApplication process takes approximately 15–30 minutesCompletion of an AI video interview is requiredCompensation is not specified in the source materialsWork must be completed without using confidential, proprietary, unreleased, employer-restricted, client-restricted, or otherwise protected code, datasets, architecture materials, or technical information belonging to any employer, client, institution, or other third party About the Platform This opportunity is available through 24-MAG LLC. We connect experienced professionals with remote consulting opportunities across technical, evaluation, and project-based workstreams. By submitting this application, you acknowledge that your information may be processed by 24-MAG LLC for recruitment and opportunity matching in accordance with our Privacy Policy: https://www.24-mag.com/privacy-policy This offer from "24-MAG" has been enriched by Jobgether.com and got a 0% flex score.