About the Role
LLMOps Engineer (Large Language Models Operations Engineer) Location: Remote
Duration: 11-12 months
Rate: DOE
Key Responsibilities:
Provide operational support for Large Language Models (LLM), including incident management, ongoing model management and maintenance, and user support. Monitor and maintain GenAI solutions, including performance monitoring, user interaction monitoring, and efficacy monitoring. Implement and uphold LLMOPS guardrails, ensuring adherence to best practices and implementation standards. Conduct bias audits and other guardrail checks for implemented GenAI solutions. Support code management, including proactive code optimization beyond basic break-fixes. Core Skills Required:
Strong experience in developing, deploying, and managing large language models, specifically on Microsoft Azure. Proven expertise in Azure cloud platforms and services, including Azure Machine Learning, Azure Kubernetes Service, and Azure DevOps. Solid understanding of LLMOPS guardrails implementation practices. Proficiency in Python for scripting and automation. Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders. Job Duties:
LLM Operations Support: Manage incidents related to LLM, ensuring timely resolution and minimal impact on operations. Continuously manage and maintain LLM models to ensure optimal performance and reliability. Provide user support, including troubleshooting and resolving user inquiries related to LLM. GenAI Solutions Monitoring and Maintenance: Monitor performance metrics of GenAI solutions and take proactive measures to optimize performance. Monitor user interactions with GenAI solutions to identify potential improvements or issues. Monitor the efficacy of GenAI solutions, ensuring they meet predefined performance benchmarks. Guardrails Implementation and Monitoring: Implement and enforce guardrails for LLMOPS to maintain operational integrity and compliance. Conduct bias audits and other guardrail checks to ensure ethical and responsible use of AI solutions. Code Management Support: Collaborate with development teams to optimize code beyond basic break-fixes, focusing on proactive improvements and efficiencies. Additional Requirements:
Experience with Terraform for infrastructure deployment and management is essential. Familiarity with CI/CD pipelines and version control systems. Ability to work independently and as part of a remote team, demonstrating strong initiative and problem-solving skills.
