About the Role
Other Locations - New York / New Jersey, Austin,San FranciscoAbout the RoleWe are looking for an AI Engineer with strong hands-on experience in prompt engineering, LLM application development, and agentic AI. You will design and build reliable, context-aware AI solutions that understand user intent, retrieve relevant information, and execute tasks securely across enterprise systems.The ideal candidate combines strong software engineering skills with a practical understanding of LLM behavior, evaluation, and sensitive-data handling, particularly in healthcare environments.
Key Responsibilities
• Prompt engineering and optimization: Design, test, and refine system prompts, prompt templates, and few-shot examples to improve response accuracy, relevance, consistency, and instruction following.
• Agentic AI development: Build AI agents that support planning, tool use, memory, and multi-step task execution, with appropriate controls for failures and human intervention.
• MCP integration: Develop and integrate Model Context Protocol (MCP) servers and tools to enable secure interaction with enterprise APIs, applications, and data sources.
• Agent architecture: Design tool-calling workflows, retrieval mechanisms, session memory, and context management strategies.
• RAG and knowledge grounding: Implement retrieval-augmented generation workflows, including document processing, embeddings, semantic search, and grounded responses.
• Evaluation and testing: Create evaluation datasets and automated tests to measure response quality, retrieval accuracy, task completion, tool selection, and agent reliability.
• Security and responsible AI: Implement safeguards against prompt injection, sensitive-data exposure, hallucinations, and unauthorized tool execution.
• Production operations: Deploy, monitor, troubleshoot, and improve AI applications, balancing quality, latency, reliability, and cost.
• Cross-functional collaboration: Work with product managers, architects, and engineering teams to translate business requirements into practical AI solutions.
• Technical documentation: Contribute to architecture reviews, design documentation, and engineering standards.Required Skills and Experience
• Hands-on experience building AI applications using LLMs such as GPT, Claude, Gemini, Llama, or comparable models.
• Strong prompt engineering skills, including system instructions, structured outputs, few-shot prompting, and iterative response optimization.
• Proficiency in Python, JavaScript/TypeScript, or Java, with experience integrating APIs and backend services.
• Practical experience with RAG, embeddings, vector databases, semantic search, and enterprise knowledge retrieval.
• Experience designing agent workflows, tool-calling integrations, and context management.
• Ability to evaluate and improve AI responses for accuracy, relevance, completeness, tone, and adherence to instructions.
• Understanding of hallucination mitigation, prompt injection risks, guardrails, and responsible AI practices.
• Knowledge of secure healthcare-data handling, including HIPAA-aligned PHI/PII practices, data minimization, de-identification, masking, access controls, and audit logging.
• Strong debugging, problem-solving, and communication skills.Preferred Qualifications
• Experience developing MCP servers and integrating MCP-enabled tools.
• Experience integrating AI applications with healthcare platforms, electronic health records, or other regulated enterprise systems.
• Experience deploying and operating AI applications on cloud platforms.
• Familiarity with LLM evaluation, observability, and automated regression testing tools.
• Experience implementing human review, approval workflows, and recovery mechanisms for AI agents.