AI Engineer

Career Area Technology, Digital and Data Job Description Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar, you're joining a global team who cares not just about the work we do – but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here – we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it. As an AI Engineer, you will design, develop, and optimize advanced AI interactions to power production-grade software applications. You will move beyond basic prompt drafting to build robust, scalable, and reliable AI systems utilizing state-of-the-art Large Language Models (LLMs). Your role is critical in orchestrating autonomous AI agents, implementing Retrieval-Augmented Generation (RAG) pipelines, and establishing rigorous testing and observability frameworks. You will bridge the gap between complex business logic and machine intelligence, ensuring high-quality, safe, and deterministic AI behaviors. What You Will Do Prompt Architecture & Tuning • Design Advanced Prompts: Architect, version, and refine highly effective system, contextual, and dynamic prompts using advanced techniques (e.g., Chain-of-Thought, Tree-of-Thoughts, ReAct, Few-Shot learning). • Mitigate Risks: Minimize model hallucinations, jailbreaking, and prompt injections through programmatic guardrails and robust adversarial prompting strategies. • Model Agnostic Tuning: Adapt, benchmark, and fine-tune prompt structures across diverse open-source and proprietary frontier models. Agentic Workflows & RAG Systems • Build AI Agents: Design and orchestrate multi-agent systems and multi-step reasoning workflows using frameworks like LangGraph, LangChain. • Optimize RAG Pipelines: Integrate and fine-tune Retrieval-Augmented Generation architectures, managing semantic search, chunking strategies, embedding models, and vector database queries. • Tool Execution: Implement precise function-calling and tool-use prompts enabling LLMs to interact reliably with external APIs, databases, and legacy codebases. Testing, Evaluation & Observability • Build automated testing suites using RAGAS and Evals frameworks to assess RAG accuracy and enable continuous validation and benchmarking of Retrieval‑Augmented Generation outputs across datasets and Models . • Agent Testing: Build automated regression testing suites and CI/CD pipelines to validate prompt stability, agent trajectory correctness, and tool-calling accuracy. • Implement LLM Observability: Set up production monitoring and observability systems using platforms like Azure Application insights, Microsoft Foundry. • Monitor Metrics: Track, analyze, and optimize production runtime metrics including semantic drift, token usage, cost efficiencies, and end-to-end latency. Collaboration & Continuous Innovation • Cross-Functional Delivery: Work alongside Product Managers, Data Scientists, and Software Engineers to translate commercial requirements into deterministic AI capabilities. • Technology Scouting: Keep pace with rapid weekly advancements in LLM capabilities, evaluation frameworks, and orchestration libraries to sustain a competitive edge. What You Have • Core Prompt Engineering: Expert-level mastery of structured prompt formats (XML, JSON wrappers), system-level steering, and context-window optimization. • Production Programming: High proficiency in Python or TypeScript for constructing backend AI logic, processing asynchronous data streams, and handling API integrations. • Orchestration Frameworks: Hands-on experience building apps using LangChain, LangGraph, native SDKs. • Vector Infrastructure: Solid experience working with vector databases (e.g., Azure AI Search, AWS Open Search) and hybrid search retrieval methods. • Data & ML Fundamentals: Strong capabilities in data manipulation (Pandas, NumPy), data cleaning, token handling, and foundational understanding of NLP transformers. Top Candidates Will Have • AI Observability Stacks: Prior implementation of Application Insights, OpenLLMetry, or custom evaluation/guardrail layers. • Machine Learning Systems: Knowledge of deep learning architectures, fine-tuning frameworks (QLoRA, PEFT), and platforms like Hugging Face, PyTorch, or TensorFlow. • Cloud Infrastructure: Experience deploying containerized AI microservices using Docker, Kubernetes, AWS, or GCP. • Education: Bachelor's or master's degree in computer science, Data Science, Artificial Intelligence, Computational Linguistics, Mathematics, or a highly technical discipline. • Experience: 2+ years of dedicated professional experience developing production software integrated with GenAI/LLM pipelines. • Track Record: Proven portfolio demonstrating successfully deployed RAG systems or operational AI agents in commercial environments. Additional Info