Staff AI Engineer

Grafana Labs is a remote-first, open-source powerhouse. There are more than 20M users of Grafana, the open source visualization tool, around the globe, monitoring everything from beehives to climate change in the Alps. The instantly recognizable dashboards have been spotted everywhere from a NASA launch and Minecraft HQ to Wimbledon and the Tour de France. Grafana Labs also helps more than 3,000 companies -- including Bloomberg, JPMorgan Chase, and eBay -- manage their observability strategies with the Grafana LGTM Stack, which can be run fully managed with Grafana Cloud or self-managed with the Grafana Enterprise Stack, both featuring scalable metrics (Grafana Mimir), logs (Grafana Loki), and traces (Grafana Tempo). We're scaling fast and staying true to what makes us different: an open-source legacy, a global collaborative culture, and a passion for meaningful work. Our team thrives in an innovation-driven environment where transparency, autonomy, and trust fuel everything we do. You may not meet every requirement, and that's okay. If this role excites you, we'd love you to raise your hand for what could be a truly career-defining opportunity. This is a remote opportunity and we are looking for candidates from Canada. Residents of Quebec are not eligible for this role. The Opportunity Grafana Labs is seeking a Staff Engineer (AI & Automation) to own the AI agent infrastructure and automation platform that powers our Marketing Operations organization. You'll build multi-agent architectures, LLM integrations, and backend services that connect AI models to internal and third-party data platforms. You'll ship production systems that teams depend on daily. This is a high-autonomy role where you own the technical direction. You'll identify the highest-leverage problems across Marketing, RevOps, and SDR teams, design the solutions, and ship them. You'll define the technical direction for the automation platform (data models, API contracts, shared libraries, reference architectures) and partner with Data Engineering, GTM Systems, and Field Operations to build scalable, self-service automation that eliminates manual work and drives operational efficiency. What You'll Be Doing Agentic Systems & AI Infrastructure • Own end-to-end development of multi-agent AI systems, from architecture and implementation through testing, deployment, and ongoing operation • Build modular, composable agentic systems using orchestration frameworks (LangChain, CrewAI, Anthropic MCP, or similar) that operate 24/7 across teams • Develop reusable agentic skills that agents invoke across interfaces (Slack, dashboards, internal apps, CLIs) • Implement observability and feedback loops including logging, performance metrics, prompt iteration, model evaluation, and cost management • Establish governance and compliance standards for AI workflows including access controls, audit trails, PII handling, and human-in-the-loop escalation paths Systems Integration & Backend Services • Build MCP servers, APIs, CLIs, and microservices connecting AI models to business systems (BigQuery, Slack, CRMs, email, calendars, analytics tools) • Architect data flows for retrieval-augmented generation (RAG), connecting LLMs to internal knowledge bases, customer data, and real-time business context • Build serverless or containerized services (GCP Cloud Functions, Cloud Run) that scale with usage and integrate with Grafana's cloud infrastructure Automation & Workflow Enablement • Partner with RevOps, Demand Generation, Regional Marketing, and SDR teams to scope high-impact automation problems, identify bottlenecks, and build solutions with measurable business outcomes • Design and deploy workflows using orchestration tools (n8n, Workato, or custom platforms) with CI/CD, testing, and production reliability standards • Build systems designed for self-service with documentation, playbooks, and enablement materials that let partner teams operate independently We invest heavily in developer productivity. You'll have access to AI coding assistants (Claude Code, Gemini CLI, OpenAI Codex, and others of your choice within security guidelines). We encourage pragmatic AI-assisted development paired with strong code review and quality standards. What Makes You a Great Fit • 8+ years of software engineering experience with depth in backend development, systems integration, or data/analytics engineering • 2+ years hands-on experience applying LLMs/AI to production workflows, not just prototypes • Strong proficiency in Python and JavaScript/Node.js with Git-based workflows, code review practices, and testing discipline • Hands-on experience with LLM frameworks and patterns including prompt engineering, RAG, function calling/tool use, structured output parsing, and evaluation • Experience building and operating multi-agent systems at scale including agent decomposition, orchestration patterns (sequential chains, router/dispatcher, parallel fan-out), state