From Black Box to Glass Box: Making AI Agents Observable (and Less Mysterious)

As AI agent systems move from POC to production, observability becomes both more critical and more complex. Unlike traditional distributed services, AI agents reason probabilistically, invoke tools dynamically, collaborate with other agents, and evolve behavior based on context and memory. This creates new observability challenges: non-deterministic execution paths, opaque model decisions, emergent failures across multi-step plans, and difficulty correlating user intent with downstream actions and outcomes. In this session, we will explore how Microsoft Agent Framework and Microsoft foundry enable robust observability by exploring the code in an agent workflow. We will start by reviewing pillars observability in Microsoft Agent Framework, we will then look at the abstractions available for agent lifecycle management, tool invocation, memory, and more. Finally, we will look at how the integration with Microsoft Foundry provides unified telemetry, distributed tracing and evaluation pipelines that allow teams to detect failures, analyze agent behavior, and continuously improve reliability and performance. By the end of this talk, attendees will have a clear understanding of how agent observability is different from traditional application monitoring and how to follow good practices when implementing agentic workflows with Microsoft Agent Framework and Microsoft Foundry to improve agent reliability, ensure responsible AI practices, and accelerate the path from prototype to production.