Building AI Agents on Azure: An Engineering Checklist for .NET Teams

Building AI Agents on Azure with .NET
Gartner predicted in June 2025 that more than 40% of agentic AI projects would be canceled by the end of 2027. The reasons include escalating costs, unclear business value and inadequate risk controls.
An Engineering Checklist for .NET Teams
For .NET teams building agents on Azure, the risks mentioned above often emerge during architecture and implementation: an agent receives too much access, retrieves information from the wrong source, executes an action without sufficient controls or reaches production without an operating owner.
|
A production deployment requires decisions across six areas: scope, identity, grounding, guardrails, observability and ownership. Azure provides the underlying services, while the engineering team defines how they work together. 1 Scope2 Identity3 Grounding4 Guardrails5 Observability6 Ownership |
|
Related reading The companion article on why enterprise AI projects fail before the model goes live examines the organizational and technical problems that often appear earlier in the project. |
The 2026 Stack: Microsoft Agent Framework for .NET and Foundry Agent Service
Microsoft Agent Framework is the successor to Semantic Kernel and AutoGen. Its 1.0 release in April 2026 introduced stable APIs for building agents and multi-agent workflows.
|
Framework Microsoft Agent Framework for .NET |
|
1.0 · stable APIs.NET · PythonGo: previewFoundry observability & evaluation: preview Microsoft Agent Framework for .NET provides a stable agent abstraction, graph-based workflows with checkpointing, orchestration patterns with human-in-the-loop approvals, MCP support for tool access and migration assistants for both predecessor frameworks. Microsoft has stated a long-term support commitment for version 1.0. The framework supports Python alongside .NET, while Go remains in public preview. At the 1.0 release, integrations for Foundry observability and evaluation were still in preview. |
|
Runtime Foundry Agent Service |
|
|
Foundry Agent Service provides a managed runtime for agent applications. It supports two agent types: |
|
|
Prompt agents Defined through configuration: instructions, a model and tools. They do not require the team to maintain a separate agent application. Hosted agents Run the team's own code, built with Agent Framework or another framework, behind a managed endpoint. Each hosted agent receives a dedicated Microsoft Entra identity. |
|
|
Foundry also snapshots agent versions as they change and publishes agents to a stable endpoint. |
|
Running Semantic Kernel? Teams maintaining Semantic Kernel applications should assess the migration path to Microsoft Agent Framework. Microsoft's migration guide and migration assistants can analyze existing code and generate a step-by-step plan, giving teams a basis for estimating the work and scheduling the transition. |
AI Agent Production Checklist: Six Areas
A production agent needs engineering decisions in six areas:
|
1
Scope Prove the task needs an agent 2
Identity Managed identity instead of keys 3
Grounding RAG with data boundaries 4
Guardrails Prompt Shields and tool limits 5
Observability Trace and evaluate before launch 6
Ownership Responsibility after launch Scope → Identity → Grounding → Guardrails → Observability → Ownership |
Each area addresses a different operational risk. The following sections cover the decisions and Azure capabilities involved.
1. Scope: Prove the Task Needs an Agent
|
“If you can write a function to handle the task, do that instead of using an AI agent.” — Microsoft Agent Framework overview, Microsoft Learn |
Microsoft's Agent Framework guidance sets a practical starting point: when a function can handle the task, write the function. Gartner analyst Anushree Verma makes a similar observation about the market, noting that many use cases currently described as agentic do not require an agent-based implementation.
Microsoft's comparison distinguishes open-ended or conversational tasks and autonomous tool use from well-defined steps with an explicit execution order, which are better suited to workflows.
Agent
Open-ended or conversational tasks, autonomous tool use.
Workflow
Well-defined steps with an explicit execution order.
Before choosing an agent, establish four things:
| A bounded task with one owner and one user group. | |
| A defined set of tools, each with a documented purpose. | |
| A measurable success criterion agreed before development. | |
| A human approval step for every action that changes data. |
|
Human-in-the-loop Agent Framework supports human-in-the-loop approval patterns. The team still needs to decide which actions require approval and where that checkpoint belongs in the workflow. |
2. Identity: Use Managed Identity Instead of Keys
|
An agent that accesses Azure services through managed identity avoids storing an application-held API key. Its role assignments determine which resources it can reach. |
Microsoft's keyless connection guidance describes the authentication mechanism. DefaultAzureCredential from the Azure Identity library uses the developer's sign-in during local development and a managed identity when the application runs in Azure. This authentication flow removes the need to store an application secret in code or configuration.
Local development
Developer sign-in
Running in Azure
Managed identity
The following example demonstrates keyless authentication for a direct Azure OpenAI chat-completion call. It does not create an agent, orchestrate tools or configure Foundry Agent Service.
using Azure.AI.OpenAI;
using Azure.Identity;
using OpenAI.Chat;
var endpoint = new Uri(
Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!);
AzureOpenAIClient client = new(
endpoint,
new DefaultAzureCredential());
ChatClient chat = client.GetChatClient("");
ChatCompletion reply = await chat.CompleteChatAsync(
"Summarize the open support case.");
Role assignments determine the resources and operations available to the application. Microsoft documents the Cognitive Services OpenAI User role for chat completions and the Contributor role for Assistants scenarios. Assign the least privilege required and scope access to the specific resource or resource group.
| Role | Scenario |
|---|---|
| Cognitive Services OpenAI User | Chat completions |
| Contributor | Assistants scenarios |
| Assign the least privilege required, scoped to the specific resource or resource group. | |
|
User-assigned identity For a user-assigned managed identity, set |
Each hosted agent in Foundry receives a dedicated Microsoft Entra identity. Prompt agents also support an identity. Microsoft Entra Agent ID provides identity governance for agents and is generally available. Microsoft requires at least one sponsor for each agent identity. Audit trails and Conditional Access policies depend on these identity decisions. The article on securing enterprise AI agents covers these controls in more detail.
3. Grounding: Use RAG with Azure AI Search and Enforce Data Boundaries
An agent's answers depend on the information retrieved for each request. The retrieval layer therefore affects answer quality and determines which data can enter the agent's context.
|
Softwarium case study Softwarium engineers built customer support and internal knowledge chatbots on Azure OpenAI Service and Azure AI Search for a cybersecurity software company. Metadata filters kept internal documents out of customer-facing answers. |
The case study describes two retrieval-augmented generation chatbots. Their source data included PDFs, Word documents, HTML pages, plain text, images, audio and video. The solution converted the content into vector embeddings stored in Azure AI Search. Hybrid search combined vector and keyword matching, while metadata tagging and filtering separated internal datasets from external ones.
|
Retrieval pipeline in the case study 01 Sources PDF, Word, HTML, plain text, images, audio, video 02 Embeddings Content converted into vector embeddings 03 Azure AI Search Hybrid search: vector + keyword 04 Metadata filter Internal and external datasets separated Internal dataset Internal knowledge chatbot External dataset only Customer support chatbot |
Three implementation decisions help maintain those boundaries:
01
Tag content during ingestion
Attach metadata identifying the dataset, audience and owner to each chunk. The retrieval layer needs these attributes to apply access filters.
02
Apply filters in the search request
Exclude restricted documents before retrieved content reaches the prompt. Instructions in a prompt alone cannot enforce a data access boundary.
03
Test with the lowest-privilege account
Run adversarial queries through the least privileged entry point. In the Softwarium case study, that means testing the customer-facing chatbot to confirm that internal documents remain inaccessible.
Agents that retrieve Microsoft 365 content also depend on the tenant's permission model. The article on preparing Microsoft 365 data for Copilot covers the access review and remediation work involved.
4. Guardrails: Configure Prompt Shields and Restrict Tool Access
Prompt Shields in Microsoft Foundry address two types of attacks.
Direct
User prompt attacks
User prompt attacks attempt to override system instructions or manipulate the model through malicious input. Foundry scans for these attacks at the user-input intervention point.
Scanned at:
User input
Indirect prompt injection
Document attacks
Malicious instructions can be embedded in third-party content such as documents, emails and web pages. For model API calls, Foundry scans document content at the user-input intervention point. Foundry Agent Service can also scan document attacks at the tool-response intervention point for supported tools. This tool-response capability is in preview.
Scanned at:
User inputTool response: preview
Coverage depends on the configured intervention points and supported integrations. Teams should verify which inputs and tool responses are actually scanned in their implementation.
|
Check the assignment Prompt Shields only work when the guardrail is assigned to the agent or deployment. Check the configuration if expected attacks go undetected. Foundry content filters also address cross-prompt injection. Annotate mode records detections without blocking them, which helps teams evaluate false positives before moving selected categories to block. Detection alone does not guarantee that an attack is prevented. |
Tool access needs equally careful limits. Every tool an agent can call creates an action surface. Start with three restrictions:
| Use read-only tools for lookups and retrieval. | |
| Put write operations behind a human approval step. | |
| Assign scoped credentials to each tool, using Microsoft Entra authentication where the connection supports it. |
Foundry toolboxes group tools behind a managed endpoint with centralized authentication, governance and versioning. This gives teams a consistent way to review and manage the tools available across agents.
5. Observability: Trace and Evaluate Before Launch
Foundry observability combines evaluation, monitoring and tracing. Agent tracing through OpenTelemetry captures model calls, tool invocations and agent decisions. It integrates with Application Insights and supports Microsoft Agent Framework. Production monitoring dashboards track token consumption, latency and error rates.
|
Agent tracing · OpenTelemetry Model callsTool invocationsAgent decisions Production dashboards Token consumptionLatencyError rates |
Before launch
Evaluate and red-team
Before launch, evaluate the agent against a dataset maintained by the team. Built-in agent evaluators include tool-call accuracy and task completion. RAG evaluators include groundedness. Microsoft's AI red teaming agent, built on its PyRIT framework, simulates attacks before deployment. Microsoft recommends combining automated red teaming with human review.
After launch
Keep evaluating in production
After deployment, teams can use several evaluation methods:
– Continuous evaluation scores a sampled share of production traffic.
– Scheduled evaluation against test datasets helps detect changes in behavior.
– Azure Monitor alerts can notify the team when outputs fail defined quality thresholds.
|
Evaluation cost Evaluation also has a cost. Evaluations in the agents playground are enabled by default for Foundry projects and billed according to consumption. Teams that do not need them in the playground can turn them off by unselecting all evaluators. |
|
Related reading The article on testing AI agents before they touch production data covers evaluation datasets, test scenarios and pre-production checks in greater detail. |
6. Ownership: Assign Responsibility After Launch
A production agent needs an operating model covering releases, people, incidents and model changes.
Releases
Versioned releases
Foundry snapshots agent versions and publishes them to a stable endpoint. Each release should have an identifiable version and a rollback target.
People
Named people
Microsoft Entra Agent ID requires a sponsor for each agent identity, while owners are optional. Teams should assign an operational owner as well as a sponsor, and identify a successor for each role.
Incidents
On-call coverage
Alerts for token consumption, latency and errors need to reach a team responsible for investigating and resolving incidents.
Model changes
Model-change control
Foundry allows teams to change models without changing agent code. A model change should still follow the release process, including evaluation against the team's test dataset before deployment.
|
Related reading The owner also needs visibility into operating costs. The article on the hidden cost of enterprise AI examines inference, integration and operational expenses. |
When the Checklist Requires Additional Engineering Capacity
.NET teams often need additional engineering support when agents must integrate with legacy applications, existing Semantic Kernel implementations require migration, evaluation datasets are missing or no team has been assigned to operate the agent.
Agents must integrate with legacy applications
Existing Semantic Kernel implementations require migration
Evaluation datasets are missing
No team has been assigned to operate the agent
These gaps involve work across several systems. Integrating an agent with a legacy .NET application requires access to existing services and data, along with controls for the actions the agent can perform. Migration work requires an assessment of current framework usage and dependencies. Evaluation requires representative datasets and defined acceptance criteria. Production ownership requires monitoring, incident response and release procedures.
Gartner's press release identifies integration with legacy systems as one of the technical challenges that can disrupt agentic AI projects.
Teams addressing these requirements can hire .NET developers to support migration, integration and evaluation work. Softwarium's Azure Certified Engineers work with Azure OpenAI Service, Azure AI Search and Microsoft Entra identity.
|
Book a 30-minute call with Anna Moskalets, Head of Digital Enhancement, to review an agent prototype against the six areas and identify the engineering gaps. |
FAQ
|
Q. Should .NET teams use Semantic Kernel or Microsoft Agent Framework? Microsoft describes Agent Framework as the successor to Semantic Kernel and AutoGen. Version 1.0 shipped in April 2026 with stable APIs. Teams maintaining existing Semantic Kernel applications should assess the migration guide and compare the work with their architecture and support requirements. |
|
Q. Do AI agents on Azure need API keys? No. Microsoft supports keyless connections through Microsoft Entra ID and managed identity. With the appropriate configuration, applications can authenticate without storing an API key or connection string in code or configuration. |
|
Q. How do agents on Azure defend against prompt injection in documents? Prompt Shields can scan user input for prompt attacks and document content for indirect prompt injection. For model API calls, document content is scanned at the user-input intervention point. Foundry Agent Service also supports scanning tool responses for supported tools through a preview capability. Coverage depends on the configured intervention points and integration, and the guardrail must be assigned to the agent or deployment. |
|
Q. When should a team avoid building an AI agent? When a conventional function or deterministic workflow can handle the task, Microsoft's Agent Framework guidance recommends using that approach instead. Agents are more appropriate for open-ended tasks that require reasoning, tool selection or autonomous interaction. |


