A while ago, I developed an agentic solution using what is now ‘classic Foundry’. At the time, I chose to leverage connected agents, a pattern that gave me two critical capabilities:
- Response ownership: ensuring that a single agent would ‘own’ the final response creation.
- Conditional routing: dynamically selecting the best suited ‘specialist’ agent based on the user request.
This combination allowed me to build a system that was both flexible in terms of which agent would augment the response with its specialty but also structured in which agent and how the final response would be generated.
However, when researching the migration to the ‘new Foundry’, I quickly realized that this pattern no longer had a direct equivalent. The absence of connected agents meant the orchestration was no longer implicit, but an explicit concern for the solution being built.
At that point, some options were considered. But first let’s talk about the tool that would sit at the core of this migration.
Microsoft Agent Framework
As mentioned earlier, orchestration had now become an explicit concern of the solution.
While Foundry was already chosen as the runtime for its built-in capabilities that make agents ready for production (article talking about that here), it does not solve, by itself, the orchestration limitation.
I still needed an orchestration layer that would seamlessly integrate with it. The answer? Microsoft Agent Framework.
Why Microsoft Agent Framework?
Microsoft Agent Framework (or MAF, as we’ll refer to it from this point forward) provides built-in interoperability with the Foundry Agent Service.
This integration is enabled through the ‘FoundryAgent’ abstraction, which extends the broader ‘AIAgent’ abstraction. As a result, Foundry-based agents can be seamlessly integrated into MAF workflows, allowing developers to leverage the full orchestration capabilities of MAF while still benefiting from the production-grade runtime provided by Foundry.
In practice, this means we are not forced to choose between orchestration flexibility and runtime robustness. We can combine both within a single, cohesive architecture.
At this point, the architectural direction started to become clearer. However, the challenges were still only beginning.
What solved the problem: MAF Graph-based workflow
After evaluating the available orchestration models, it became clear that none of the ‘out-of-the-box’ approaches fully reproduced the behavior that connected agents had previously provided.
The solution was to stop thinking in terms of agent-to-agent conversation and instead model the entire interaction via a custom-designed workflow.
Rather than allowing agents to decide who should execute next, or relying on a static predefined sequence, the workflow itself became responsible for orchestration. Execution paths, routing decisions, and response ownership were explicitly defined within the workflow code, making the overall behavior both predictable and transparent.
This approach successfully recreated the two capabilities that originally motivated the use of connected agents, while also introducing additional benefits:
- Response ownership: only one dedicated agent produces the final response, defined in the workflow design.
- Conditional routing: router agent produces typed response with route decision and confidence score. This typed response is used in code to deterministically route to specialist agents.
- Fine-grained execution control: workflow execution can be influenced through custom Executors, enabling deterministic behavior at each step (pre and post inference).
- Extensibility: new specialist agents can be introduced with minimal impact on the existing workflow.
In other words, the intelligence of deciding which capability is required remained AI-driven while how and when became deterministic.
How the problem was solved
While the high-level architecture appears relatively straightforward, several implementation details were necessary to ensure the workflow behaved consistently and predictably.
The Router Agent
The workflow begins with a dedicated Router Agent. Its responsibility is intentionally limited: analyze the user request and determine which specialist capability is required.
The router does not retrieve data and does not generate responses. Instead, it produces a structured JSON response with the routing decision that the workflow consumes to determine the next execution step.
With the development of the solution, new specialist agents could be introduced with minimal impact on the rest of the system. The router simply gains a new routing option.
Specialist Agents as Capability Providers
Each specialist agent was designed around a single responsibility. Rather than behaving like chatbots, these agents acted more like intelligent services.
Their role was to:
- Execute tool calls.
- Retrieve domain-specific information.
- Perform any required reasoning over that information.
- Produce factual outputs for downstream consumption.
Importantly, they were not responsible for interacting with the user. This design decision eliminated a major issue encountered during the evaluation of handoff orchestration: the possibility of intermediate agents generating user-facing responses.
Instead, specialist agents became implementation details of the workflow rather than visible participants in the conversation.
One important lesson learned: keeping agents thin and focused has a huge impact on accuracy. Less reasoning overhead means less mistakes. With focused agents our instructions also naturally become shorter, keeping input tokens controlled and further improving inference performance.
Maintaining Strict Response Ownership
One of the original requirements from the connected-agent architecture was to preserve a single point of response generation. To maintain this behavior, the workflow always concludes with a dedicated Response Agent.
This agent receives:
- The original user request.
- The outputs produced by the selected specialist agents.
- Any additional workflow context accumulated during execution.
Its sole responsibility is to transform gathered information into a coherent response. Because all user-facing content originates from a single location, formatting instructions, response guidelines, tone requirements only need to be maintained in one place.
This significantly reduced prompt duplication across agents and improved consistency of generated responses.
Why this works better than Connected Agents
Interestingly, the final solution did more than simply replace connected agents. By making orchestration explicit, it became easier to:
- Understand execution flow.
- Debug incorrect routing decisions.
- Introduce new specialist agents.
- Add evaluation and observability capabilities.
- Enforce architectural boundaries between agents.
- Fine-grained control over data shared with inference task through the ‘Executors’ (this is HUGE)
What was initially a migration challenge became an opportunity to move from an implicit orchestration model to an explicit workflow architecture.
The result preserved the flexibility of specialized agents while improving overall performance (more responsive chat experience) and keeping costs mostly the same. With it we achieved greater control, governance, predictability, and maintainability, setting the path for future expansion.
References
Subscribe to our RSS feed
Talk to the author
Contact Julio
Consultant