CoPhilot
An example of a simple agent architecture pattern
Nico Sprotti
November 2024
Download Document
Purpose:
Describe the architecture of a simple bot in Teams where users can chat with open-source models Llama 3.1 and Phi 3, leveraging only GA features as of Nov 2025. The goal for the reader is to learn by building a useful and re-usable Power Platform solution while hitting on some core concepts of the Power Platform. The solution is simple and includes 2 tables, 1 app, 2 custom topics and 3 Power Automate flows.
A recorded demonstration of this document can be found HERE.
Interview of this solution is on LinkedIn: Leveraging Copilot Studio to chat with Phi 3 and Llama 3.1 in Microsoft Teams
Deploying Llama 3.1 and Phi 3 in Azure AI Studio
Azure AI Studio is pro code. Azure AI Studio is simple.
Azure AI Studio offers thousands of AI models, including frontier AI models available the “day of” the release. Many can be deployed as Model as a Service. For our example we deploy Llama 3.1 and Phi 3, which are leading open-source models in their own category.
Setting up the solution
The solution in Power Platform is an important concept. It is more than a folder. It is structuring for the future ALM process. When well architected and governed, solutions allow us to decouple complexity and iterate faster.
A common pattern is the IT team building pro code custom connectors into legacy apps empowering their respective business partners (Power Platform coders or ‘makers’) to use such connectors in their solutions.
Defining the environment variables
Environment variables are defined at the solution level. They are usually variables that change during the ALM process but do not influence the business logic.
For our example all we need are four environments’ variables, Llama 3.1 end point and API key and Phi 3 end point and API key.
Creating the data model
The data model is often the hardest part of an elegant solution.
In our example we are using Dataverse to store, view, log, test and benchmark the conversations.
Out of the data model we built Model Driven apps, that are driven by the data model.
Those types of apps are very fast to build and iterate on. Always start there to prototype, we can always switch to a different database later should storage cost become a concern.
I find the hard part of data modeling is knowing which data to bring and where, so we don’t end up duplicating data across systems. It requires domain expertise to bring just the data we need to the right place to just solve our problem. In spirit, the less data we must bring, the less variables we must create, the better. I iterate my data models. During the prototyping phase, Power Platform is so fast that at times it is faster to restart from scratch vs iterating.
Model Driven Apps offer a robust ALM process. Model Driven apps have low TCO and scale securely fast.
All Microsoft Dynamics 365 CRM business applications suite are shipped as first party model driven apps. Dynamics 365 is a $6B Microsoft business unit and growing.
Dataverse encompasses a decade of engineering experience from Microsoft Business Application product team. With Dataverse, we are sharing with everyone, Microsoft business application development platform.
Dataverse is all about how to securely scale fast while optimizing the engineering cost of my business application. At Microsoft we like to say we aim to do more with less, Dataverse is a great example.
In its simplest form, I like to think of Dataverse as Excel + Access + Security + Cloud + AI + Git + Governance +++
When we create a table, Dataverse automatically create the core fields needed in all good business application data models (Create on, Created by, Modified on, Modified by …). Pay attention to the ‘name’ and ‘Id’ columns. The tables below have been filtered to show only the added custom columns. When we create a table, Dataverse automatically create the core fields needed in all good business application data models (Create on, Created by, Modified on, Modified by …). Pay attention to the ‘name’ and ‘Id’ columns.
The tables below have been filtered to show only the added custom columns.
The Conversation Table
Conversation history: Contains the message table in an almost JSON ready to be parsed and sent to the model (just missing the system prompt). This column is a performance trick, it is not to be displayed to the user. The trick is when we call the model we don’t have to loop over all the records in the message table for that conversation.
Make sure to create it as a multi-line text and expand the amount of text you can store in there. Same applies wherever we store long text.
Chat history: Contains the actual chat of the conversation in a user readable format.
Feedback / Feeback user comment: Optional, I add these columns to sometime help me capture a chat or an automation record I want to keep for future reference (especially when it fails).
Number of input / output token: Optional. I don’t have it here. The conversation or (automation) table is a great place to add a roll up sum of the token (or cost) per conversation.
Always remember we are prototyping, ready to fail and iterate fast. I usually prototype first with GPT 4o calls everywhere, making it work with a no cost constraint spirit, value first mindset. If our prototype works and ends up having legs, explore cheaper SML options where it makes sense: Example of AI cost optimization with Phi.
The Message Table
When: It is the primary column (usually called ‘name’ when you create the table) being renamed as ‘When’. We will store the date/time/sec in there that we will sort by.
Who system: It is either ‘user’ or ‘assistant’.
Who: The actual name of the user or the assistant
said What: What did the user or the assistant said
System Prompt: For assistant messages, it is the log of the system prompt used to generate the message (as we’ll be able to change the system prompt dynamically during the conversation).
Conversation: It is a lookup column pointing to our Conversation table (One conversation can have N messages)
Setting up the relationship between the two tables
When creating the lookup field to the Conversation table in the Message table, Dataverse automatically creates a relationship between the two tables.
Make sure it is a Parental type of behavior relationship.
