This year has been a fun year for product marketing now with Generative AI on the loose and everyone working on embedding and building new services using Generative AI. One of those vendors that has embraced this to the full extent is Microsoft with all their “Copilots“. When reading the documentation and marketing slides it is not always clear
1: How they work?
2: What kind of data sources do they use?
3: What does it actually cost?
Therefore I wanted to create this blog post to go a bit more into detail into how the different Copilots that Microsoft have and how they actually work, since I am getting a lot of questions about it and the documentation is not always helpful.
I started with creating this visual, that shows the different Copilot services that Microsoft have (currently excluding GitHub Copilot and not all details are known on the other Copilots yet since much is still in preview)
All of these services that are now avaialble as part of the Microsoft ecosystem such as Microsoft 365 Copilot is underneath the covers actually using Azure OpenAI. Azure OpenAI is Microsoft own way of providing the OpenAI Large Language models trough a PaaS service. In addition to this most of the services are handled trough an orchestrator which handles different functions and features depending on what the users wants to do. Such as with Microsoft 365 Copilot where Microsoft has an orchestrator engine that handles different GPT functions depending on Actions.
For instance one example with using PowerPoint with Copilot to create slides. Which involves DALL-E API and GPT API to generate data, topics which are then sent into the Graph API to PowerPoint to create the content.
Many of the features also use a feature called RAG (Retrieval Augmented Generation) which uses an an Search engine to the user, when the users needs to find something or get something it does a search against the data, find relevant content and paste that data to the language model to create a summary. Copilot 365 and Dynamics Copilot uses this approach to make data available for the Copilot.
So how do each work?
Microsoft 365 Copilot has much of the data stored in Microsoft 365 as its data sources, such as Exchange, SharePoint, Teams and OneDrive. That means that data sources that is connected to PowerBI is not going to be supported with Copilot 365. Also you can extend the data available in Microsoft 365 Copilot with Graph Connectors that can be used to connect to 3.party data sources such as Service Now, Confluence and such. Microsoft 365 Copilot while it uses RAG to handle data, when you activate the license Microsoft will also enabled Semantic Data which uses a combination of Vector embedding content stored in Microsoft 365 but also uses Hybrid search (Key Words + Vector Search) in combination with Semantic ranking to get more relevant data. In short Microsoft 365 Search becomes a lot better. Requires a minimum of 300 licenses.
Microsoft Copilot Studio Is a rewamp of Power Virtual Agents that is a bot framework ( to create chatbots) that can either interact with using predefined topics or keywords, but now you can also use it with generative AI. That allows the bot to have a conversation about topics that have not been defined. Copilot Studio can also be configured with custom data per bot, such as a SharePoint Site. It should be noted that this integration DOES NOT USE SEMANTIC INDEX so the search feature is not as good as it would be with Microsoft 365 Copilot. You can also integrate it with Azure OpenAI where you might have built your own custom application. This service is a seperate SKU hence does not require Microsoft 365 Copilot licenses to use. Easiest ways to create Chat Bots using Generative AI that can be published in Teams, websites or other channels.
You also have one feature called Conversational plugins in Copilot Studio, where you can build addons/extensions to Microsoft 365 Copilot, allowing you to build functions that the Copilot Orchestrator should handle, this can also be custom actions against 3.party data sources.
You also have Microsoft Copilot (formerly known as Bing Chat) which combines GPT + DALL-E + Bing Search Engine to use generative AI either to find information to generate content etc. By default does not have access to Microsoft 365 data.
This feature is also embedded into Windows using Windows Copilot. That allows you to use this outside of the browser. This Windows Copilot also has a set of functions that you can use to “chat” with different settings in Windows locally. Such as changing power settings, new background etc. So the language model is still coming from the cloud service and not running locally.
Then we have Microsoft Fabric Copilot and PowerBI Copilot, these are much of the same capabilities. Which is currently still in preview, however the features are aimed at providing copilto for data science, data factory and PowerBI. (It will most likely be available at the end of March 2024)
This feature does not allow you to “talk with your data” directly, but help with making transofmration and create queries or help create reports. So for instance you cannot directly ask it about “what are the total amount of sales in 2023” and point it to a set of data. You can however ask it to help creating PowerBI reports.
Microsoft Security Copilot which is currently in an EAP (Early adopter program) which uses much of the same capabilities as Microsoft 365 Copilot using the Graph API to query data sources from Defender, Sentinel, Entra ID, Purview and Intune to get information in combination with a virtual agent that is responsible for creating tasks based upon the prompt that is being sent in. This service will be available as part of the management UI in Azure but also as a seperate portal that will be used to access.
Microsoft has also introduced new Admin Copilots for Microsoft Azure and Microsoft 365, which will only be available in the management UIs. These can be helpful to get an understand of what kind of resources you have in Azure, but also how to for instance configure Exchange online settings. Most likely these feature are using RAG against the Microsoft documentation but is also using functions to run queries against Graph API and Microsoft Azure Resource Graph to get content.
