Recently Microsoft announced that they will be providing a new feature for Microsoft called “Copilot” which you can read more about here –> Introducing Microsoft 365 Copilot – your copilot for work – The Official Microsoft Blog. This will allow you to use ChatGPT like features directly into Microsoft 365, including the different Microsoft 365 applications and within the data in your organization.
NOTE: The information on this blog is based upon information that is publicly available from the Microsoft blogs and based upon my own knowledge about GPT, the blog will also be updated when the feature is released or more information becomes available.
Microsoft have also announced Copilot features for other core services as well, such as Viva, Security, Power Apps and even Dynamics 365.
So how will this feature work? Well based upon the information that is available now from the video and blog we have a good understanding about the mechanisms underneath. When Copilot will become available it will be available as a separate service within Microsoft 365 and integrated into the different Microsoft 365 applications such as (Word, Excel, Teams, PowerPoint)
I’ve tried to summarize in the design drawing underneath what it looks like and will go through how it works. First of it is important to note that Copilot and the language model behind it will not be trained on your data, the data sources that is uses is also described in this research (https://arxiv.org/pdf/2005.14165.pdf) article and is based upon licensed data sources and not from your own organization.
As of now if Microsoft is using GPT 3.5 (which is the one used in ChatGPT or GPT-4 which is the newer model is not described yet, but none of these new models support fine-tuning, meaning that you cannot directly get the data model to learn new things. There are only two ways for it to learn information, either by OpenAI/Microsoft by generating a new model based upon new sources, or it can learn temporary based upon the data that is inserted into the prompt.
Meaning that Copilot will most likely not have any “memory” of previous events, of if you have asked it about something before it will not remember it. Unless Microsoft introduces some vector database or caching mechanism in Copilot.
So, when Copilot becomes available within an organization and a user tries it from the Office 365 Portal what happens?
- End-users sends a request to Copilot “How much vacation am I entitled too?”
- This will trigger a search against the Microsoft 365 Graph via Search to see if there are data sources that might contain information about those keywords.
- The search provides back two documents which contain some information about the request from the prompt.
- The result is then collected and then inserted as a prompt into the LLM (Which will most likely be a dedicated Azure OpenAI instance that is used for Office 365)
- The LLM will then interpret the data and provide the end-user with a response via the Copilot UI.
A similar workflow will be if using Copilot as part of Microsoft Teams. Let’s say you want a summary about an on-going chat in Microsoft Teams around an on-going incident, you can ask Copilot to write a summary about the chat, which will collect the entire conversation in text format, send it to the LLM and ask the LLM to generate a summary.
Now the biggest challenges with this are going to be integrating other sources into the Copilot feature. Think about having the possibility to extend the copilot to other data sources outside of Office 365 as well, which will require that you have a data source that is supported by Graph Connectors such as these listed here –> Microsoft Graph connectors overview for Microsoft Search | Microsoft Learn
The problem now is that the service availability for running OpenAI in Azure is still quite limited, at the time of writing this blog post it is only available in 3 different regions (US and West Europe) meaning that it will take some time before it is available for all customers on a global scale as part of Office 365, I reckon.
I also wanted to add a common FAQ to provide simple answers to some of the frequent questions that I get
- Is the data going through Copilot used to teach or improve the model? – No, the data (as part of the prompt) will only be used when a user ask for a specific action or request, as part of Azure OpenAI policy the prompt can be stored for 30 days, so that Microsoft engineers can determine if it has been used for malicious purposes.
- Is this service running against the same service as ChatGPT? – No, this service is specific for Microsoft 365 but uses the same models underneath that powers ChatGPT.
- Will my prompts be available for other users within my Office 365 organization? – No, only for you as a user.
- How does the service able to access the data? When do send in a prompt, the service will first user the built-in search mechanism which uses the users access rights to determine information and content that is accessible by the users. The information is then passed into the LLM and used to provide feedback to the user. So the data that is given back to the user is only data that the user has access to.