There has been an incredible development in LLM and AI in just the last few weeks! Personally, I have spent quite a bit of time testing, learning, and exploring, so I thought I would write a summary of what I have seen and what we can expect in the future, as well as some information about the landscape.
With ChatGPT v4, LLM (Language Models) has given us a unique opportunity to request information and create content. Microsoft has now introduced GPTv4 as part of their Open AI services on Azure, which now supports up to 32000 tokens (Estimated around 5500 words), allowing it to interpret larger documents and content.
(I have written a blog post on how to use this with your own data –> How to setup Azure OpenAI with ChatGPT using your own data? – msandbu.org
The service is currently limited to certain Microsoft regions (such as the East US data center) as it requires dedicated infrastructure with GPU to deliver it, but it makes it possible to provide a dedicated GPT service for an organization.
The challenge with ChatGPT and Azure OpenAI services has been that it has not had direct access to the internet which prohibits it from gaining access to current information and it does not have any built-in memory. Therefore, extensions and integrations need to be created to provide that capability.
OpenAI has now announced “plugins” that allow users to expand ChatGPT with various APIs. However, these things are still done in the context of ChatGPT. This is currently in closed beta and will likely be made available soon to a broader audience.
Fortunately, there have also been other opportunities that have emerged, such as LangChain which makes it possible to connect numerous service such as GPT with both memory and internet access using different integrations. LangChain can also be used to make various data sources directly available to GPT, or it can be a vector database containing search terms that can be used for lookup of data.
One example of using Langchain together with Google Search API via SerpAPI.
LangChain has a range of integrations that can be used together to create applications that leverage GPT while providing additional features. For example, we can use LangChain to create a sample application that can use the internet to search for specific information, such as the Google Search API, as well as Pinecone for vector data storage (or language APIs for voice-activated tasks). Alternatively, you can use more fully automated solutions to perform specific operations or do something fully automated, as evidenced by recent examples such as
AutoGPT (https://github.com/Torantulino/Auto-GPT) This screenshot below shows AutoGPT in default mode, where I’ve asked it how to be a leading-edge managed service provider in 2023 and requires consent before each command that is executed, however it can also run in a fully autonomous mode.
or BabyAGI (https://github.com/yoheinakajima/babyagi).
Both examples demonstrate that there are many possibilities to explore with the use of LLM to create/find business ideas, explore different scenarios, or collect and analyze data. At the same time, there is also more innovation in using LLM solutions locally on one’s own infrastructure without relying on a public cloud service like OpenAI. These include, for example:
GPT4All (GitHub – nomic-ai/gpt4all: gpt4all: an ecosystem of open-source chatbots trained on a massive collection of clean assistant data including code, stories, and dialogue)
Alpaca (Stanford’s GPT-3 Clone, based on LLaMA) (GitHub – tatsu-lab/stanford_alpaca: Code and documentation to train Stanford’s Alpaca models, and generate the data.)
At the same time, we are also seeing new versions that are pre-trained on various sources and literature, such as MedAlpaca, which is designed for services in the healthcare industry (GitHub – kbressem/medAlpaca: LLM finetuned for medical question answering)
I can also mention that if you look at the GitHub trending repositories over the past month, almost all of them are related to LLM. Among the trending repositories on GitHub this week is Microsoft’s project Jarvis (GitHub – microsoft/JARVIS) Jarvis aims to provide a chat interface between those working with ML and various AI features available through Huggingface.
We are also seeing more organizations and users now using ChatGPT for things other than just creating content/text. We have also seen several services that use ChatGPT and LLM functions for more operational services, such as an AIOps product, such as k8sgpt-AI (k8sgpt · GitHub). What if a tool could analyze all your logs and text and at the same time use the internet to find out what is wrong or challenging with your infrastructure? This shows some of the possibilities.
Furthermore, there is also a lot happening in the field of Image/Audio/Video AI. OpenAI has, for instance, introduced DALL-E, which is their library for creating images based on text. However, there are also other options available, such as Midjourney (which released its latest version a few weeks ago).
Note: Midjourney is a small company with 10-15 employees, and it’s impressive what they have accomplished!
Here is an image I created last week based on the following prompt:
(puppy white golden retriever, sun light，insane detail, smooth light, real photography fujifilm superia, full HD, taken on a Canon EOS R5 F1.2 ISO100 35MM --ar 4:3 --s 750 --q 2)
At the same time, we are also seeing that this technology can be used for different purposes. For instance, someone used AI to create a new product that was announced by Steve Jobs using numerous services: Jim Fan on Twitter: “Below is Steve Jobs announcing a new product, “WiFi-connected socks” in his eloquent voice. I’m sold. The Black Mirror episode “Be Right Back” describes a technology that revives people from their digital footprint. Where we are today: – LLMs prompted by past chat logs can do a… https://t.co/NEkUxR0P42” / Twitter
So, there is a wild development happening here, and Microsoft plans to integrate this functionality into all their products such as collaboration (Teams, Office) with Microsoft 365 Copilot, Security Copilot, and Github Copilot X with is the next evolution of Copilot to help developers.
Therefore, one must take precautions and understand how these services process and handle information and how they can bring value to an organization.