Today I had a presentation at the SQL Saturday event in Norway about how to use OpenAI to talk with your databases. Therefore I wanted to describe some of the options here and lastly how you can extend the capabilities using Langchain and other 3. party tools to communicate directly with your data. (Semantic Kernel fra Microsoft which is a Langchain alternative can also be used as this approach)
Out of the box, services like ChatGPT cannot talk directly with your databases. Sure you have some plugins that are available that can be used to communicate with some data sources but allowing a direct integration requires some programming to set up communication.
One of the tools that we can use is Langchain, which is an LLM integration framework that allows us to connect language models with data, provide memory, and integrate with other 3. party tools such as Google Search. The list of different integrations that Langchain supports is quite long.
To allow Langchain to communicate with a let’s say SQL database requires the use of a Python library called SQLAlchemy.
SQLAlchemy is a pretty extensive Python SQL library that can pretty much communicate with any type of SQL database. So Langchain can use that library to talk directly with the database to run queries to get information and context.
The example shown here https://github.com/msandbu/gpt-ai/blob/main/ex-langchain.py can be used as an example where Langchain is talking directly with a database using the SQLAlchemy library. Then run a specific prompt to figure out which user has the ID nr 5. It should be noted that in this example I have defined some variables that need to be configured, such as SQL Endpoint, Username and password, and also OpenAI Key. Then those variables are used in the script to set up an ODBC connection.
When I trigger the script with a specific prompt “Who has ID nr 8 in the database?” the workflow has no insight into how the database is structured and will therefore use the chain of thoughts to start with inspecting the database, and tables to determine the structure before sending a set of SELECT queries.
So here is just showing an example Langchain workflows can then be set up to talk directly with your database. While Langchain CLI is nifty we can also make it even better and use a framework like Streamlit to provide a web UI on top. This is using this code example here gpt-ai/ex-langchainsqlstreamlit.py at main · msandbu/gpt-ai (github.com) which is then hosted from a private GitHub repo. When someone enters their OpenAI key they can use a set of predefined queries or enter their own query.
Of course, we would not in most cases have a langchain flow to talk directly with a SQL database. We can also use other integrations such as GraphQL with tools like Azure API Management or Directus as a gateway between the Langchain workflow and the database. Or have a read-only replica of the database that the workflow can communicate with.
However I am lazy so writing questions is still a lot of effort, what if we can use voice recognition to make this instead?
This is where the last example comes in which can be found here –> gpt-ai/jarvissql.py at main · msandbu/gpt-ai (github.com) AKA TableTalker AI (Not actually AI but just chaining these services together) . This uses a combination of Langchain, OpenAI Functions, Elevenlabs and Azure Speech Recognition to handle voice. The flow is such
1# Parses speech as text using Azure Speech Recognition
2# Text handled as input to Langchain agent context
3# Depending on keyword will use a OpenAI Function (For instance – db.chain.run for SQL related queries)
4# SQL Questions will be sent to the SQL Database
5# Output parsed using Elevenlabs
6# Context back to the user
Currently, this uses multiple libraries including MKV to collect and stream audio. However, it works pretty well, even with using the free service from Elevenlabs.
Using this we can have voice-activated search directly to our databases. We can even have different functions that talk with different data sources. In an upcoming blog post I will go into more detail on how you can optimize and some more of the limitations, but this blog post has been more focused on the possibilities that Langchain and the ecosystem provide.