In this segment, you will learn about the generative AI software stack.
After defining generative AI and exploring some tools, you are now ready to structure the tools according to their purpose while building generative AI-based applications. The collection of tools that might be used to create software is sometimes referred to as a software stack. If you’re interested, you can learn about software stacks here.
For developing generative AI applications also, you need to define your software stack. Let’s hear from Kshitij as he organises the generative AI tools into a stack.
Models & APIs
Kshitij started the discussion of the generative AI software AI software stack by talking about the different categories of generative AI models. Some examples of models that lie within the four categories descrided by Kshitij are given below. We strongly encourage you to follow these links to compare and contrast these models.
Although these are the four categories of models that Kshitij talked about in detail, models that don’t neatly fall into these four categories also exist. You have seen some of these before in the previous segments; for example, GitHub Copilot for generating code, Runway.ai for generating videos, and so on.
Vector Databases
After talking about the different LLMs and APIs that can be used in the generative AI stack, Kshitij talked about vector databases. Vector databases provide a new way of structuring data that improves the performance of the database in multiple ways.
In a nutshell,vector databases store the data in the form of high-dimensional vectors which can achieved using a transformation or an embedding function. These high-dimensional vectors allow for a faster search using a technique called semantic similarity. Instead of relying on conventional approaches such as querying databases using precise matches or predetermined criteria, a vector database allows you to locate data that is most similar or relevant by considering their semantic or contextual significance.
Some examples of vector databases are given below:
- Pinecone
- Chroma
- Qdrant
In the next video, Kshitij will continue his exposition of the generative AI software stack.
LLM Frameworks
Kshitij continued his discussion of the generative AI stack by talking about LLM frameworks allow developers to build complex applications using LLMs that can execute a series of tasks and access external tools to complete complex tasks accurately. Specifically, they allow large language models to access other data sources and APIs and allow them to interact with the software’s environment.
One example that Kshitij talked about was how generative AI models may be used to solve complex maths problems. As large language models are trained on text data, and as they don’t really have built-in capabilities to solve complex maths problems, we may want to use an external Asuch as WolframAlpha to solve the maths problem.
Some examples of LLM frameworks are given below:
- Langchain
- LIamaIndex
- Anarchy
Deployment
Finally, Kshitij talked about deploying generative AI applications such that they are scalable. As an example of such deployment infrastructure, Kshitij talked about the suite of tools provided by Microsoft in its Azure OpenAI Services. This product by Microsoft provides a single place to deploy your application in the cloud and to easily access the ChatGPT API. Other examples of deployment infrastructure are Vertex AI by Google and HuggingFace Inference Endpoints.
To summarise, Kshitij talked about the four levels in the generative AI software stack that are given below.
These four levels of the generative AI software software stack can be summarised as follows:
- LLMs and APIs: The large language models (LLMs) are the main component in this software stack. It processes the user’s data and provides a response. As LLMs are very expensive to train and host, developers prefer to access them using application programming interfaces (APIs) provided by well-established models, such as GPT-4, Midjourney and so on.
- Vector databases: Vector databases provide a way to unlock a new level of performance by allowing the LLM to access the database semantically.
- LLM frameworks: LLM frameworks are collections of software tools that allow the developer to leverage other technologies to improve the performance of the LLM. For example, the Langchain framework allows you to use the WolframAlpha API to perform complex mathematical calculations.
- Deploy: These are tools that are used to deploy your application on the internet. As LLM software has some specific requirements, many deployment service providers have started providing end-to-end services specifically for LLM applications.
You may feel overwhelmed by looking at all of these advanced software engineering tools at this point. But the purpose of the preceding videos was to simply provide you with an overview of these tools. We will get into each of these tools individually in much more detail as the course progresses.
But to build intuition about these tools, you should thoroughly explore the links provided to you through this segment. Based on your exploration, answer the following questions.
You are now familiar with the landscape of generative AI and the applications and tools that lie within.