In this segment, we will examine the timeline of developments in artificial intelligence . This timeline has been divided into two parts, with some overlap between the two parts are:
- Pre-generative AI era
- Generative AI era
This division will allow us to focus on the relevant developments that happened before generative AI emerged and gain a comprehensive understanding of the different technologies that constitute generative AI.
Note
You may encounter some technical technical terminologies in this segment. Words such as ‘machine learning’, ‘deep learning’, ‘parameters’, etc. may be unfamiliar to you now. However, don’t fret, as you will get a deeper understanding of the relevant terminologies and concepts as you progress through the program.
In the next video, Kshitij will outline the topics that will be covered in this session and discuss significant technological breakthroughs that occurred in the pre-generative AI era.
The following image was used to illustrate the key breakthroughs in AI before the advent of generative AI.
Kshitij focused on a few milestones from the image. We encourage you to use it as a guide and delve deeper into these topics. Major breakthroughs were those that captivated the public’s imagination or introduced AI into previously unexplored domains. Instances like computer programs defeating world champions in different games such as chess, Jeopardy!, and Go popularised artificial intelligence among non-technical individuals. Additionally, AI’s forays into new areas with innovations like Roomba, self-driving cars, and personal assistants such as Siri expanded the horizon of the application of AI models.
Although AI models progressed towards mastering various human capabilities in different tasks, they still struggled to generate human-like natural language output. However, this changed with the advent of generative AI technologies.
In the next video, Kshitij will discuss the timeline of the development of generative AI technologies.
Before delving into the development of generative AI technologies, Kshitij provided a note regarding the technical terminologies used in the previous videos. You will be provided with an explanation of these terms in the subsequent videos.
The following image was used to illustrate the key breakthroughs in generative AI.
We began by discussing the introduction of transformer models in the paper Attention Is All You Need. For learners with a technical inclination, this paper is an excellent introduction to the workings of transformer models and, subsequently, other large language models. You can find the paper here. These models introduced an architectural innovation called the ‘attention mechanism’, allowing researchers to increase the amount of data being used to train the model.
Next, we explored the first GPT model released by OpenAI. GPT stands for generative pre-trained trained transformer. The term ‘pre-trained’ in GPT refers to the fact that the model being initially trained on a large corpus of language data to grasp language patterns. Subsequently, this pre-trained model is fine-tuned for various language-related tasks. GPT was followed by GPT-2 and GPT-3, demonstrating that one model could be used for multiple language tasks such as translation, summarization, and more.
To gain a better understanding of the breakthroughs achieved by current current state-of-the-art models inspired by GPT and GPT-2 models, you can refer to articles here and here.
Additionally, it would be beneficial for you to explore the evolution of image generation AI models independently. To kickstart your exploration, here are some keywords to guide your research: Generative Adversarial Networks (GANs), DeepDream, Style Transfer, diffusion models, and so on.
Now that you have familiarised yourself with the timeline of artificial intelligence algorithm development, answer the following questions to evaluate your understanding of the topics covered in this session.
In the next segment, you will learn about the different paradigms in artificial intelligence and how they are related to each other.