Welcome to this new module on ‘Real-Time Data Streaming With Apache Kafka‘. Let’s hear from the Ajay about everything that you will cover as part of this module.
In this module
You will first learn about some of the basic theoretical concepts related to batch and real-time processing. You will also learn in detail about the different components that constitute Kafka. Then, you will get a demonstration of how to create topics and how to push and real a message to and from a topic using the command line. You will also understand how to do the same using Python. Next, you will learn about Kafka connect and Kafka Streams and also get a demonstration of these concepts. In the video provided above, Ajay also mentioned that Kafka Streams has no support in Python. So, Kafka Streams part will be covered in Java, and for the rest of the module, you will use Python as the programming language, For this module, you will create a separate EC2 instance.
The following document lists all the steps that you need to follow to create this instance.
Also, you will need to create a Twitter developer account and a Twitter app, Once you do that, you will get an access token and access token secret. You will need these credentials in the demonstration of Kafka connect. Follow the steps mentioned in the document attached below to create a developer account and the app.
Please note your Twitter developer account may take 2-3 days to be reviewed and approved, hence we recommend you to apply for the same during the start of the module itself.
Guidelines for this module
This module is a mix of theory and demonstration. it is recommended that you start this module early in order to complete it within a week. Some sessions of this module are presentation-based. The presentation used in these sessions will be provided in the corresponding ‘Session Summary’ segment. The lecture notes will be provided in the ‘Module Summary’ segment. Also, you need to use the Kafka instance that you have created carefully.
Guidelines for in-segment and graded questions
There will be a separate session for graded questions. The other sessions will contain questions that will not be graded. The graded questions in this module will each have to marks for a correct response and o marks for an incorrect response. Each graded question will have only one attempt, while each non-graded question will have one or two attempts depending upon the type of question and the number of options.
People you will hear from in this module
Subject Matter Expert
Ajay Shukla
Senior Software Engineer, LinkedIn
Ajay is currently working as a senior software engineer at LinkedIn, an online employment-oriented platform. He has over nine and a half years of experience in the IT industry and has worked at various companies, including Amazon, Walmart and Oracle. He has a deep knowledge of various tools and technologies used today.
Subject Matter Expert
Ajay Shukla
Big Data Technical Architect
Report an error