Now that you have understood the concepts of Producers and Consumers, and learnt how to push data into a topic and consume data from topics, in this session, you will learn what Kafka Connect is. So, let’s hera from Vishwa as he explains how Kafka Connect is used as a tool.
In the video above, you learnt about Kafka Connect. Kafka Connect is a framework that connects Kafka with external systems. These systems could be any streaming services, databases or file systems. Using Kafka Connect, you can move data either from these external systems to a Kafka topic or from a Kafka topic to these external systems. There are many connectors that have been written for different types of sources and sinks. Some of these connectors are open source and can be utilised to write data to and from a Kafka topic.
In the next video, you will be introduced to some of the important concepts of Kafka Connect.
In the video above, you learnt about some of the key terminologies of Kafka Connect. Connectors define the direction of the flow of data; the data can either be removed from a Kafka topic to an external system or from an external system to a Kafka topic. Tasks refer to the actual job of copying the data, which are coordinated by connectors. Workers are processes that execute the connectors and tasks.
In the next segment, you will learn more in detail about the various concepts of Kafka Connect.
Additional Reading
Kafka Connect – You can read more about Kafka Connect and its features from this link.
Report an error