In the previous segment, you learnt about the functionalities of various components of the Apache Flink ecosystem and its programming model. Now, let’s take a look at some of the use cases of Apache Flink.
In the following video, the expert will talk about some companies that use Apache Flink for their services.
Let’s summarise what you have learnt in this video.
Event-Driven Applications
Event-driven applications take events from one or more than one input event streams. The incoming events are handled by triggering computations or updating states or external actions. In comparison with traditional application designs, event-driven applications have separate compute and data storage. Event-driven applications are based on stateful processing. The following image depicts the difference between traditional applications and event-driven applications.
Examples of event-driven applications are as follows:
- Fraud detection
- Anomaly detection
- Rule-based alerting
- Business process monitoring
- Web application
Data Analytics Applications
In analytical jobs, information is extracted from raw data. Traditional analytical applications are used to perform batch processing on bounded datasets of recorded events. Now, whenever there is a need to incorporate latest changes, an application has to rerun the query on the entire dataset. But, with a good stream processing engine, analytics can also be done in real time as well. Real-time events are ingested, and the results are produced continuously as events are consumed. The results generated can be written to an external database or maintained internally.
Apache Flink supports both batch and stream analytics. The following image depicts the same.
Examples of data pipeline applications are as follows:
Real-time search building in e-commerce
Continuous ETL in e-commerce
Examples of data analytics applications are as follows:
- Quality monitoring of telecom networks
- Analysis of product updates in mobile applications
- Ad-hoc analysis of live data
- Large-scale graph analysis
Data Pipeline Applications
Extract-transform-load (ETL) is the basic approach used to convert and move data between storage systems. Data pipeline jobs and ETL jobs work in a similar fashion. Data pipeline jobs operate continuously rather than being triggered periodically. Hence, they are able to read data continuously from the source as soon as it is produced and move it to the destination with minimal latency. The following image depicts the difference between periodic ETL and data pipeline jobs.
Examples of data pipeline applications are as follows:
- Real-time search building in e-commerce
- Continuous ETL in e-commerce
Additional Reading
- Powered By Flink – This page lists down conferences, companies, projects etc. powered by Flink.