Flask - Pulling the live stream kafka data - Integrating Kafka with Python Flask

python flask socket.io flask-socketio kafka-python
Soundarya Thiagarajan picture Soundarya Thiagarajan · Feb 5, 2016 · Viewed 8.3k times · Source

This project is for real time search engine - log analysis performance.

I have a live streaming data out from Spark processing to Kafka.

Now with the Kafka output, I want to get the data from the Kafka using Flask.. and visualize it using Chartjs or some other visualization..

How do I get the live streaming data from Kafka using the python flask?

Any idea how do I start with?

Any help would be greatly appreciated!

Thanks!

Answer

pech0rin picture pech0rin · Feb 5, 2016

I would check out the Kafka package for python:

http://kafka-python.readthedocs.org/en/master/usage.html

This should get you setup to stream data from Kafka. Additionally, I might check out this project: https://github.com/travel-intelligence/flasfka which has to do with using Flask and Kafka together (just found it on a google search).

Related questions

Get the data received in a Flask request

I want to be able to get the data sent to my Flask app. I've tried accessing request.data but it is an empty string. How do you access request data? from flask import request @app.route('/', methods=[…

Read More
How to serve static files in Flask

So this is embarrassing. I've got an application that I threw together in Flask and for now it is just serving up a single static HTML page with some links to CSS and JS. And I can't find where in …

Read More
Return JSON response from Flask view

I have a function that analyzes a CSV file with Pandas and produces a dict with summary information. I want to return the results as a response from a Flask view. How do I return a JSON response? @app.route("/…

Read More