Top "Kafka-consumer-api" questions

Use for questions related to the Apache Kafka consumer API

Limit Kafka batches size when using Spark Streaming

Is it possible to limit the size of the batches returned by the Kafka consumer for Spark Streaming? I am …

apache-spark apache-kafka spark-streaming kafka-consumer-api
Kafka Consumer: No entry found for connection

I am trying to check the kafka consumer by consuming the data from a topic on a remote Kafka cluster. …

apache-kafka kafka-consumer-api
How to save latest offset that Spark consumed to ZK or Kafka and can read back after restart

I am using Kafka 0.8.2 to receive data from AdExchange then I use Spark Streaming 1.4.1 to store data to MongoDB. My …

apache-spark apache-kafka spark-streaming kafka-consumer-api
Retry logic in kafka consumer

I have a use case where i consume certain logs from a queue and hit some third party API's with …

apache-kafka kafka-consumer-api kafka-producer-api
kafka-python consumer start reading from offset (automatically)

I'm trying to build an application with kafka-python where a consumer reads data from a range of topics. It is …

python-3.x apache-kafka offset kafka-consumer-api kafka-python
Kafka optimal retention and deletion policy

I am fairly new to kafka so forgive me if this question is trivial. I have a very simple setup …

apache-kafka kafka-consumer-api kafka-producer-api kafka-python
Kafka Consumer: How to start consuming from the last message in Python

I am using Kafka 0.8.1 and Kafka python-0.9.0. In my setup, I have 2 kafka brokers setup. When I run my kafka …

python apache-kafka kafka-consumer-api kafka-python
Error UNKNOWN_MEMBER_ID occurred while committing offsets for group xxx

With Kafka client Java library, consuming logs has worked for some time but with the following errors it doesn't work …

java apache-kafka kafka-consumer-api
Spring Kafka Consumer Retry

I am using Spring Kafka consumer which fetches messages from a topic and persist them into a db. If a …

spring-boot apache-kafka kafka-consumer-api spring-kafka