Spark Streaming is an extension of the core Apache Spark API that enables high-throughput, fault-tolerant stream processing of live data streams.
I'm working on a Spark Streaming program which retrieves a Kafka stream, does very basic transformation on the stream and …
java apache-spark jmx spark-streaming codahale-metricsI am having issue in reading data from azure blobs via spark streaming JavaDStream<String> lines = ssc.textFileStream("…
java azure apache-spark azure-storage-blobs spark-streamingI am new to spark and trying to install spark on Amazon cluster with version 1.3.1. when i do SparkConf sparkConfig = …
apache-spark spark-streaming amazon-emrI'm trying to open a Kafka (tried versions 0.11.0.2 and 1.0.1) stream using createDirectStream method and getting this AbstractMethodError error: Exception in …
scala apache-spark apache-kafka spark-streamingSpark 2.0.0 Apache Kafka 0.10.1.0 scala 2.11.8 When I use spark streaming and kafka integration with kafka broker version 0.10.1.0 with the following Scala …
apache-spark serialization apache-kafka spark-streamingI have a particular use case where I have multiple rows for same customer where each row object looks like: …
java scala apache-spark apache-spark-sql spark-streamingI have a spark 2.0 application that reads messages from kafka using spark streaming (with spark-streaming-kafka-0-10_2.11). Structured streaming looks really …
scala apache-kafka spark-streaming avro apache-spark-2.0The spark-streaming website at https://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations-on-dstreams mentions the following code: dstream.foreachRDD { rdd =&…
apache-spark spark-streamingI have a problem when i use spark streaming to read from Cassandra. https://github.com/datastax/spark-cassandra-connector/blob/master/…
scala spark-streaming spark-cassandra-connectorI want to create a system where I can read logs in real time, and use apache spark to process …
apache-spark apache-kafka flume spark-streaming