Apache Spark vs Apache Ignite

apache-spark ignite
Normal picture Normal · Mar 16, 2016 · Viewed 18.1k times · Source

Currently I'm studying Apache spark and Apache ignite frameworks.

Some principle differences between them are described in this article ignite vs spark But I realized that I still don't understand their purposes.

I mean for which problems spark more preferable than ignite and vice versa?

Answer

Valentin Kulichenko picture Valentin Kulichenko · Mar 16, 2016

I would say that Spark is a good product for interactive analytics, while Ignite is better for real-time analytics and high performance transactional processing. Ignite achieves this by providing efficient and scalable in-memory key-value storage, as well as rich capabilities for indexing, querying the data and running computations.

Another common use for Ignite is distributed caching, which is often used to improve performance of applications that interact with relational databases or any other data sources.

Related questions

How can I change column types in Spark SQL's DataFrame?

Suppose I'm doing something like: val df = sqlContext.load("com.databricks.spark.csv", Map("path" -> "cars.csv", "header" -> "true")) df.printSchema() root |-- year: string (nullable = true) |-- make: string (nullable = true) |-- model: string (nullable = …

Read More
Spark - load CSV file as DataFrame?

I would like to read a CSV in spark and convert it as DataFrame and store it in HDFS with df.registerTempTable("table_name") I have tried: scala> val df = sqlContext.load("hdfs:///csv/file/dir/file.csv") Error …

Read More
How to change dataframe column names in pyspark?

I come from pandas background and am used to reading data from CSV files into a dataframe and then simply changing the column names to something useful using the simple command: df.columns = new_column_name_list However, the same …

Read More