Top "Partitioning" questions

Partitioning is a performance strategy whereby you divide possibly very large groups of data into some number of smaller groups of data.

Is it possible to create a kafka topic with dynamic partition count?

I am using kafka to stream the events of page visits by the website users to an analytics service. Each …

apache-kafka partitioning kafka-consumer-api
How does partitioning work in Spark?

I'm trying to understand how partitioning is done in Apache Spark. Can you guys help please? Here is the scenario: …

apache-spark partitioning
spark parquet write gets slow as partitions grow

I have a spark streaming application that writes parquet data from stream. sqlContext.sql( """ |select |to_date(from_utc_timestamp(…

apache-spark partitioning parquet
Can MySQL create new partitions from the event scheduler

I'm having a table looking something like this: CREATE TABLE `Calls` ( `calendar_id` int(11) NOT NULL, `db_date` timestamp NOT …

mysql partitioning
Database - Designing an "Events" Table

After reading the tips from this great Nettuts+ article I've come up with a table schema that would separate highly …

mysql database database-design relational partitioning
#1016 - Can't open file: './database_name/#sql-38f_36aa.frm' (errno: 24)

I have table in mysql with MyISAM storage engine. I want to create partition on a particular table, for this …

mysql partitioning
Partition Hive table by existing field?

Can I partition a Hive table upon insert by an existing field? I have a 10 GB file with a date …

hadoop hive partitioning hdfs database-partitioning
How to do automatic data archiving in SQL Server?

I have table for which every day I want to do automatic archiving. So to be clear every day I …

sql-server sql-server-2005 partitioning continuous archiving
Using Partition on oracle table from a DB link

I am working on Oracle 9i my question is: can I use a partition over a table through a dblink …

oracle partitioning oracle9i dblink
How to perform one operation on each executor once in spark

I have a weka model stored in S3 which is of size around 400MB. Now, I have some set of …

scala apache-spark weka partitioning