Top "Distributed-computing" questions

utilizing more than one computer, connected to each other with a communication link to accomplish a common task.

Determining optimal number of Spark partitions based on workers, cores and DataFrame size

There are several similar-yet-different concepts in Spark-land surrounding how work gets farmed out to different nodes and executed concurrently. Specifically, …

apache-spark spark-dataframe distributed-computing partitioning bigdata
Heartbeat Protocols/Algorithms or best practices

Recently I've added some load-balancing capabilities to a piece of software that I wrote. It is a networked application that …

sockets network-programming network-protocols distributed-computing
Distributed Computing Framework (.NET) - Specifically for CPU Intensive operations

I am currently researching the options that are available (both Open Source and Commercial) for developing a distributed application. "A …

.net frameworks distributed distributed-computing grid-computing
Differences between AMQP and ZeroMQ

Recently started looking into these AMQP (RabbitMQ, ActiveMQ) and ZeroMQ technologies, being interested in distributed systems/computation. Been Googling and …

distributed-computing actor amqp zeromq
what is a data serialization system?

according to Apache AVRO project, "Avro is a serialization system". By saying data serialization system, does it mean that avro …

hadoop distributed-computing data-serialization
What is the dIfference between a distributed system and a clustered system?

Both are defined to be a set of computers that work together and give the end users a perception of …

cluster-computing distributed distributed-computing
Service discovery vs load balancing

I am trying to understand in which scenario I should pick a service registry over a load balancer. From my …

web-services amazon-web-services cloud distributed-computing microservices
Spark Java Error: Size exceeds Integer.MAX_VALUE

I am trying to use spark for some simple machine learning task. I used pyspark and spark 1.2.0 to do a …

java python apache-spark distributed-computing logistic-regression
Centralised Java Logging

I'm looking for a way to centralise the logging concerns of distributed software (written in Java) which would be quite …

java logging log4j distributed-computing logback
Leader Election Algorithm

I am exploring various architectures in cluster computing. Some of the popular ones are: Master-Slave. RPC ... In Master-slave, the normal …

algorithm distributed-computing cluster-computing master-slave