How does Dropbox synchronization work?

algorithm synchronization dropbox file-transfer transfer
Michael Emad picture Michael Emad · Aug 15, 2011 · Viewed 8.5k times · Source

I want to know how dropBox is able to synchronize the large data files without replacing or re-uploading the files again to the dropbox server

Example: an encrypted zip archive

Suppose I've a 1GB encrypted zip archive file Fully synchronized on my computer and on the dropbox servers,

On my computer I added to that zip archive file a file of size about 5MB then saved the file on my computer,

dropbox is able to synchronize zip archive file without re-uploading the whole file again instead it just update it with the small change I've done.

Also TrueCrypt containers works in that manner

Any keywords, ideas, topics, reviews, links, code is greatly appreciated.

Answer

G Chris DCosta picture G Chris DCosta · Aug 10, 2012

Dropbox uses the rsync algorithm to generate delta files with the difference from file A1 to file A2. Only the delta(usually much smaller than A2) is uploaded to the dropbox servers since dropbox already has file A1. The delta file can then be applied to file A1, turning it into file A2.

You can learn more about the algorithm here. http://en.wikipedia.org/wiki/Rdiff-backup#Variations

The source code for the library behind the delta creation can be found here. http://librsync.sourceforge.net/

Related questions

synchronizing audio over a network

I'm in startup of designing a client/server audio system which can stream audio arbitrarily over a network. One central server pumps out an audio stream and x number of clients receives the audio data and plays it. So far …

Read More
What is the most clever and easy approach to sync data between multiple entities?

In today’s world where a lot of computers, mobile devices or web services share data or act like hubs, syncing gets more important. As we all know solutions that sync aren’t the most comfortable ones and it’s …

Read More
Whats the difference between Paxos and W+R>=N in Cassandra?

Dynamo-like databases (e.g. Cassandra) can enforce consistency by means of quorum, i.e. a number of synchronously written replicas (W) and a number of replicas to read (R) should be chosen in such a way that W+R>…

Read More