EC2 EBS Snapshots as Incremental Backups
I understand that AWS snapshots can create incremental backups of EBS volumes. Does AWS automatically handle the incremental part (i.e., storing only what has changed) as long as snapshots are generated from the same volume?
It's unclear to me because they do not list the actual size of the snapshots or allow you to view them in S3 (as far as I know). There is no indication snapshots are related other than the volume they were created from. Couldn't any snapshots made (including the first) just be considered an increment on the original AMI? I would be interested to know if this how they actually implement this or if the first snapshot is a completely independent image stored in my personal S3 account.
Answer
Each EBS snapshot only incrementally adds the blocks that have been modified since the last snapshot.
Each EBS snapshot has all of the blocks that have been used ever on the EBS volume. You can delete any snapshot without reducing the completeness of any other snapshot.
It's magic.
Well, it's actually a bit of technological indirection where each snapshot has pointers to the blocks that it cares about and multiple snapshots can share the same blocks. As long as there is at least one snapshot that points to a particular set of data on a block the block is preserved in S3.
This makes it difficult for Amazon to tell you how much space a single snapshot takes up, because their sizes are not exclusive of each other.
Here's an old article from RightScale that has some nice pictures explaining how snapshots work behind the scenes:
Note also that snapshots only save the blocks on the EBS volume that have been used and snapshots are compressed, further reducing your data storage cost.