File sync and cloud storage client comparison

25 July 2013

I’ve been trying out a number of file sync clients recently, finding that many are trying to solve the same problem and meeting with varying success. Probably the best-known service in this category is Dropbox, but nearly every big cloud services company, and a multitude of startups which base their services on top of them, are providing a ‘cloud drive’ of some description.

First off, I don’t want to reinvent the wheel here, so you might be interested in some background reading on cloud storage:

The following table breaks down some of the services I’ve tried out, which hopefully gives you an idea of the range on offer. Further down I examine some of the issues that you might want to consider when choosing a cloud storage provider. Even though I’m in the UK, I’ve used USD for pricing, since most services offer this currency.

If you would like to calculate how much various cloud storage providers would cost you for a set amount of data storage, you might like to check out CloudPricing.co.uk.

Note I’m aware the table below is broken, I will update it and fix it in a future post.

| | | | | | | | | | | | | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | Service | Free storage | Small | | $/GB/mo | Large | | * $/GB/mo* | Linking | Shared Sync | Versions | Online access | Microbilling | | Dropbox | 2GB - 18GB | $9.99 | 100GB | 0.1 | $49.99 | 500GB | 0.1 | Yes | Yes | Yes | Yes | No | | Backblaze | $5 | Unlimited | No | No | Yes | Yes | No | | Cloud Drive ( Amazon) | 5GB | $0.50* | 20GB | 0.025 | $26.67* | 1TB | 0.027 | Yes | No | Deleted | Yes | No | | Google Drive † | 15GB | $4.99 | 100GB | 0.05 | $799.99 | 16TB | 0.05 | Yes | Yes? | Yes | Yes | No | | Bitcasa | 10GB | $11.68 | 1TB | 0.012 | Yes | No | Yes | Yes | No | | Creative Cloud ( Adobe) ‡ | 2GB | $49.99* | 20GB | ? | Yes | Yes | Yes | No | | SkyDrive (Microsoft) | 7GB | $10 | 20GB | 0.5 | $50 | 100GB | 0.5 | Yes | Yes? | Yes | Yes | No | | iCloud (Apple) † | 5GB | $1.67* | 10GB | 0.167 | $8.33* | 50GB | 0.167 | Photos | No | No? | Yes | No | | Livedrive | $12+* | 2TB | 0.006 | Yes | No | No? | Yes | No | | SugarSync | $6.25* | 60GB | 0.1 | $33.25* | 500GB | 0.067 | Yes | Yes | Yes | Yes | No | | **Bring your own storage ** | | BitTorrent Sync | Yes | Yes | No | | Sparkleshare | No | Yes | Yes | Yes | | rsync | No | No | | s3sync + Amazon S3 | < 1TB | 0.095 | >5PB | 0.055 | Yes | No | Yes | Yes | Yes |

* Annual subscription - † Shared with other services - ‡ Includes other services - Prices and storage were current on July 26th 2013

Usage tiers and microbilling

When I first read about Dropbox, I thought: ‘100GB for only $10 a month - how can they do that?!‘. As you can see from the table, they are not even the cheapest service, and come in only a hair above Amazon’s S3 ( Simple Storage Service) on a per-GB rate. Given that they use S3 for their underlying storage, how do Dropbox make any money? And they give away loads of free accounts too!

The answer should be obvious to people who are familiar with the way Dropbox works. When you share a folder, the space used counts against both user accounts, essentially saving Dropbox storage space. The more people sharing a folder, the less space Dropbox actually needs to be able to provide overall. In addition, due to the way the storage tiers work, you have to pay for the entire amount, even if you are only using some of it. Say you have filled your 2GB free account, and upgrade to 100GB. Now, until you store the full 100GB, you’re not getting the actual $0.10 per GB rate. In fact, when you’re storing 50GB of data, you’re paying $0.20/GB, and some of that will most likely be shared too.

Amazon CloudDrive is another service which is much cheaper per GB than the base storage costs would suggest. OK, so Amazon operate S3, so they get better rates, but they also include MP3 streaming as a large part of their service. MP3s you purchase from Amazon don’t count against the storage space, but in theory, if lots of people upload identical DRM-free MP3 files, Amazon would only need to store one copy of the file; I don’t know if they actually apply that kind of process though.

So, basically, most users are going to end up paying relatively high average unit costs for the amount of data they store. Maybe some users will fill their allotment, but then nobody likes to live that close to the edge - what if they need to share another file, and their account is full? My conclusion is that services which offer relatively small increments (say 10GB at a time), rather than massive ‘Pro’ usage tiers which you won’t fill for a long time, are probably better value in the end.

Unlimited storage

Some services offer ‘Unlimited’ storage. Even if prices are calculated as if they were providing 1TB of storage, these services still look cheap.

Out of the two I looked at here, Backblaze is pitched as a pure backup service. It doesn’t offer live sync, linking or sharing; only one-way backup and restore. They can get their costs down because they don’t need to have fast access to all their data all the time - they can optimise for cost above all other considerations.

This has some similarities to the Amazon Glacier service, which is the long-term archival version of S3. It has costs starting at $0.01/GB, which would place it in the realm of a $10/TB/month storage service. Backblaze is still cheaper. As with Dropbox, I suspect they can optimise here by assuming that most users actually only want to store their files, not the entire contents of their hard drives. In my case, my Backblaze data set is just over 100GB, and assuming that some compression is also used to reduce the final size on the server, I can imagine the costs for Backblaze coming in at around $1/month. If they charge me $5, they can afford to support a few users who want to back up terabytes of data.

I assume similar ecomomies are made with the other ‘Unlimited’ provider listed here, Bitcasa. They aren’t pitching themselves as ‘just’ a backup service, though they do make mention of using their service for backing up data. It costs a bit more per month, but you get live access to your data, and two-way sync.

Data Security

Is your data secure on these cloud services? Do you trust them not to go bust or lose the data, and do you trust them not to let hackers get into their systems and rummage through your files? What if you max out your credit card and a payment bounces? Paranoia will only take you so far, and in the end the data centres where these companies have their servers are probably more secure than your home. That’s not to say you can’t take precautions.

Keeping an offline backup in addition to an online backup is probably wise. If nothing else, it will be quicker to restore if you need it. Backblaze will send you up to 3TB of files from your archive on an external hard drive, but they have to still have your data.

For ultimate security, you can supply your own encryption key. This is supported by some of these services, but not all. The only one that I noticed advertising support was Backblaze. This of course has the tradeoff that you have to store the key, possibly on a USB drive or SD card, and put it somewhere safe. If you lose it, your backup is useless. Maybe you could tattoo it onto your arm.

Backblaze encryption settings dialog

Bring your own storage

A few pieces of software exist which can help you recreate some of the features of these services on your own server, or using other data storage services with pay-as-you-go pricing, such as Amazon S3 and Glacier.

One of the most basic ways to transfer files is using an FTP or SFTP client. This can be automated, but synchronising lots of files this way can be very inefficient and error prone. For this reason, tools like rsync and unison are generally used for this purpose instead.

But where are you going to transfer your files to? If you want to use these tools you will need a server somewhere with which to synchronise. Hard drives currently cost around $0.045/GB at least $0.034/GB based on $100 for a 3TB drive (updated March 2014), and are warrantied for between 2 and 3 years. If you assume that you will get at least 2 years out of them, run them as a mirrored pair for reliability, and you can find a low-powered NAS unit to run as a server for around $100, which costs about $75 a year to run ( a lot of estimates here), you can estimate a cost for storage to $450/2 years, or $18.75/month for ~3TB. This works out as $0.00625/GB/month. Not bad, if you need the majority of the 3TB.

You’d still need to put this mini-server somewhere, and ideally it would be off-site. There’s a lot to consider, and it takes time to set up and configure. On the other hand, if you take out a basic hosting package you can expect to pay around $1/GB/month for storage, depending on the hosting provider. Some services for web hosting might offer ‘unlimited’ storage, but they generally state that this doesn’t cover use for private file storage and backup.

So if none of these approaches seem attractive, and you don’t already have a server with some spare storage, what other options are there? The obvious thing to do is to look at how services like Dropbox store their data, and cut out the middleman. In this case, we go to Amazon S3.

S3 is a much lower-level storage service than the consumer-targeted sync clients. It’s aimed mainly at developers to use as the backing store for their data-intensive apps and websites, and until recently has had very few user-friendly tools. s3sync is a tool designed to behave in the same way as rsync, but using S3 as a remote store instead of a standard file system. Once you have the files on S3, the web interface is very good, and allows you to browse the files and folders as you would expect.

If you can find tools you are happy with, S3 has one huge advantage, assuming you aren’t close to the upper end of one of the usage tiers on a subsidised service: it’s PAYG. This means if you’re storing 12GB, you pay $1.14, rather than the $10 you would need to pay Dropbox for anything over the free allowance. On the downside, you lose most of the cool features Dropbox provides. It’s possible to configure the S3 permissions in similar ways to Dropbox, including sending ‘authorized’ links to people (see below), but it’s nowhere near as user-friendly.

Sharing links

The ability to share files is essentially a form of social networking, and has driven adoption of many sharing services like weTrasfer - it’s free word-of-mouth marketing. Dropbox has one of the best implementations of this feature.

If you are providing your own storage, can you give someone a link to a file you want to share? In general, yes, but you need to set up your server to provide external access to those files. At its simplest, this could just be a web site with a directory index, maybe with a password protecting it. But there’s no limit really to how sophisticated you can make such sharing features, depending on how much time you want to spend setting up a functional file server.

If you are using S3 to host your files, the web interface allows you to make time-limited links to a single file, or to set permissions which allow the public or specific Amazon Web Services users to access the files.

Peer to peer

Finally, a new service which I read about only recently is BitTorrent Sync. If you only need the folder sync aspect of one of these file sharing services, and can rely on the number of copies on different systems being sufficient to safeguard the files, then this might be a good solution. It allows all the systems which wish to share files with each other to talk directly (most of the time) and swap the files without ever using a central server. It does rely on peers being online at the same time to swap files though. Of course, you could still use a copy of BTSync on your server to keep a public copy as described above, and this would also mean that a copy would always be available for peers to sync with.

BTSync even provides a way to give people a one-way sync of a folder, which is a bit like sending them a share link, although they would need to install the client software to allow them to receive the files.

Conclusions

After all that, I’m not sure I really have much in the way of a conclusion. Peer-to-peer sharing with BTSync seems to be the easiest way to get file sync working without needing your own server or paying for anything, and provides some features for sharing links with other people and collaborating on shared folders. On the down side, it doesn’t provide a simple web interface to download a shared file, and you would still need to set up a server to handle those kinds of features.

Out of the paid services, the best option depends on what you need.

If you want a backup service, Backblaze seems like the best option at the moment.
If you want a folder sync client for collaborative work, Dropbox is hard to beat, but expensive. BitTorrent Sync is very close, but missing the easy file sharing features which make Dropbox so popular. Sparkleshare seems like it could compete for collaboration, but requires server setup.
And if you just want to share files with people, Bitcasa is probably the best value, as its unlimited plan removes file size from consideration.

Of course, there’s nothing to stop you using more than one service, perhaps even with overlapping synchronisation folders. For example, you could use Bitcasa to store all your files on the server, and also create a BTSync folder inside it which would allow you to retrofit Bitcasa with the collaboration features it’s missing…