I’ve recently had some discussions about backups with people who aren’t computer experts, so I decided to blog about this for the benefit of everyone. Note that this post will deliberately avoid issues that require great knowledge of computers. I have written other posts that will benefit experts.
Table of Contents
Essential Requirements
Everything that matters must be stored in at least 3 places. Every storage device will die eventually. Every backup will die eventually. If you have 2 backups then you are covered for the primary storage failing and the first backup failing. Note that I’m not saying “only have 2 backups” (I have many more) but 2 is the bare minimum.
Backups must be in multiple places. One way of losing data is if your house burns down, if that happens all backup devices stored there will be destroyed. You must have backups off-site. A good option is to have backup devices stored by trusted people (friends and relatives are often good options).
It must not be possible for one event to wipe out all backups. Some people use “cloud” backups, there are many ways of doing this with Dropbox, Google Drive, etc. Some of these even have free options for small amounts of storage, for example Google Drive appears to have 15G of free storage which is more than enough for all your best photos and all your financial records. The downside to cloud backups is that a computer criminal who gets access to your PC can wipe it and the backups. Cloud backup can be a part of a sensible backup strategy but it can’t be relied on (also see the paragraph about having at least 2 backups).
Backup Devices
USB flash “sticks” are cheap and easy to use. The quality of some of those devices isn’t too good, but the low price and small size means that you can buy more of them. It would be quite easy to buy 10 USB sticks for multiple copies of data.
Stores that sell office-supplies sell USB attached hard drives which are quite affordable now. It’s easy to buy a couple of those for backup use.
The cheapest option for backing up moderate amounts of data is to get a USB-SATA device. This connects to the PC by USB and has a cradle to accept a SATA hard drive. That allows you to buy cheap SATA disks for backups and even use older disks as backups.
With choosing backup devices consider the environment that they will be stored in. If you want to store a backup in the glove box of your car (which could be good when travelling) then a SD card or USB flash device would be a good choice because they are resistant to physical damage. Note that if you have no other options for off-site storage then the glove box of your car will probably survive if your house burns down.
Multiple Backups
It’s not uncommon for data corruption or mistakes to be discovered some time after it happens. Also in recent times there is a variety of malware that encrypts files and then demands a ransom payment for the decryption key.
To address these problems you should have older backups stored. It’s not uncommon in a corporate environment to have backups every day stored for a week, backups every week stored for a month, and monthly backups stored for some years.
For a home use scenario it’s more common to make backups every week or so and take backups to store off-site when it’s convenient.
Offsite Backups
One common form of off-site backup is to store backup devices at work. If you work in an office then you will probably have some space in a desk drawer for personal items. If you don’t work in an office but have a locker at work then that’s good for storage too, if there is high humidity then SD cards will survive better than hard drives. Make sure that you encrypt all data you store in such places or make sure that it’s not the secret data!
Banks have a variety of ways of storing items. Bank safe deposit boxes can be used for anything that fits and can fit hard drives. If you have a mortgage your bank might give you free storage of “papers” as part of the service (Commonwealth Bank of Australia used to offer that). A few USB sticks or SD cards in an envelope could fit the “papers” criteria. An accounting firm may also store documents for free for you.
If you put a backup on USB or SD storage in your waller then that can also be a good offsite backup. For most people losing data from disk is more common than losing their wallet.
A modern mobile phone can also be used for backing up data while travelling. For a few years I’ve been doing that. But note that you have to encrypt all data stored on a phone so an attacker who compromises your phone can’t steal it. In a typical phone configuration the mass storage area is much less protected than application data. Also note that customs and border control agents for some countries can compel you to provide the keys for encrypted data.
A friend suggested burying a backup device in a sealed plastic container filled with dessicant. That would survive your house burning down and in theory should work. I don’t know of anyone who’s tried it.
Testing
On occasion you should try to read the data from your backups and compare it to the original data. It sometimes happens that backups are discovered to be useless after years of operation.
Secret Data
Before starting a backup it’s worth considering which of the data is secret and which isn’t. Data that is secret needs to be treated differently and a mixture of secret and less secret data needs to be treated as if it’s all secret.
One category of secret data is financial data. If your accountant provides document storage then they can store that, generally your accountant will have all of your secret financial data anyway.
Passwords need to be kept secret but they are also very small. So making a written or printed copy of the passwords is part of a good backup strategy. There are options for backing up paper that don’t apply to data.
One category of data that is not secret is photos. Photos of holidays, friends, etc are generally not that secret and they can also comprise a large portion of the data volume that needs to be backed up. Apparently some people have a backup strategy for such photos that involves downloading from Facebook to restore, that will help with some problems but it’s not adequate overall. But any data that is on Facebook isn’t that secret and can be stored off-site without encryption.
Backup Corruption
With the amounts of data that are used nowadays the probability of data corruption is increasing. If you use any compression program with the data that is backed up (even data that can’t be compressed such as JPEGs) then errors will be detected when you extract the data. So if you have backup ZIP files on 2 hard drives and one of them gets corrupt you will easily be able to determine which one has the correct data.
Failing Systems – update 2016-08-22
When a system starts to fail it may limp along for years and work reasonably well, or it may totally fail soon. At the first sign of trouble you should immediately make a full backup to separate media. Use different media to your regular backups in case the data is corrupt so you don’t overwrite good backups with bad ones.
One traditional sign of problems has been hard drives that make unusual sounds. Modern drives are fairly quiet so this might not be loud enough to notice. Another sign is hard drives that take unusually large amounts of time to read data. If a drive has some problems it might read a sector hundreds or even thousands of times until it gets the data which dramatically reduces system performance. There are lots of other performance problems that can occur (system overheating, software misconfiguration, and others), most of which are correlated with potential data loss.
A modern SSD storage device (as used in a lot of the recent laptops) doesn’t tend to go slow when it nears the end of it’s life. It is more likely to just randomly fail entirely and then work again after a reboot. There are many causes of systems randomly hanging or crashing (of which overheating is common), but they are all correlated with data loss so a good backup is a good idea.
When in doubt make a backup.
Any Suggestions?
If you have any other ideas for backups by typical home users then please leave a comment. Don’t comment on expert issues though, I have other posts for that.
Nice summary Russell.
Another reason for having at least two backups: if you are using connected storage such as a USB drive, then during the time one is doing the backup, both the system and backup are vulnerable to power surges and fires and so on. So it is possible to lose both the main drive and a backup from a single event. Ideally the backup period should be kept short (ie, don’t leave removable backups running unattended if it can be helped).
Also on avoiding Facebook for photos/videos, is that the site lowers the quality of the images to save space.
Of course, some of this is a matter of risk/convenience trade-off. It is typically better to have risky backups than no backups at all, as long as one is not deluded into a false sense of security.
The most common backup failure I see is people leaving their photos stored (solely) on their camera/phone, and important documents stored (solely) on USB drives.
(another) nice post. Thanks.
Re: offsite backups. I tell people that a carton of eggs is not one egg and 11 backups. It’s important to consider how long it will take to restore data when deciding on off-site backups, and what happens if you don’t have identical hardware. i.e. if it will take 24 hours to restore all your data using backups online over a slow connection, then it might be an idea to break the backups into one small backup with the most important data, and another larger backup with lower priority data – that way you can be working quickly.
With encryption it’s important to backup at least your private key (your public key should be uploaded to keyservers/keybase already). Joey Hess has recently started a project that allows you to backup your private key with reasonable security using shards – there is also paper keys (a good thing to store at your bank). Some major security companies rent out lock boxes with 24/7 access.
If cloud storage is used for backups then the account user-name, email address, and password (even better, your entire password manager) must be backed up also. If dual authentication using your mobile phone is required (by dangerously naive provider policies) to access your cloud storage – then all the information required to obtain a replacement SIM card should also be considered – i.e. if flood or fire wipes out your computers it may also wipe out your ability to recover data in a useful time-frame.
Re: backup storage medium. microSSD cards *are* cheap.
It’s also important that your backups are versioned as you may not notice corruption of loss of data until many backups later
Glenn: one of the people who inspired this post had 5 years of photos on his phone and his wife’s phone. The phones were Samsung Galaxy S2 and due to ancient OS and lack of storage space got into a state where no new apps could be installed so it wasn’t possible to install a good file transfer app. Fortunately I could use adb, but on his own he would have had serious problems.
Scott: I agree that transfer speed is an issue for some situations, but for basic use I don’t think it’s much of a big deal. I know people who have systems in a failed state for weeks before they seek help. I’ll update the post with a mention of this.
Regarding backing up of keys, that’s strictly an expert issue.
Jason: Versioned backups is more of an expert issue. That’s why I suggest zip files, they preserve data and are easy to use. I use BTRFS for home systems and ZFS for work systems to avoid corruption, but it’s not something I can advise people to do.