- Joined
- Apr 26, 2010
- Messages
- 31,262
- Reaction score
- 13,900
- Points
- 26,065
- Age
- 34
- Location
- Norkem Park, Gauteng
Firstly, this applies to absolutely critical, irreplaceable data such as company databases, university theses, etc. The type of data where loss would be career or life changing. Don't comment that this is overkill for the video of your cat you're editing for YouTube, that's not critical. Your save-game with 2,500 hours of gameplay is not that important either. If it so happens that either ARE critical for you, continue. If you have data that is so important your life would fall apart without it (almost all of us do, think really hard before you say you don't), this is for you. Think carefully now, a LOT of your data isn't nearly as valuable as you think it is, but some is far more valuable than you think. I've seen everything from relationships ending to companies ceasing to exist.
I've lost count of the number of times I've had someone come to me with a failed hard drive containing their only copy of irreplaceable data. Everything from databases of people that owe their company money to the only images of a deceased relative. The data is normally worth 10x what data recovery would cost, but the people don't necessarily have the money at hand. In the case of pictures they could always be recovered at a later date, but when their company info gets lost they could be out of business within a few days should they not be able to retrieve the info instantly. Even if they do have the money at hand, the time taken for data recovery might be enough to put them out of business.
Defining a backup and archive
Let's get something out of the way. When it comes to absolutely mission critical information, your first copy is NOT a backup. Your second copy is NOT a backup, it is a second copy. RAID is NOT a backup, it is a means of minimizing downtime. Filing system corruption, an accidental delete or overwrite or controller failure could be the end of your file or even the entire array, hence it is not a backup. It is to minimize downtime while you would normally be replacing the failed drive and restoring from your backup. Your third copy is your FIRST backup. Your fourth copy is your SECOND backup. Your backups are NOT your archives. Your fifth copy can be considered an archive, but nothing before that.
Good backup habits
So what does this mean, you should make five copies of the data in one folder? Absolutely. If you want to have a bad time.
The first thing to do is get into a good backup habit. Pick a time each day (or week, but keep in mind that weekly backups could mean you lose up to six days worth of data) and make sure you do that backup come hell or high water. If you miss the time you've set, don't leave it until that time the following day (or week), do it the second you can.
Another good practice when working on critical documents is to save new files after each revision instead of overwriting the old copy. In the event of losing power during a save it could corrupt your file, and user error can also play a part. You may have deleted a page close to the beginning of your document thinking it's no longer needed, moved on to work on the end of your unfinished document, saved and closed Work and then realized there was one paragraph you needed from your deleted page. With multiple revisions you can open the previous revision and grab the text you need. Mark your important files as read-only, the program you're using to edit them will then insist you give the file a new name should you accidentally hit Save instead of Save As.
Which media should you use for the backup?
Now onto the backup media. Hard drives or SSDs are a preferred storage medium for your first and second copy. Your backups should be on external hard drives, store them on opposite ends of the house and preferably in fireproof safe(s) if possible. Flash drives should never be considered a backup, as they are far to easy to lose and they fail more often than most other media. An off-site backup should not be considered optional, just do it. Be it at a trusted friend's house, a safety deposit box or even your car, anywhere is better than nowhere. Whatever you do, make sure that at least one of your backup devices is disconnected from your computer/network when not in use, as ransomware can spread over a network.
Cloud backup solutions that can store multiple revisions of files are recommended as something such as ransomware could affect ALL of your backups. I can already hear you moaning about slow upload speeds and the cost of bandwidth, but the uploads can be done overnight or while you're at work. Bandwidth costs are lower than the cost of losing mission critical data. Cloud storage is not something that should be relied on as an only backup by ANY means, as if the provider reckons you're doing something against their terms of service they can delete your data. Mweb's hosting solutions also proved that human error can happen anywhere (for those who missed it, thousands of customers had their servers and all backups wiped). If you consider your data worth little enough to rely on cloud storage as your only backup, by all means. Just remember that cloud storage is a backup that is completely out of your hands.
Archiving information/long term storage
Now that we have two hot copies, one on-site backup, one off-site backup and a cloud solution, do we still need to archive? HELL YES. You never know what could happen. There could be any number of reasons for needing the data 20 years from now, by which time your backups are probably long gone. While hard drives and SSDs are great for storage and backups, they are NOT suitable for archiving as SSDs only retain the data for as short a period as a year or two after being powered off (in theory they can do about a decade, but we all know that theory and practice are hardly ever the same thing) and hard drives closer to five years (magnetic field breakdown). A flipped bit in a BMP means you get a pixel that's the wrong colour, in an encrypted file it means you lose the file. Of course, dropping a drive could also render the drive unreadable and needing to be sent for the data recovery this entire article is trying to avoid.
There's only one reliable option for long term data storage, and that is... The DVD. Yes, that archaic media is actually still very much relevant, enough so that I'm about to start using them again. Magnetic (even including tapes) and solid state NAND media cannot stand the test of time the way optical media can. Again, two copies minimum, kept in different parts of the house (consider the garage or attic). The media is cheap enough to keep many copies, and the more the merrier.
If 4.4 GB isn't enough to store a file that you need archived, use BluRay or create a spanned archive to split the file into chunks of 4.4 GB or less. If you choose the spanning method, make sure you have several copies - I wouldn't be comfortable with only two. If any single disc from the spanned set has the slightest unrecoverable read error it will render that entire file unusable.
As for brands, stay away from your cheap, no name brands such as Maxell, Imation, E-Lotus, etc. I'm even going to throw TDK into that grouping. Slightly better options are Verbatim and Sony, but if you can find *genuine* Taiyo Yuden (commonly sold as professional-grade JVC but there are plenty of fakes around, they're identifiable by a ring on the recording surface as seen here, discs without that ring are guaranteed fakes) you've got a winner - I've had mass failure with all other brands. Steer clear of DVD-R as DVD+R has far better error handling; unless you're burning videos to play on an old set top DVD player (or a newer one that only supports DVD-R, although I don't know of any) there is no reason to use DVD-R as it is a far worse technology.
Why I'm recommending DVD even though I know you've had plenty fail
There are a few reasons for this. Firstly, you're using consumer grade discs - Verbatim, Sony, TDK, etc. These discs are MUCH less rugged than professional grade DVDs. Secondly, chances are they're DVDs you handle a lot, not discs that have been packed away for archival purposes. Thirdly, DVDs are cheap. You can get the Taiyo Yuden discs at Amazon for $ 43.90/100 - that's a cost of R 552 for 100 of the highest quality writable DVD available to the public. The fourth and final reason is optical media is the only media which you have any hope of reading after it has been left in storage for a few decades - magnetic and NAND storage both lose their data over time.
Worst case scenario, hardware failure and data loss
If things do go wrong and you end up with an only copy on a dead drive, don't try to repair it yourself. Chances are very high that you will make what would have been an expensive recovery an incredibly expensive recovery, if even possible. If the drive is still working, decent recovery software could be your cue card but if the drive has failed, be it ticking, not spinning up, not being detected, etc, let the professionals sort it out before you mess things up further.
Despite what you may read online, freezing the drive is a terrible idea. Everyone mentions insulating the drive to prevent condensation buildup on the electronics, but that doesn't help with the air inside the drive. When that condenses on the heads it starts causing corrosion, which will then literally grind the data off of the media. The data recovery agents can see if a drive has been frozen from the obvious signs of corrosion, and recovery is often impossible.
Do not bang or drop the drive. Unless you've already established that professional recovery is out of the question and you know for sure that you'll never send the drive in at a later stage, banging the drive is a bad idea. There are plenty of amateurs giving advice on how to repair a drive by banging it until the heads become "unstuck." If your banging causes a head crash, the heads will grind the data away and render recovery impossible.
Do not open the drive. No matter how clean you think your room is, the air is far dirtier than a professional cleanroom which contains under around 100,000 particles per cubic meter of air. That sounds dirty, right? Your room air contains around 1,000,000,000 particles per cubic meter - the cleanroom is 10,000 times cleaner than your room. Those particles are bad news for drives, the distance between the read/write head and the platter is less than the size of a single particle of cigarette smoke. The internet being what it is, you'll find people recommending that you open a drive and give the platters a spin, do not do this. While there is a chance that your drive won't fail, this is once again an absolute last resort.
If you've already ruled out using a data recovery agency at any stage and you're going to throw the drive away there's no harm in trying the banging, freezing or opening methods but as I've said these are to be kept as ABSOLUTE LAST MEASURES.
I've lost count of the number of times I've had someone come to me with a failed hard drive containing their only copy of irreplaceable data. Everything from databases of people that owe their company money to the only images of a deceased relative. The data is normally worth 10x what data recovery would cost, but the people don't necessarily have the money at hand. In the case of pictures they could always be recovered at a later date, but when their company info gets lost they could be out of business within a few days should they not be able to retrieve the info instantly. Even if they do have the money at hand, the time taken for data recovery might be enough to put them out of business.
Defining a backup and archive
Let's get something out of the way. When it comes to absolutely mission critical information, your first copy is NOT a backup. Your second copy is NOT a backup, it is a second copy. RAID is NOT a backup, it is a means of minimizing downtime. Filing system corruption, an accidental delete or overwrite or controller failure could be the end of your file or even the entire array, hence it is not a backup. It is to minimize downtime while you would normally be replacing the failed drive and restoring from your backup. Your third copy is your FIRST backup. Your fourth copy is your SECOND backup. Your backups are NOT your archives. Your fifth copy can be considered an archive, but nothing before that.
Good backup habits
So what does this mean, you should make five copies of the data in one folder? Absolutely. If you want to have a bad time.
The first thing to do is get into a good backup habit. Pick a time each day (or week, but keep in mind that weekly backups could mean you lose up to six days worth of data) and make sure you do that backup come hell or high water. If you miss the time you've set, don't leave it until that time the following day (or week), do it the second you can.
Another good practice when working on critical documents is to save new files after each revision instead of overwriting the old copy. In the event of losing power during a save it could corrupt your file, and user error can also play a part. You may have deleted a page close to the beginning of your document thinking it's no longer needed, moved on to work on the end of your unfinished document, saved and closed Work and then realized there was one paragraph you needed from your deleted page. With multiple revisions you can open the previous revision and grab the text you need. Mark your important files as read-only, the program you're using to edit them will then insist you give the file a new name should you accidentally hit Save instead of Save As.
Which media should you use for the backup?
Now onto the backup media. Hard drives or SSDs are a preferred storage medium for your first and second copy. Your backups should be on external hard drives, store them on opposite ends of the house and preferably in fireproof safe(s) if possible. Flash drives should never be considered a backup, as they are far to easy to lose and they fail more often than most other media. An off-site backup should not be considered optional, just do it. Be it at a trusted friend's house, a safety deposit box or even your car, anywhere is better than nowhere. Whatever you do, make sure that at least one of your backup devices is disconnected from your computer/network when not in use, as ransomware can spread over a network.
Cloud backup solutions that can store multiple revisions of files are recommended as something such as ransomware could affect ALL of your backups. I can already hear you moaning about slow upload speeds and the cost of bandwidth, but the uploads can be done overnight or while you're at work. Bandwidth costs are lower than the cost of losing mission critical data. Cloud storage is not something that should be relied on as an only backup by ANY means, as if the provider reckons you're doing something against their terms of service they can delete your data. Mweb's hosting solutions also proved that human error can happen anywhere (for those who missed it, thousands of customers had their servers and all backups wiped). If you consider your data worth little enough to rely on cloud storage as your only backup, by all means. Just remember that cloud storage is a backup that is completely out of your hands.
Archiving information/long term storage
Now that we have two hot copies, one on-site backup, one off-site backup and a cloud solution, do we still need to archive? HELL YES. You never know what could happen. There could be any number of reasons for needing the data 20 years from now, by which time your backups are probably long gone. While hard drives and SSDs are great for storage and backups, they are NOT suitable for archiving as SSDs only retain the data for as short a period as a year or two after being powered off (in theory they can do about a decade, but we all know that theory and practice are hardly ever the same thing) and hard drives closer to five years (magnetic field breakdown). A flipped bit in a BMP means you get a pixel that's the wrong colour, in an encrypted file it means you lose the file. Of course, dropping a drive could also render the drive unreadable and needing to be sent for the data recovery this entire article is trying to avoid.
There's only one reliable option for long term data storage, and that is... The DVD. Yes, that archaic media is actually still very much relevant, enough so that I'm about to start using them again. Magnetic (even including tapes) and solid state NAND media cannot stand the test of time the way optical media can. Again, two copies minimum, kept in different parts of the house (consider the garage or attic). The media is cheap enough to keep many copies, and the more the merrier.
If 4.4 GB isn't enough to store a file that you need archived, use BluRay or create a spanned archive to split the file into chunks of 4.4 GB or less. If you choose the spanning method, make sure you have several copies - I wouldn't be comfortable with only two. If any single disc from the spanned set has the slightest unrecoverable read error it will render that entire file unusable.
As for brands, stay away from your cheap, no name brands such as Maxell, Imation, E-Lotus, etc. I'm even going to throw TDK into that grouping. Slightly better options are Verbatim and Sony, but if you can find *genuine* Taiyo Yuden (commonly sold as professional-grade JVC but there are plenty of fakes around, they're identifiable by a ring on the recording surface as seen here, discs without that ring are guaranteed fakes) you've got a winner - I've had mass failure with all other brands. Steer clear of DVD-R as DVD+R has far better error handling; unless you're burning videos to play on an old set top DVD player (or a newer one that only supports DVD-R, although I don't know of any) there is no reason to use DVD-R as it is a far worse technology.
Why I'm recommending DVD even though I know you've had plenty fail
There are a few reasons for this. Firstly, you're using consumer grade discs - Verbatim, Sony, TDK, etc. These discs are MUCH less rugged than professional grade DVDs. Secondly, chances are they're DVDs you handle a lot, not discs that have been packed away for archival purposes. Thirdly, DVDs are cheap. You can get the Taiyo Yuden discs at Amazon for $ 43.90/100 - that's a cost of R 552 for 100 of the highest quality writable DVD available to the public. The fourth and final reason is optical media is the only media which you have any hope of reading after it has been left in storage for a few decades - magnetic and NAND storage both lose their data over time.
Worst case scenario, hardware failure and data loss
If things do go wrong and you end up with an only copy on a dead drive, don't try to repair it yourself. Chances are very high that you will make what would have been an expensive recovery an incredibly expensive recovery, if even possible. If the drive is still working, decent recovery software could be your cue card but if the drive has failed, be it ticking, not spinning up, not being detected, etc, let the professionals sort it out before you mess things up further.
Despite what you may read online, freezing the drive is a terrible idea. Everyone mentions insulating the drive to prevent condensation buildup on the electronics, but that doesn't help with the air inside the drive. When that condenses on the heads it starts causing corrosion, which will then literally grind the data off of the media. The data recovery agents can see if a drive has been frozen from the obvious signs of corrosion, and recovery is often impossible.
Do not bang or drop the drive. Unless you've already established that professional recovery is out of the question and you know for sure that you'll never send the drive in at a later stage, banging the drive is a bad idea. There are plenty of amateurs giving advice on how to repair a drive by banging it until the heads become "unstuck." If your banging causes a head crash, the heads will grind the data away and render recovery impossible.
Do not open the drive. No matter how clean you think your room is, the air is far dirtier than a professional cleanroom which contains under around 100,000 particles per cubic meter of air. That sounds dirty, right? Your room air contains around 1,000,000,000 particles per cubic meter - the cleanroom is 10,000 times cleaner than your room. Those particles are bad news for drives, the distance between the read/write head and the platter is less than the size of a single particle of cigarette smoke. The internet being what it is, you'll find people recommending that you open a drive and give the platters a spin, do not do this. While there is a chance that your drive won't fail, this is once again an absolute last resort.
If you've already ruled out using a data recovery agency at any stage and you're going to throw the drive away there's no harm in trying the banging, freezing or opening methods but as I've said these are to be kept as ABSOLUTE LAST MEASURES.
