There are four times in my life I have suffered regrettable data loss incidents.
The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover everything.
The second was storing data in OneDrive - a change to their terms surrounding "lifetime" unlikely noted storage, combined with a client that was unusably slow to download and a deadline for data retrieval meant that I lost most of my files.
The third was SD card failure in digital camera on holiday, the controller chip died catastrophically, leaving the card completely unrecognised. It was a brand new Sony 128GB card, manufactured by Toshiba, and it seemed to be a common issue. I now shoot to two cards simultaneously.
And the fourth time was ... Performing a backup. An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup. I've been weary of using rewritable media for some time now as a consequence, but I think backups themselves are high risk activities.
>I've been weary of using rewritable media for some time now as a consequence
I briefly considered using bluray disks as a backup for my photos and other critical docs. But getting a decent bluray burner seems not so easy these days with most production winding down. Next best thing looks like the "object lock" feature on object store services that prevents deleting objects for a certain time.
I pretty much instantly wrote off physical media because I know I won't reliably move it to a separate location. If I'm going to have backups, I want ones that are not within fire/flood range of my computer.
In terms of preventing "oops" moments, I'm mainly relying on software (restic) for that, where I trust that (A) backups always append data rather than replacing and (B) it's logic works for marking which data to purge based on rules is accurate. [0]
You want to make sure the append only is enforced by the hosting provider rather than duplicity. Your backups should be resistant to having an attacker on the system who can run arbitrary commands with duplicity.
Right, defense against internal "oops" accidents looks rather different than defense against purposeful sabotage.
While the local backup service uses an API-key with limited privileges, I need to go back to the bucket/storage-zone settings and see what I can enable, how it would affect total storage, and whether I (with additional privileges) can reliably grab the correct mix of old-blobs to do a restore.
I kinda overdid it with a tape library (40 tapes capacity, LTO-9), but I'm using it for my company's AI model backups. _That_ setup was about $12k.
I had a smaller setup before, with a simple external LTO-5 drive. Used drives are now are about $300, and you can probably find them cheaper. And LTO-5 is the minimum realistic version, it's the first one that supports LTFS and it has reasonable tape capacity.
This is for backups. The threat here is that random ware uses your backup script to delete/encrypt the backups too. With object lock the api key the script has can only add data.
The probability that the object store goes out of business at the exact time your own copy dies is insignificant.
That only matters if it happens literally at the same time as a failure of your primary storage. If they increase the price you can just close your account and move the backup to another service.
The probability of a hard drive failure or ransomware at the same time as backblaze or aws going out of business is pretty much not worth thinking about.
The probability of an attacker using the api key in your backup script to destroy the backups is far more possible.
publlus_enigma · · focus · HN ↗
The first was when the telephone pole outside our house was struck directly by lightning. Not only was it the loudest thing I have ever heard, the current surged through the telephone line, into the internal fax modem, and fries everything within its vicinity. I was 10. I did have backuos, but only only floppy and they didn't cover everything.
The second was storing data in OneDrive - a change to their terms surrounding "lifetime" unlikely noted storage, combined with a client that was unusably slow to download and a deadline for data retrieval meant that I lost most of my files.
The third was SD card failure in digital camera on holiday, the controller chip died catastrophically, leaving the card completely unrecognised. It was a brand new Sony 128GB card, manufactured by Toshiba, and it seemed to be a common issue. I now shoot to two cards simultaneously.
And the fourth time was ... Performing a backup. An errant script deleted the source content, but I'd also deleted the existing backup to free up space for the new backup. I've been weary of using rewritable media for some time now as a consequence, but I think backups themselves are high risk activities.
Gigachad · · focus · HN ↗
I briefly considered using bluray disks as a backup for my photos and other critical docs. But getting a decent bluray burner seems not so easy these days with most production winding down. Next best thing looks like the "object lock" feature on object store services that prevents deleting objects for a certain time.
Terr_ · · focus · HN ↗
In terms of preventing "oops" moments, I'm mainly relying on software (restic) for that, where I trust that (A) backups always append data rather than replacing and (B) it's logic works for marking which data to purge based on rules is accurate. [0]
[0] <a href="https://restic.readthedocs.io/en/stable/060_forget.html#removing-snapshots-according-to-a-policy" rel="nofollow">https://restic.readthedocs.io/en/stable/060_forget.html#remo...
Gigachad · · focus · HN ↗
Terr_ · · focus · HN ↗
While the local backup service uses an API-key with limited privileges, I need to go back to the bucket/storage-zone settings and see what I can enable, how it would affect total storage, and whether I (with additional privileges) can reliably grab the correct mix of old-blobs to do a restore.
cyberax · · focus · HN ↗
On the other hand, LTO-5 drives are now pretty affordable. And each tape cartridge holds around 2Tb of data for about $20.
XorNot · · focus · HN ↗
cyberax · · focus · HN ↗
I had a smaller setup before, with a simple external LTO-5 drive. Used drives are now are about $300, and you can probably find them cheaper. And LTO-5 is the minimum realistic version, it's the first one that supports LTFS and it has reasonable tape capacity.
justsomehnguy · · focus · HN ↗
Lol, no. "As of today we are closed. Goodbye."
Especially considering what giving even a two weeks now considered "generous".
Gigachad · · focus · HN ↗
The probability that the object store goes out of business at the exact time your own copy dies is insignificant.
justsomehnguy · · focus · HN ↗
Or even fancier - just changing the price for the egress.
Gigachad · · focus · HN ↗
The probability of a hard drive failure or ransomware at the same time as backblaze or aws going out of business is pretty much not worth thinking about.
The probability of an attacker using the api key in your backup script to destroy the backups is far more possible.