The most reliable device for long term backups?

Morgan said

HP Z440, Xeon 1650? you should have enough PCIe lanes to slap some more NVME drives in at some point :).

It's a Xeon E5-1603 with 32GB of RAM.  Both the system and editing drives are NVME in PCI slots. The refurbished Z440 was a brilliant suggestion from Huw :-)

Can't remember if I already posted this here, but for everyone using or considering cloud storage, this might be of interest:

https://www.dpreview.com/opinion/1995639497/cloud-companies-may-lease-your-images-for-ai-learning-what-can-you-do-about-it

In another forum that I'm on there is a lot of talk about encryption.  Some companies don't offer it, Some do it as an extra and some include it in the subscription.  There seems to be a further division where some encryption keys are known to the provider and some are not.

Last weekend I had to find some old images, taken in 2012. Found one of my boxes of DVD’s and although the USB drive wouldn’t be read by my new Mac, the windows machine brought images up no problem. Don’t use this method with new image backup nowadays.

I would say investing in cloud storage is a must.

Gothic Image said

Morgan said

3rd option being cloud storage, for a consumer this is probably the easiest and cheapest, it wont offer much redundancy be it beyond whatever the cloud service offers you but it its definatly the easiest, MEGA is probably one of the cheapest with 8TB costing around £172 a year, 

Backblaze is $99 per computer per year for unlimited storage.  I'm currently 5TB through a 17TB upload thanks to my snazzy new toob symmetrical 900Mbps fibre connection.

If your internet connection is fast enough, that is quite an attractive deal. Sadly ATM our internet is bloody awful. But various companies have dug our street up several times in the last year or so. The nearest fibre conduit is within about two metres of our property. When fibre finally arrives here, I had thought of simply putting one of those big 20TB disks encrypted in a small Raspberry Pi-based machine in a cupboard in my mum's flat. Her new flat is a new-build and the broadband is superb. And of course once one has completed a whole rsync (or similar), then subsequent backups are incremental and only involve a very tiny part of our 17TB.

But from what you indicate, the cost of my proposed "Pi + big disk" setup could effectively buy five Backblaze years instead - with zero additional energy costs and zero maintenance. Even for someone as cloud-averse as I, that is quite an attractive option. Especially as it would only ever be just one limb of my overall backup strategy.

A few questions if I may, please...

1. What sort of sustainable upload speeds are you achieving, in terms of actual bytes (not bits) per second, averaged over a reasonable time period?
2. How exactly does Backblaze determine a user's number of computers? Number on your LAN (in our case dozens)? Or number that actually make a connection to it (in our case just the Media Server)?
3. If it is the latter, how does Backblaze actually identify that machine? I.e what would happen if I changed the system drive in the Media Server for example?
4. Is the data encrypted (really important for us)?

Sorry to be all questions, but I doubt I'm the only photographer here for whom this could offer an affordable and sustainable backup option.

waist.it said

If your internet connection is fast enough, that is quite an attractive deal. Sadly ATM our internet is bloody awful. But various companies have dug our street up several times in the last year or so. The nearest fibre conduit is within about two metres of our property. When fibre finally arrives here, I had thought of simply putting one of those big 20TB disks encrypted in a small Raspberry Pi-based machine in a cupboard in my mum's flat. Her new flat is a new-build and the broadband is superb. And of course once one has completed a whole rsync (or similar), then subsequent backups are incremental and only involve a very tiny part of our 17TB.

But from what you indicate, the cost of my proposed "Pi + big disk" setup could effectively buy five Backblaze years instead - with zero additional energy costs and zero maintenance. Even for someone as cloud-averse as I, that is quite an attractive option. Especially as it would only ever be just one limb of my overall backup strategy.

A few questions if I may, please...

1. What sort of sustainable upload speeds are you achieving, in terms of actual bytes (not bits) per second, averaged over a reasonable time period?
2. How exactly does Backblaze determine a user's number of computers? Number on your LAN (in our case dozens)? Or number that actually make a connection to it (in our case just the Media Server)?
3. If it is the latter, how does Backblaze actually identify that machine? I.e what would happen if I changed the system drive in the Media Server for example?
4. Is the data encrypted (really important for us)?

Sorry to be all questions, but I doubt I'm the only photographer here for whom this could offer an affordable and sustainable backup option.

Sorry I can't help with this specifically, but on another forum I'm on Backblaze is quite popular and from what I've read the initial upload can take a long time, but then it becomes incremental, so whether you have to have it running 24/7 or if you can let it update in the background when your computer is on, I'm not sure.

My computer guy is very anti leaving a computer running 24/7 especially if using HDDs, but the cloud services and other organisations must have computers ruuning 24/7 all of the time.

waist.it said

Gothic Image said

Morgan said

3rd option being cloud storage, for a consumer this is probably the easiest and cheapest, it wont offer much redundancy be it beyond whatever the cloud service offers you but it its definatly the easiest, MEGA is probably one of the cheapest with 8TB costing around £172 a year, 

Backblaze is $99 per computer per year for unlimited storage.  I'm currently 5TB through a 17TB upload thanks to my snazzy new toob symmetrical 900Mbps fibre connection.

If your internet connection is fast enough, that is quite an attractive deal. Sadly ATM our internet is bloody awful. But various companies have dug our street up several times in the last year or so. The nearest fibre conduit is within about two metres of our property. When fibre finally arrives here, I had thought of simply putting one of those big 20TB disks encrypted in a small Raspberry Pi-based machine in a cupboard in my mum's flat. Her new flat is a new-build and the broadband is superb. And of course once one has completed a whole rsync (or similar), then subsequent backups are incremental and only involve a very tiny part of our 17TB.

But from what you indicate, the cost of my proposed "Pi + big disk" setup could effectively buy five Backblaze years instead - with zero additional energy costs and zero maintenance. Even for someone as cloud-averse as I, that is quite an attractive option. Especially as it would only ever be just one limb of my overall backup strategy.

A few questions if I may, please...

1. What sort of sustainable upload speeds are you achieving, in terms of actual bytes (not bits) per second, averaged over a reasonable time period?
2. How exactly does Backblaze determine a user's number of computers? Number on your LAN (in our case dozens)? Or number that actually make a connection to it (in our case just the Media Server)?
3. If it is the latter, how does Backblaze actually identify that machine? I.e what would happen if I changed the system drive in the Media Server for example?
4. Is the data encrypted (really important for us)?

Sorry to be all questions, but I doubt I'm the only photographer here for whom this could offer an affordable and sustainable backup option.

 
Long-time Backblaze user here.

• At my typically 60Mbit upload speed (5G router) in theory Backblaze will compress and upload 1TB every 2 days or so, running continuously. I recently backed up 10TB (on and off during the day rather than continuous backup, as it's just my laptop and gets frequently disconnected from the drives it's backing up during the day) and it easily finished the 10TB across 3 weeks or so. At fibre speeds I'd expect a 10x reduction in backup time.
• Each install of the Backblaze software is tied to a specific machine and licence fee. You can connect as many external drives as you like to that machine, no issues. I don't think it includes network drives though, just physically connected drives.
• If you change the drive or machine of a licensed install, you follow a simple process to reconnect the new machine/drive to your existing remote backup data inventory without hiccups, so it picks up where the old one left off/
• The data is rigorously encrypted, automatically. You can override the default encryption keys with your own.

Hope this helps.

Edited by brewster

Dabhand16 said

waist.it said

If your internet connection is fast enough, that is quite an attractive deal. Sadly ATM our internet is bloody awful. But various companies have dug our street up several times in the last year or so. The nearest fibre conduit is within about two metres of our property. When fibre finally arrives here, I had thought of simply putting one of those big 20TB disks encrypted in a small Raspberry Pi-based machine in a cupboard in my mum's flat. Her new flat is a new-build and the broadband is superb. And of course once one has completed a whole rsync (or similar), then subsequent backups are incremental and only involve a very tiny part of our 17TB.

But from what you indicate, the cost of my proposed "Pi + big disk" setup could effectively buy five Backblaze years instead - with zero additional energy costs and zero maintenance. Even for someone as cloud-averse as I, that is quite an attractive option. Especially as it would only ever be just one limb of my overall backup strategy.

A few questions if I may, please...

1. What sort of sustainable upload speeds are you achieving, in terms of actual bytes (not bits) per second, averaged over a reasonable time period?
2. How exactly does Backblaze determine a user's number of computers? Number on your LAN (in our case dozens)? Or number that actually make a connection to it (in our case just the Media Server)?
3. If it is the latter, how does Backblaze actually identify that machine? I.e what would happen if I changed the system drive in the Media Server for example?
4. Is the data encrypted (really important for us)?

Sorry to be all questions, but I doubt I'm the only photographer here for whom this could offer an affordable and sustainable backup option.

Sorry I can't help with this specifically, but on another forum I'm on Backblaze is quite popular and from what I've read the initial upload can take a long time, but then it becomes incremental, so whether you have to have it running 24/7 or if you can let it update in the background when your computer is on, I'm not sure.

My computer guy is very anti leaving a computer running 24/7 especially if using HDDs, but the cloud services and other organisations must have computers ruuning 24/7 all of the time.

Thanks for the info.

On the subject of 24/7 operation, the 20TB encrypted data disk in my Media Server is on 24/7. It's a so-called "enterprise" grade disk and designed to be left running 24/7. Disk average temperature is ranges between about 36°C to 43°C - well within spec. And such drives generally seem to last indefinitely in that sort of environment. Or at least they last well beyond the 5-year warranty period, and are only replaced and redeployed elsewhere because they have become too small.

Main thing I have against the 24/7 operation is energy costs. Granted it's relatively small, but it soon adds up, and has to be paid for!

brewster said

waist.it said

Gothic Image said

Morgan said

3rd option being cloud storage, for a consumer this is probably the easiest and cheapest, it wont offer much redundancy be it beyond whatever the cloud service offers you but it its definatly the easiest, MEGA is probably one of the cheapest with 8TB costing around £172 a year, 

Backblaze is $99 per computer per year for unlimited storage.  I'm currently 5TB through a 17TB upload thanks to my snazzy new toob symmetrical 900Mbps fibre connection.

If your internet connection is fast enough, that is quite an attractive deal. Sadly ATM our internet is bloody awful. But various companies have dug our street up several times in the last year or so. The nearest fibre conduit is within about two metres of our property. When fibre finally arrives here, I had thought of simply putting one of those big 20TB disks encrypted in a small Raspberry Pi-based machine in a cupboard in my mum's flat. Her new flat is a new-build and the broadband is superb. And of course once one has completed a whole rsync (or similar), then subsequent backups are incremental and only involve a very tiny part of our 17TB.

But from what you indicate, the cost of my proposed "Pi + big disk" setup could effectively buy five Backblaze years instead - with zero additional energy costs and zero maintenance. Even for someone as cloud-averse as I, that is quite an attractive option. Especially as it would only ever be just one limb of my overall backup strategy.

A few questions if I may, please...

1. What sort of sustainable upload speeds are you achieving, in terms of actual bytes (not bits) per second, averaged over a reasonable time period?
2. How exactly does Backblaze determine a user's number of computers? Number on your LAN (in our case dozens)? Or number that actually make a connection to it (in our case just the Media Server)?
3. If it is the latter, how does Backblaze actually identify that machine? I.e what would happen if I changed the system drive in the Media Server for example?
4. Is the data encrypted (really important for us)?

Sorry to be all questions, but I doubt I'm the only photographer here for whom this could offer an affordable and sustainable backup option.

 
Long-time Backblaze user here.

• At my typically 60Mbit upload speed (5G router) in theory Backblaze will compress and upload 1TB every 2 days or so, running continuously. I recently backed up 10TB (on and off during the day rather than continuous backup, as it's just my laptop and gets frequently disconnected from the drives it's backing up during the day) and it easily finished the 10TB across 3 weeks or so. At fibre speeds I'd expect a 10x reduction in backup time.
• Each install of the Backblaze software is tied to a specific machine and licence fee. You can connect as many external drives as you like to that machine, no issues. I don't think it includes network drives though, just physically connected drives.
• If you change the drive or machine of a licensed install, you follow a simple process to reconnect the new machine/drive to your existing remote backup data inventory without hiccups, so it picks up where the old one left off/
• The data is rigorously encrypted, automatically. You can override the default encryption keys with your own.

Hope this helps.

Edited by brewster

Thanks, brewster That's what I really needed to know. If it took 2-3 weeks to do the first rsync, then say ten to fifteen minutes to do the subsequent incrementals, then we'd be very happy indeed with that. :-)

Just to focus on the "network drive" issue for a moment, if I may: in Linux we generally don't use "network drives" as such. Well OK, you can mount remote drives if you wish. But these days, for this sort of task, we simply rsync over ssh. The connection is made, encrypted and broken again, on the fly.

So unless Backblaze forces its users to install some intrusive logging type client, which I don't think it does, I'd imagine that Backblaze would never actually know if other machines had been rsync'd to the authorised machine, or not, would it? By the time the Backblaze connection occurred, all the other machines would already have disconnected and vanished to the Ether? Or am I missing something obvious here?

Your question re: Linux, rsync and ssh is outside my pay grade, alas! All-Mac here. But Backblaze has a great FAQs and docs section.

Probably worth adding:

• I have found Backblaze to be fit-and-forget
• The incrementals are hourly (or as defined by you), invisible to the user if on a non-server machine; and when I check in at an hotel or airbnb in another country with my laptop, the backup just carries on unabashed (obviously excluding any stay-at-home drives)
• Good reporting built in (issues with backup, files in the queue, etc)
• The restoration of backup data really works (have tested it to a few TB); they'll even post you 8TB drives for recovery if your download's too slow
• You can leave backed-up drives unconnected for as long as you like with the Forever upgrade (i.e., it doesn't discard backups of drives that it hasn't seen for a while)
• In a pinch, you can download backed-up files you select onto your phone.

Another happy Backblaze customer here.
As has already been mentioned, once you have done the initial upload (about 1 Tb in my case), you simply don't notice the incremental updates - I'm on quite an old PC, but I've not noticed any slowdown as a result of using Backblaze (I do have a decent connection, wired 1Gb network to router, 50Mb upload on VM).

If you turn off your PC and go away for a 2 week holiday, you will get back to find a few "Your backup hasn't happened" emails from them :)

brewster said

Your question re: Linux, rsync and ssh is outside my pay grade, alas! All-Mac here. But Backblaze has a great FAQs and docs section.

Probably worth adding:

• I have found Backblaze to be fit-and-forget
• The incrementals are hourly (or as defined by you), invisible to the user if on a non-server machine; and when I check in at an hotel or airbnb in another country with my laptop, the backup just carries on unabashed (obviously excluding any stay-at-home drives)
• Good reporting built in (issues with backup, files in the queue, etc)
• The restoration of backup data really works (have tested it to a few TB); they'll even post you 8TB drives for recovery if your download's too slow
• You can leave backed-up drives unconnected for as long as you like with the Forever upgrade (i.e., it doesn't discard backups of drives that it hasn't seen for a while)
• In a pinch, you can download backed-up files you select onto your phone.

Thank you. I think my point is that in my configuration, BB would never see any backed-up network drives at all. Merely one big, single local 20TB drive on the one authorised machine.

I guess in order to see how that pans out, I really need to create a Backblaze account and go have a play. Thanks again for the info. Much appreciated. :-)

Jonathan C said

Another happy Backblaze customer here.
As has already been mentioned, once you have done the initial upload (about 1 Tb in my case), you simply don't notice the incremental updates - I'm on quite an old PC, but I've not noticed any slowdown as a result of using Backblaze (I do have a decent connection, wired 1Gb network to router, 50Mb upload on VM).

If you turn off your PC and go away for a 2 week holiday, you will get back to find a few "Your backup hasn't happened" emails from them :)

Thank you. I can certainly see the benefits. Sadly ATM our broadband is bloody awful. With our current upload speed, I calculate it would take over two years to do the first backup - and that's assuming it doesn't rain! lol  But we're hoping that will change here quite soon. Hence my renewed interest in services such as BB.

However, the answer I'm really trying to tease-out here is if one has a "master" computer, with a very large single data disk (say 20TB or 22TB), which is a culmination of lots of backups and archives from lots of other machines, does BB regard this as one machine? And charge for one machine? Or does BB have some means of determining that this is really a multiple backup of lots of machines? And charge for every machine that has backed up to the master?

Edited by waist.it

From where I'm sitting, an agglomeration of files brought in from other devices is still backing up only one machine. I'd have thought you'd be fine.

With our current upload speed, I calculate it would take over two years to do the first backup…

Dunno what your budget will run to, but do look into Backblaze's Fireball programme.

Edited by brewster

waist.it said

Jonathan C said

Another happy Backblaze customer here.
As has already been mentioned, once you have done the initial upload (about 1 Tb in my case), you simply don't notice the incremental updates - I'm on quite an old PC, but I've not noticed any slowdown as a result of using Backblaze (I do have a decent connection, wired 1Gb network to router, 50Mb upload on VM).

If you turn off your PC and go away for a 2 week holiday, you will get back to find a few "Your backup hasn't happened" emails from them :)

Thank you. I can certainly see the benefits. Sadly ATM our broadband is bloody awful. With our current upload speed, I calculate it would take over two years to do the first backup - and that's assuming it doesn't rain! lol  But we're hoping that will change here quite soon. Hence my renewed interest in services such as BB.

However, the answer I'm really trying to tease-out here is if one has a "master" computer, with a very large single data disk (say 20TB or 22TB), which is a culmination of lots of backups and archives from lots of other machines, does BB regard this as one machine? And charge for one machine? Or does BB have some means of determining that this is really a multiple backup of lots of machines? And charge for every machine that has backed up to the master?

Edited by waist.it

As far as I can tell, 'one computer' is a particular machine, with as many drives as you want in or connected to it.

So in the case you give, that should count as just one machine.

You could also take backups of a bunch of machines on removable drives, and then hook those removable drives to the Backblaze PC and set it to back them up, but you might find you get nag emails when it spots they're not there.