Storing 100TB in an RV

Tue Nov 23 2021

This week we picked up another 14TB Easystore drive in the sales, bringing our total to over 100TB of data storage in the RV - here's how we do it!

Storing 100TB in an RV
As an Amazon Associate we earn from qualifying purchases.

We have a lot of data in the RV. Typically large datasets like our documents, personal media collection and even my digitized version of all my scanned film negatives pale in comparison to our photos and particularly video footage for our YouTube channel.

Every photo we take on our Sony a7R III is over 80MB, and 4K video footage is over 600MB per minute! While we could purge old footage, we have chosen to hold onto much of it as a record and in case we ever need it. And of course, it all needs backing up.

Camera Gear

Want to know what gear we use for our photos and videos? We share it all!

Camera Gear

Safely storing this much data in our RV is a challenge - after all, we have constraints on power, physical space and budget.

Why no NAS?

Oftentimes if you need a lot of storage, a good solution is a NAS - Network Attached Storage. Companies like Synology and QNap make consumer-friendly devices that are essentially a low power computer - you just add disks.

Honestly, NASs are great - we used to have one (the DS412+ that I had has long since been discontinued, but the DS920+ is similar)! In our house, this was great - highly available reliable storage accessible from wherever we were, at home or remote.

But when we moved into the RV, I gave the NAS to my parents where it's been running 24/7 ever since - getting it from the US to the UK on a plane along with the hard drives was fun!

So why didn't we keep it in the RV? Well, really five reasons:

  1. Size - a NAS is pretty compact in the grand scheme of things, but still a relatively large device in a small RV.
  2. Power - a NAS needs power to run, and we didn't want to leave it running 24/7 when off-grid.
  3. Vibration - magnetic drives (aka spinning rust) really don't like vibration when they're powered up, and driving down the road is a lot of vibration! Particularly bad if we forget to turn off the NAS.
  4. Portability - if we want to take our laptops and work in a coffee shop, we can't just bring the NAS with us.
  5. Expansion - we knew our storage demands would grow as we shoot more footage, and while a NAS can support some expansion, it's not unlimited.

When we have built our new house, we'll have some network storage again - although probably something more DIY than an off-the-shelf NAS.

External USB drives

Instead of a NAS, we elected to use external USB drives. Most of our data is on one of around seventeen (!!) WD 4TB My Passport portable drives.

WD 4TB My Passport Drives
These brightly colored 4TB drives are the mainstay of our data storage system.

They're not going to win any performance awards - they use SMR and run at 5,400rpm. However, they're relatively affordable, compact and so far have been very reliable - we haven't had a single drive failure in the 3 years we've been on the road.

We also have two monster WD 14TB Easystore drives - one that we bought in the Black Friday sales last year, and one that we bought in this year's sales. These are less convenient than the smaller drives - they're significantly larger and require external power. But the price per TB is much lower than the 4TB drives, so they're great for archive data.

Our Intel NUC serves as our home server, managing our home automation system and media library among other duties. While much of the media is on a 4TB drive that I unplug when we travel, a lot of our data logging is handled with a 2TB SanDisk Extreme SSD drive connected via USB.

Smart RV Tour

A tour of our DIY smart RV, packed full of home automation technology that makes our full-time RV life more comfortable, secure and efficient!

Smart RV Tour

For the ultimate in portable performance, Diana has a 1TB Samsung T5 SSD drive. We each both have a handful of old external drives of varying capacities that we've accumulated over the years.

Internal SSD drives

All three of our laptops use internal m.2 NVMe SSD drives for superfast performance. I specced my 13" MacBook Pro with a 1TB internal drive since it can't be changed later, and Diana's Lenovo Carbon X1 is also just using the drive it came with.

But for Diana's video editing laptop, a custom-ordered Lenovo P53, we got a little more advanced. It came with a 1TB NVMe drive pre-installed, and that's used as the OS disk. We then added two more m.2 NVMe drives - a high performance 1TB Samsung 970 PRO SSD that's used as a scratch drive for video editing, and a 2TB Sabrent Rocket SSD for storing media files for the current project.

4TB of m.2 NVMe storage, 80GB of memory (16GB pre-installed plus 2x 32GB Samsung memory modules we added), an 8-core Intel Core i9-9880H CPU and an Nvidia Quadro RTX 4000 GPU make it a beast of a laptop!

When we first built the Intel NUC to run our home automation system, I used a 250GB Samsung 970 EVO m.2 NVMe drive. This worked great, but as we found more and more use cases for the NUC, it was time for an upgrade - I chose a 1TB Samsung 980 drive to replace it. This is the OS drive but also stores most of the VM disks that we manage through Proxmox too.

Running Home Assistant in an RV

Our Smart RV would be nothing without Home Assistant. I'll share why we chose Home Assistant and how we run it in our RV.

Running Home Assistant in an RV

Not wanting the 250GB drive to go to waste, I also picked up an m.2 NVMe enclosure so we can now use it as a super-fast transfer drive for moving files around.

Online backups

We've talked before about the importance of good backups - and following the so-called 3-2-1 backup rule.

Backing Up

This week we've been settling back into our normal routine, but World Backup Day is a great reminder to make sure all our data is safe and secure!

Backing Up

This is something we try to stay diligent about, and use our SYLO days to keep on top of it and ensure our backups are working and up-to-date.

Sort Your Life Out (SYLO) Days

A year ago, we set ourselves the goal of spending one day each month focused on life admin tasks and the results have been so fantastic, it's time to share our approach!

Sort Your Life Out (SYLO) Days

Storing backups is a bit of a challenge for us though - we obviously can't have a big, heavy fireproof safe in our RV, and we don't have an off-site location like a friend's house or office that we can leave copies at because we're always traveling. We know we're taking a risk by having both copies in the RV, but we don't have a lot of choice.

For our most important data, we use online backup tools - a combination of Google Drive, remote backing up to the NAS at my parents' house in the UK, and Backblaze.

So why don't we backup everything online? Well, uploading 100TB to a cloud service can get very expensive very quickly! And while some cloud providers claim "unlimited storage", in reality there are acceptable use policies that they can invoke to restrict your usage.

On top of that, simply uploading that amount of data takes an astronomical amount of time! While traveling around the US, we were typically using a cellular connection, and that amount of data upload would have raised eyebrows.

How we get Internet in our RV

Whether you just want to check emails and social media, stream movies or work from the road, reliable internet connectivity in your RV has never been more important. Here's how we do it!

How we get Internet in our RV

Even here in the campground where we've had Xfinity cable internet plugged directly into our RV, our upload speed is a mere 14Mbps or so. If you could completely saturate your upload, it'd still take almost a month to upload the contents of just one 4TB drive!

Xfinity Cable Internet in an RV

Sick of unreliable and slow RV park WiFi? Struggling with cellular connectivity? We signed up for Xfinity for high-speed cable internet in our RV this winter.

Xfinity Cable Internet in an RV

Since earlier this year though, we've been using Backblaze. For just $7 per month, their Cloud Backup offers unlimited backups - we've been using it with Diana's laptop and now have more than 10TB of data backed up.

I also use Backblaze B2 to manage backups from the NUC. I use restic to manage those backups and currently have over 200GB of important data backed up.

New drive rituals

Whenever we get a new spinning drive, I always go through a burn-in process to verify it is working perfectly. Drives are mechanical devices with physically moving parts that can (and at some point will) fail.

Before putting my data on the drive, I want to know that it's not faulty. I'd much rather find a problem now while it's within the return window so I can easily exchange it.

Alex Kretzschmar, tech guru and host of the Self-Hosted podcast, has a great blog post explaining his new hard drive rituals in more detail. I follow the exact same process as Alex, and am currently about a day into the verification of our new 14TB drive - a drive this large will take over a week to complete!

Self-Hosted Podcast: Adventurous Build (Episode 53)

This week I was invited to join Self-Hosted show hosts Chris Fisher and Alex Kretzschmar to talk about my experience with self-hosting  in the RV.

Self-Hosted Podcast: Adventurous Build (Episode 53)

Summary

While the purchase of our latest 14TB Easystore may have tipped us over the 100TB mark, in reality we've been steadily approaching this point for a while.

For many home-dwellers, a NAS may be a much more sensible option than managing 20 external drives, there are some advantages to our approach. In addition to lower power consumption and more portability, it's also allowed us to seamless expand our storage as our needs have grown.

That said, we're looking forward to having the mechanical building on our property where we'll be self-hosting some local storage!

Previous postMilling Pine, First Snow & New ToolsNext postMilling 2x4s on our Sawmill