By: Luke J.H. Engstrom
I'm Luke. Tech fan, engineer, history buff, developer, and closet Sim Racer. I work for Other World Computing as a Product Manager for our networked line of solutions. More specifically, the OWC Jupiter system.
“Sneakernet” is a term that describes the transfer of electronic information by physically moving media such as USB flash drives, External Hard Drives, SD cards, etc., rather than using a computer network. A typical example of this would be taking an SD card with media, ingesting it into your workstation, then backing it up to an external Thunderbolt enclosure for editing, with some form of an archive after deliverables have been given. Typically, on yet another separate device, such as one or several 3.5" HDDs to follow traditional three-teared storage practices. Every few years, cameras get better, media files get bigger, and storage gets faster. That trusty Firewire 800 unit you still cherish is just not going to cut the mustard with today's resource-hungry resolutions and codecs. Let's feature some common workflows I have seen in various creative fields, and how we can augment, and improve them by adding networked-based storage systems in the mix.
Sneakernet ingest and backup
Sneakernet media sharing via external SSD
Edit on an external drive and replicate media to backup for archiving
I have personally seen even the largest of names in the production community sometimes use some variation of external storage and complicated flow charts and workflows to make sure they are backing up and moving data around somewhat efficiently. Unfortunately, with today's high-paced media workflow, we sometimes get stuck in our ways without seeing if there is a better way of doing things. Enter NAS.
Network Attached Storage (NAS)
A “NAS” in straightforward terms is a device that shares data to devices connected to a data network, such as a home or business network. This can be as simple as a low-cost single-drive NAS device with limited feature sets and in most case limited performance, DIY servers made from off the shelf components, or even exotic Production grade solutions with 12 or more hard drives, flash devices to increase performance and stability, and advanced File systems like ZFS to protect your data from bit rot. Mostly, it's a computer designed to protect, share, and expand your data. What might have taken hours or days to transfer and copy might take minutes with a server in your life.
Let’s see how NAS can change the previous workflows and fuel synergy between users in a collaborative workgroup. NAS solutions have ethernet connections to share data on a network or directly to a workstation. All that needs to be done in most cases is to physically connect the NAS into your network switch or in your network router at home. Congratulations, you now have networked storage! From here, you connect to the NAS over a network protocol such as SMB or NFS. How you connect can vary significantly; some even have companion apps to help with setup. Once you mount your network storage, all you do is copy and paste the data from your ingest media to your mounted share. Essentially, this is a network-accessible folder.
Now that your NAS has the raw media ingested, it can be edited from, or shared by as many users as you have on your network. And because the data is distributed over a network protocol, you can mix and match Macs, PCs, and even Linux systems on the same workflow without worrying about filesystem compatibility. Your editor can work on that new shiny cheese grater while your render job runs simultaneously on a PC in another room – no computer gets left behind!
One of the most significant selling points to higher-end NAS solutions, especially ones meant for professional environments, is scalability. Simply put, it’s hard to determine accurately what your data needs are for each project. Some might take just a few hundred gigabytes, while others might require petabytes. Yes, I said that right, that’s one thousand terabytes. Knowing you have a solution that is designed to grow your data but is also protecting it is vital. Not all NAS units are created equal.
Let’s show how NAS would typically expand to allow for more projects. In this depiction, we have an 8-Bay NAS with 10TB hard drives using a ZFS RAID with double parity – similar to RAID 6. An 8-drive RAID is typical of a small media production environment with an excellent balance of redundancy and performance. Once you get to the point where you need to expand, a general rule of thumb is that when 80% full, you should extend your NAS using an expansion unit. These are usually called JBODs (Just a Bunch of Disks) and JBOFs (Just a Bunch of Flash). They are connected using external SAS, or similar reliable data connections. Notice how both the performance and redundancy go up each time you expand the unit. This is because by adding more drives, the NAS system can pull from more drive sets, thus allowing for much better performance.
The nice thing is that these units usually daisy chain, so you can start small and grow as your capacity and business grows. And the best part is that by consolidating your primary storage into one turnkey solution, you can then expand as you see fit – and not just based on what is available on the market. Need to edit in 4K? Well, add some SSDs for read or write cache. Need more network throughput? Add a 10Gb or 40Gb ethernet connection. Want to edit in 8K? Add a 100Gb fiber connection and lots of SSDs. No longer is productivity hampered by the slowest Hard Drive in your workflow!
A NAS implementation brings practicality and peace of mind by using built-in replication services. This means that you can back up your NAS to another NAS locally, or halfway across the planet. What’s more, is that you own the NAS, so you don’t have leasing fees, or the hidden costs that cloud storage might incur. Want to connect to a cloud service as well? Well, most NAS solutions have support for the major cloud storage providers such as G-Drive, AWS, Acronis, and others. The important thing is that these processes can be hosted on the NAS system using its built-in horsepower, so once set up, it just does its job – allowing you to concentrate on what you do best.
Security is a big deal. With data breaches, hacking, and data terrorism committed on a global scale, it’s paramount to your livelihood to make sure your data is safe and secure. Many NAS solutions allow for the ability to add permission schemas to your data. Need to give read-only access to that new intern? That is just a few clicks away. Want to add military-grade encryption? Well, many NAS operating systems allow for native encryption algorithms such as AES-256. Only users that have a special encryption “key” can use the data, making theft almost foolhardy. More importantly, these devices usually log every connection so you can see who has accessed the device. In many cases, these can even be viewed remotely, even on a phone.
Time savings. Have you ever stopped to think about all the time you spend moving data from drive to drive, computer to computer or production to back up? Well, let’s go over some numbers. The example above shows that to move a 1TB media file from one workstation to another via an external HDD, it would take over an hour!
The thing to remember here is that once data is on the NAS server, it can be instantly used, edited, copied, or removed by anyone connected to it. So, in this example, we cut an entire step out of the process by not copying the data twice.
A World of Plugins
It’s an open-source world! Currently, community-driven software has, in many cases, not only caught up but even in some cases, surpassed some of the offerings from some of the world’s biggest tech companies. Many NAS solutions allow plugins to be made by the community to increase simplicity, value, and functionality. Think of a plugin as a software package written to do a particular job or function. For example, you can install a plugin to allow your NAS installation to do automated render and encoding tasks when it’s not busy doing something else. Want to host a personal media library and access it anywhere in the world? There is a plugin for that. Want to host your own email server? What about host a multiplayer video game? What about backing up your phones and tablets? There are plugins for all of that! Can your external storage do that?
In this last image, we show a NAS configuration in a very optimal setup. Multiple collaborative teams in different physical locations can not only share content in real-time but also allow for near-zero time wasted due to the dreaded transfer time. As soon as the media is ingested, it can be edited, rendered, shared, and backed up. This allows for teams to be deployed in multiple locations and provides for senior staff to check-in and approve content at various steps in the production cycle.
Digital storage is a complicated place. There are so many options and ways to do the same thing. I started using NAS devices over a decade ago in production environments, and it’s truly amazing what it can do for a pro, company, or deployment. From backing up your workstation to hosting your cloud-accessible storage to military-grade encryption, the humble NAS has come a long way. And now, with so many great options available, it is the best time to make the jump!
Simply put, once you have NAS integrated into your workflow, you really can’t go back to the old way of transferring and storing your data. It would be like returning to the archaic time before HD, touchscreens on smartphones, or indoor plumbing. And who would want to live like that?