Today I am going to run you through all of the components that make up a great Data Management Kit. My philosophy is to have every tool in the toolkit, that way no matter what problem arises you have all the resources on hand to tackle it head on.
The Data Kit that I outline below is my personal Data Kit. It has been built on years of experience and problem solving on-set. The best way to think about it is to break it down into three categories, 'Hardware', 'Software' and 'Accessories'. I will address each category individually and identify which items are absolute essentials and which are nice to have. Let's dive in!
HARDWARE
LAPTOP
(Essential)
The primary component on which the rest of the Data Management Kit is built upon is the laptop. The Apple MacBook Pro is the industry standard for a Data Laptop but you have a few flavors that you can choose from. The first being Old MacBook Pro (Retina Display) and New MacBook Pro (Touch Bar), the second being 15" or 13" Screen Size.
OLDER RETINA MACBOOK PRO
The Older Style Retina MacBook Pro is an absolute workhorse and holds up to this day. There were many iterations on this model between 2012 and 2015, over that time period it evolved from Thunderbolt 1 Ports to Thunderbolt 2 Ports, the CPUs and GPUs got upgrades and the internal solid state storage went from SATA to PCIe. It's main advantage in the current data landscape is the fact that is has native USB3.0 and Thunderbolt 1/2 Ports. The world is in a transition period from USB3.0 and Thunderbolt with a Mini Display Port Connector to the all in one USB-C Connector that sports USB3.1 and Thunderbolt 3. Many peripherals that you will use while data wrangling still rely on this legacy technology. The advantages of USB-C are clear but while the world catches up this still needs to be a consideration, it's infinitly easier to use your card readers and hard drives without additional adapters which is where this MacBook Pro really shines.
NEW TOUCH BAR MACBOOK PRO
The New Touch Bar MacBook Pro is a solid machine. It's certainly equipped for the next generation with 4x USB-C Ports that support both Thunderbolt 3 and USB3.1, though this does come with some disadvantages. As the USB-C Port doubles as a charging port for the laptop you basically sacrifice one data port for the sake of power. In the Old Style MacBook Pro you had four data lanes, 2x USB3.0 and 2x Thunderbolt, in this New Style MacBook Pro you have only three data lanes, 3x TB3/USB3.1 as one is tied up with charging. You also need to deal with that fact that this laptop is on the cutting edge of technology, meaning that you'll need adapters to use USB3.0 and Thunderbolt 2 Devices. Not a deal breaker as there are many options out there to split your USB-C Port into multiple USB3.0 and TB2 Ports but it does add additional clutter and complication to your data setup. One really cool advantage of this machine is that it can be charged from a power bank via USB-C. This means that you could run this laptop all day with bus powered drives and never need external power, a big advantage for run and gun jobs where you are constantly moving from place to place.
15" vs. 13"
You could setup a data laptop with either of these screen sizes as they generally have identical connections but they don't have identical specs. In general the 15" MacBooks will have Quad-Core CPUs and slightly beefier GPUs than their 13" Dual-Core Equivalents. The higher specs won't have a huge impact on your day to day Data Wrangling but they will impact your output when you get asked to do a quick transcode for a job. It's always best to build your kit for the worst case scenario, this means top notch specs. Max out your CPU, GPU and RAM and get a decent amount of Internal SSD Storage (512GB). If you max out your specs when you buy you laptop new it will last many, many years. Computers do date as quick as they say but they certainly don't become redundant as quick as you think, particularly in the context of data management. The other thing to consider is the actual screen size, you'll have considerably more real estate on a 15" Screen than a 13" Screen. This makes your life much easier while working and is also better when a Director or DP comes over wanting to look at the rushes. Bigger isn't always better, but in this case it is.
MY LAPTOP
I'm currently using a Mid 2012 15" Retina MacBook Pro and it's still going strong. It's equipped with a 2.6GHZ Quad Core CPU, 1GB GPU, 16GB RAM and 256GB SSD. The battery has been replaced to maintain optimal portability and I've replaced the thermal paste on the CPU/GPU for better thermal regulation. I clean the internals from time to time and run regular software maintenance to make sure it's always performing at it's peak.
UPS
(Essential)
The Uninterruptible Power Supply. An absolute must have item for every Data Management Kit and should be used all the time. You'll sometimes be running card readers or drives that require external power, running these through a UPS will save your offload should a circuit be tripped or the power get pulled. The basic idea is that you can't interrupt power flow. If you lose mains power your UPS will continue to provide power to your devices for a few minutes while you identify the problem and fix it. I've seen many inexperienced data wranglers operate without a UPS and get stitched up when they are 40 Mins into an offload and they lose power thus corrupting their backup. This means they have lost 40 Mins of offload time, a significant set back in data heavy environments. Don't make that mistake, get a UPS.
I use an Eaton 3S 700 UPS which I've had for a couple of years now. I really like that it has 6 Power Ports, 3 are UPS Protected and the other 3 Ports are just power pass through. This gives me enough UPS Ports for Data while having enough spare power ports to add battery chargers as well. This means I don't need additional power boards, just my UPS. Easy. You should also make sure you have a few 10 Amp Cables in your kit as well, if you get caught up with a backlog of data at the end of the day, the Electrics won't be too happy having to wait around for their cable, nor will production be too happy having to pay them overtime. Always setup data with your own power cables, that way no one else is affected if you need to do some overtime.
SOFTWARE
SHOTPUT PRO
(Essential)
Shotput Pro is the backbone of many Data Kits all over the world. It's very simple to use, it's got a one off purchase cost and is incredibly reliable. It's used to facilitate data transfers, it can copy to multiple destinations and perform checksums on those offloads while generating basic reports. It can be setup with separate offloads for different cameras which is super handy for multi-camera jobs. I've used this software to manage data on hundreds of jobs over many years, I've used it on Single Camera Jobs, I've used it on 20 Camera Jobs. Buy it. Learn it. Use it.
SILVERSTACK
(Optional)
In more recent times I have switched from using Shotput Pro to Silverstack. Silverstack offers a more advanced feature set than Shotput which can be super useful for bigger productions or when you're seeking to deliver higher quality output to your production. It has many built-in report options which can be customised with clip metadata, it has project management features, built-in false colour and peaking for quality control checks as well as the ability to do transcodes. Pretty amazing. It doesn't come cheap though, it runs on a subscription fee that comes in at roughly $800AUD/year. I needed to use Silverstack for a specific job a few years back, once I got into it there was no turning back as it allowed me to deliver great reports at the end of each shooting day which production found invaluable.
HEDGE
(Optional)
Hedge is a relatively new data management software that competes with Shotput Pro and Silverstack. I haven't extensively tested it but have heard some great things about it's speed and reliability. It's got quite a modern design and a good development team behind it. I plan at taking a closer look at this in the future, when I do I'll be sure to let you know!
DAVINCI RESOLVE
(Essential)
This is the best free software that you will ever get. It's amazing. You get industry standard colour grading software that can output UHD, it works as a NLE, it has built in compositing with Fusion as well as audio editing. A must have for any Data Management Kit. I use it for transcoding, quality control checks, test grades and many other tasks that come up on a day to day basis.
CODEX VAULT PLATFORM
(Essential)
The Codex Vault Platform is a must have for your Data Kit. You need this to setup the Codex VFS (Virtual File System) on your computer which is required to access ARRIRAW from an Alexa XT, SXT or LF. If you shoot ARRIRAW on an Alexa Mini it isn't required as that flavour of ARRIRAW is in a .mxf File Container which works natively on Mac as opposed to the .ari File Container. If you ever find yourself using a Codex Vault you'll need this software to interface with the hardware. You'll need to sign up for a free Codex Account to download this software.
REDCINE-X
(Essential)
This is the all-in-one RED Toolkit from RED Digital Cinema. While you can playback R3D Files in a number of different softwares this is still my go to when dealing with RED Footage. It allows you to transcode, quality control check and apply colour correction to your footage. It's optimised to work with RED RAW from any RED Camera, it should be in your kit hands down.
BLACKMAGIC DISK SPEED TEST
(Essential)
Running a disk speed test is a daily occurrence for Data Wranglers. Everytime I get a new hard drive I will run a disk speed test on it before I use it, that way I can verify that it is running correctly at the speeds that I expect. When you have an offload performing slow this is an invaluable tool for troubleshooting. Super simple in design but it's solid as a rock. A must have for your toolkit.
iSTAT MENUS
(Essential)
One of my favorite utilities. Basically it displays your computer's vitals in the task bar so you can check them at a glance. It's highly customisable so you'll be able to tweak it to best suit you. I use this to check my CPU/GPU performance during transcoding, to monitor Hard Drive Read/Write Speeds during offloads, to look over my Data Speeds for Uploads/Downloads online and to quickly check CPU/GPU Temperatures when I'm demanding lots from my system such as transcoding or just working in hot environments.
LABEL MAKER
(Essential)
At the core of every good data management process is organisation. This tool was custom built for making beautiful hard drives labels in record time. I use it to colour code my hard drives for easy identification, it let's me label drives with job name and job number as well as putting client and production logos on there which always goes down a treat with my clients.
ARRIRAW CONVERTOR
(Optional)
This is ARRIs Propriety Software for dealing with ARRIRAW from an Alexa XT, SXT or LF. It used to be the bees knees for playing back ARRIRAW on your computer but I often found that it sometimes didn't work in which case I'd use Resolve instead. Now days I just use Resolve exclusively but it's nice to have a backup for playing ARRIRAW should I encounter some form of issue with Resolve while on a job.
ARRI COLOR TOOL
(Essential)
This is a great piece of software that will allow you to convert a 3D LUT .cube File to an ARRI LOOK File which can be used with any Alexa or Amira Camera. This comes into play a lot on-set when a DP has a LUT that they want to toggle on/off in the camera. They give you the LUT as a .cube, you convert it to a LOOK with this tool and then transfer it to the camera via USB/SD Card. Very handy to have.
ARRI META EXTRACT
(Optional)
This is a tool to extract metadata from footage that has been shot on any flavour of Alexa Camera. It's mainly intended to be a Post-Production Tool but I have found that it can be handy on-set every now and then for pretty niche purposes.
One job didn't have any camera sheets from Shoot Day 01 and we needed to match a shot on Shoot Day 02, the focus puller had an idea of the distances but couldn't quite remember. As we were using an Alexa Mini with a WCU-4 I was able to find the shot and extract the metadata where I found the focus data, which I then interpreted into feet and inches and was able to confirm the exact focus distances for that shot. Pretty cool.
Use for this software won't come up all the time but it can be super handy from time to time.
ARRI FRAMELINE COMPOSER
(Essential Web Tool)
This isn't software that you download to your computer, it's online software that can be accessed from the web on any computer. It's used to generate custom framelines for any Alexa Camera. I use this all the time. There is always a request for custom shading or for specific framelines that aren't standard. You'll hear 'Give me a 1:1 Frameline for social' or 'Can I get 2.40:1 Framelines in 4:3 Sensor Mode with 50% Shading?'. Nothing to download, just know it exists and be familiar with how it works.
ADOBE PREMIERE PRO
(Optional)
Premiere Pro is a popular editing software used by editors in the industry. It's incredibly versatile and performs well. Sometimes you may encounter a really weird file format that none of your other software will open, try Premiere, it almost never fails. If you ever get asked for a quick edit from a Director or DP now you can effortlessly deliver. Not essential but definitely a valuable tool for your kit.
ADOBE MEDIA ENCODER
(Optional)
While I use DaVinci Resolve for most of my transcoding I still find Media Encoder to be a great backup option. Just as Premiere is great at reading weird file formats, Media Encoder is great at exporting to weird file formats. I have used it quite extensively over the past year for doing HEVC (H.265) Transcodes which Resolve doesn't support. For me it's essential, for you I think it'd prove handy.
ADOBE AFTER EFFECTS
(Optional)
Many times I've been on a Green Screen Shoot without VTR and the question has come up 'Can you just do a quick test key on this please?'. My answer is 'Absolutely!'. I can import rushes into AE and do a quick key with KeyLight. I'm no Green Screen Wizard but this is generally more than enough to give whoever is asking peace of mind that their key is going to work.
ADOBE PHOTOSHOP
(Optional)
Photoshop is my go to tool for a lot of things. In the world where stills, graphics and video crossover all the time I find having Photoshop super handy. I used to use it to make my Hard Drive Labels before Label Maker came along, I still use it for slate covers and many other graphic and image manipulation tasks.
iFUN BOX
(Optional)
This is software for looking at data that is directly on your iPhone. Why is this included in my Data Kit you ask? Well, I've had jobs where a Director will shoot some additional footage on their iPhone that is intended to be cut in with the actual TVC. They ask me to wrangle it, no worries, this software makes it easy. It allows you to look at your iPhone Camera Roll and copy specific clips to your computer or the drives. While it doesn't do the checksums that you'd come to expect in a professional data wrangling environment it gives you reliable data transfer speeds and guarantees no loss of quality from the original media.
ACCESSORIES
CARD READERS
There are two ways to look at Card Readers. They are provided by either the 'Camera Rental House' or the 'Data Wrangler'. I sit somewhere in the middle. High-End Card Readers should be provided by the Camera Rental House, this includes the Codex Mag Readers, Codex CFast Readers, RED Mag Readers, XQD Card Readers, etc. Card Readers that the Data Wrangler could have in their kit include SD/CF Card Reader, SxS Card Reader, CFast Card Reader. I'll outline what is in my kit below.
SD/CF CARD READER
(Essential)
The main reason to have an SD/CF Card Reader in your kit is to copy sound cards. Almost every Sound Recordist will record to either an SD Card or a CF Card and you should have a reliable way to read it on your system. Some MacBook Pros have an SD Card Slot on the side, this isn't as fast as a USB3.0 SD/CF Card Reader. So even if you have an in-built SD Card Reader you should still get an external one as it's faster, it will allow you to read CF Cards and then your internal reader will serve as a backup. Also, if you find yourself shooting on a DSLR that uses SD/CF you also have a Card Reader ready to go in your kit. It's not just for sound, it's for camera too.
Is use this Lexar USB3.0 SD/CF Card Reader. I've had it for years and it's never skipped a beat. The speeds are good and it's reliable. Plus the click up, click down actuation mechanism is amazing!
SxS CARD READER
(Essential)
While the humble SxS Card is fairly dated now, having an SxS Reader in my kit is something that I consider standard. It spawned back in the day when the Alexa Classic recorded to SxS Cards, all rental houses in Melbourne would send out the Sony USB2.0 SxS Card Reader with the kit which was very slow. To combat that all of us Data Wranglers purchased a Sonnet Thunderbolt SxS Pro Card Reader which improved offload speeds considerably. Nowadays Sony has a USB3.0 SxS Card Reader which may or may not come with your camera kit. In my eyes it's much better to have your own, if it doesn't get used it serves as a backup.
CFAST 2.0 CARD READER
(Optional)
I never used to have a CFast Card Reader in my kit, then one day I was on a job and had a Codex CFast Reader fail on me. Fortunately the rental house could send out a replacement on that same day but it took 2 Hours which set me back significantly. I'll never let that happen again so I purchased my own CFast Reader for my kit which has come in handy many times since.
I have the Lexar CFast Reader which boasts both USB3.0 and Thunderbolt.
Your other option is the SanDisk CFast Card Reader.
ADAPTERS
No matter what flavour of MacBook Pro you are using you'll likely need to include adapters in your Data Kit.
KANEX THUNDERBOLT TO USB3.0 ADAPTER
(Essential)
These are amazing adapters. They allow you to turn a Thunderbolt 1/2 Port into a USB3.0 Port. I am a firm believer in running each Card Reader/Drive on a separate bus or computer port, this adapter allows me to do that even with 3x USB3.0 Devices. I ended up buying two which let's me turn my Old Style MacBook Pro into a system that has 4x USB3.0 Ports, I've used this on many jobs where the client wanted me to backup to 3x USB3.0 Hard Drives from a Codex USB3.0 CFast Card Reader. Nothing in your kit should slow data management down, these adapters are absolutely essential.
APPLE THUNDERBOLT ADAPTERS
(Essential)
If you need to convert one port to another I have found the Apple Adapters to be very reliable. Even though I don't have a New Style MacBook Pro I still carry newer adapters so that I can be equipped to handle any situation that arises.
I have the following in my kit:
USB-C to THUNDERBOLT 3 x2
USB-C to USB TYPE-A
THUNDERBOLT 2 to FIREWIRE 800
THUNDERBOLT 2 to ETHERNET
SD TO MicroSD ADAPTERS
(Essential)
I have many of these in my kit. They allow me to copy footage from a MicroSD Card via my SD Card Reader should I need to. Super handy if you get a GoPro or something similar thrown into the mix.
CABLES
(Essential)
Without a hard wired cable connection you can't wrangle data. You need cables. You need spare cables. You need all flavours of cable so that no matter the connection, no matter the interface, you've got something that will work. These are the cables I have in my kit that aren't bundled with card readers:
USB-C to USB-C CABLE x2
STANDARD USB3.0 CABLE x6
1M THUNDERBOLT CABLE
eSATA CABLE
GIGABIT ETHERNET CABLE x2
FW800 to FW800 CABLE
FW800 to FW400 CABLE
FW400 to FW400 CABLE
USB 2.0 TYPE-A to USB 2.0 TYPE-B CABLE
USB 2.0 to MINI-USB
USB 2.0 to MICRO-USB
Have a lot of cables. Have spares. Be prepared.
OTHER
LAPTOP CHARGER
(Essential)
You'll get a charger with your laptop. Make sure it's in your kit at all times. Ideally carry a spare. If your laptop has no power you can't wrangle data. I originally only traveled with one charger, then it broke so I purchased a replacement and repaired the old one. Now I use the old charger as a spare. Sometimes I set it up in my van with a car inverter to keep my rig running while in transit.
CAR INVERTER
(Essential)
As I've said before 'If your laptop has no power you can't wrangle data'. Some jobs will be run and gun, constantly moving from location to location, it's easy to get stitched up. An inverter will help mitigate that risk. It doesn't need to be a huge inverter as it's only charging your laptop which usually draws around 85W. This will help you stay charged throughout the day even with ten location moves!
LABEL MAKER
(Essential)
While these days I prefer to make high quality labels using Label Maker I still carry a Dymo Labeller in my kit as a spare, my model is a Dymo LetraTag 100H. This comes in handy from time to time for labeling drives or other miscellaneous things on-set. You could also go a P-Touch Labeller which is also quite popular.
USB3.0 THUMB DRIVES
(Essential)
Often you'll need to quickly copy a small file here or there. You might have a Production Manager asking if you've got a USB Stick. It's easy to grab a few USB Thumb Drives for your kit so you've got a quick way to solve these problems when they arise. Definitely recommended.
HARD DRIVE DOCKING STATION
(Optional)
You definitely don't need one of these in your Data Kit, it's just a nice safety to have. I carry it as a fail safe to help troubleshoot drives. If I have a hard drive fail and want to investigate further I can use this to mount the bare bones SATA Drive and see if it reads thus determining if the problem is with the drive itself or the enclosure. Also if I'm dealing with RAID Drives, sometimes RAID Arrays with 5 Drives will have a problem and I'll need to look at one drive individually, this is the tool that allows me to do that.
I use a VoyagerQ HDD Docking Station which sports USB3.0, FW800 and eSATA Connection. It requires external power to run and ships with a power adapter.
iPHONE CHARGERS
(Optional)
What do all people on-set use everyday? Their phones! They always die and one of my most asked questions is 'Do you have an iPhone charger?'. So I started carrying iPhone Chargers in my kit. I have one 10W Charger for iPads and one 5W Charger for iPhones. I also carry the old school 30-Pin Charge Cable as well as multiple Lightning Charge Cables. If you throw in a Micro USB Cable you can charge most Android Phones as well. Be a hero, carry a phone charger in your kit!
POWER BANKS
(Optional)
If you want to one-up your service have a power bank or two in your kit. That way if someone asks for a charger you can offer a portable option as well. I don't use mine that often but it has come in handy many times.
One time in particular we were on a Movi all day using the Alexa Mini WIFI Control on iPhone, the 1st ACs phone was close to dying which would have caught us out, I brought out my Power Bank and a Cable and we were good to go for the rest of the day. Not essential but a nice addition to the kit.
DOUBLE ADAPTER
(Essential)
These are great if you need to add just one more bank of power to your UPS or if you need to split off power and share with someone else. Ties in again with having your own power infrastructure for on-set data wrangling.
USB MOUSE
(Optional)
Funny story behind this mouse. One time on a job I had someone spill water on my laptop. They didn't tell me about it either, I went back to put on the next card and found water all over the trackpad. I immediately soaked it all up but something was wrong, the trackpad didn't work. Thankfully everything else functioned but I was unable to wrangle data as I couldn't interact with my computer due to the damaged trackpad. Luckily someone had a spare mouse which I plugged in to control my computer and that got me through the day. Now I carry my own spare just encase this very rare instance happens again.
BACKPACK
(Essential)
Last but not least, the backpack! How on earth do you carry all of this stuff around? Well, you have a few options. Some people like an over the shoulder laptop bag, others like Pelican Cases, me, I prefer a backpack. Currently I am running an old Lowepro CompuTrekker AW which I've used the past couple of years. Soon I plan on moving over to a USA Gear S17 DSLR Camera Backpack. The most important thing is to make sure you have a spit for your laptop and a whole bunch of compartments. Compartments allow organisation which is key to laying out an efficient data management kit.
CONCLUSION
It's often daunting to figure out what you need when building a Data Kit. Thankfully I've managed data in almost every situation imaginable and have developed quite an understanding for what tools you will need. If you equip yourself with the tools listed above and you know how to use them you will be fully sorted on-set 99% of the time.
Do you have other super helpful tools in your Data Kit? I'd love to hear about them. Please let me know in the comments section below.
Same goes for questions, please reach out in the comments below and I'll do my best to help.
Thanks! I hope this guide has been useful to you!