RAID suggestions

[katy-kid 5]

Hi there, I have a 2013 macbook pro, and have a seagate hard drive and am looking into a RAID system. From my understanding RAID 5 or 6 is the best choice for image security and speed; does anyone have any brand recommendations they could offer? I'm looking into spending in the $500 range, not sure if this is even viable or if RAID is all it's cracked up to be. Thank you guys for your help, it's always really appreciated.

[TimG 62]

Hi Kay

 

Are you looking for drives that will mirror each other?

 

If so, I bought the G-SAFE system about 4-5 years ago. 2x 3TB drives in one housing which use USB to plug into my iMac and mirror each other. I use Good Sync software to sync my LR photo collection to the drives - which automatically mirror.

 

I also have a third G-SAFE 3TB drive which I rotate in and out of the G-SAFE housing with the other two drives so having 3 copies plus the original on my iMac. The third copy is kept "off-site" in a Pelican case!

 

I have been really pleased with this system and feel very confident in its backup capabilities. I think, at the time, it cost about £400 and it has proved very reliable.

 

G-Technology is the manufacturer and there is now an updated system.

[sunnyboy010101 12]

Just my opinion, but I would recommend RAID 5 minimum. The optimum RAID solution these days is any NAS (network attached storage). You might be able to get a package deal, but bought individually you will need to buy the bare NAS box plus 4 identical hard drives. I recommend 2TB or greater for the HDD as these things are getting pretty cheap.

 

Buy the biggest hard drives you can afford. Look for a "hot swap" NAS, and one that you can just plug drives into (or remove) without having to attach carriers.

 

The benefit of RAID 5 is that it will detect one bad drive and let you "hot swap" that drive while the system is running. It will then rebuild the "volume" (all the drives make up a volume) without losing any data.

 

So RAID 5 = single drive failure correction, more than one drive fails it will detect but you lose your data. That's pretty rare.

 

There are other levels of RAID like basic mirroring with 2 drives, or even the super-fancy RAID 10 which is RAID 5 + mirroring (so lots of drives needed).

 

I am running a LG NAS box that's discontinued, but also run a DROBO which I got cheap and love. The advantage of the DROBO if you can afford it is that mine has 8 drive slots, and you can just plug in ANY drives and it will calculate an optimal use and RAID level for them. I started with like 6 scrounged drives, but now have 4 x 4TB drives for lots and lots of RAID 5 storage.

 

Last point - if you don't have a good backup program that will back up all your important stuff EVERY DAY, you will lose data someday. Probably when you really needed it.

[katy-kid 5]

Hey guys, very solid advice. Tim, that is pretty much what I was looking for I believe, is that considered RAID 5? Also, sunnyboy, is there a particular brand you use for the hard drives? I have used seagate and they seem ok. Also, when you say 'lose data' (*shudders), do you mean if I am not uploading images daily, or even turning on my drives daily, that they may fail? What would be considered a backup program? Thank you both, really appreciate your help.

[TimG 62]

Sorry Katy, I have no idea which RAID it is (!) and I'm not at home at the moment to check.

 

Really useful advice from sunnyboy.

[bvanant 191]

You might also look at Drobo. Not Raid but it has the advantage of using drives of different sizes. We have 5 of them running continuously; 2 for video, 2 for photos and one backing up general stuff. For drives for any NAS/Drobo/Raid box make sure you get NAS drives. I have a mix of Hitachi/WD/Seagates in my Drobos.

 

Bill

[sunnyboy010101 12]

Katy,

 

Yes, RAID 5 is what you want. I've used Seagate and Western Digital drives in my NAS boxes - whatever is cheapest at the time I bought them. I do not like to mix drive brands. I normally like to buy all 4 at one time.

 

As for data protection, 2 weeks ago I had one WD drive go bad in my LG box. I got the yellow light and warning that a drive had failed, so found another identical spec drive on amazon and bought it. Once arrived I just unplugged the dead drive and plugged in the new drive and about a day later the system was rebult and working. NO DATA LOSS. (it does take time to rebuild the volume when a drive is replaced).

 

With RAID 5, if one drive fails and you replace it fairly quickly, you will not lose data. If two drives fail, you WILL lose data, even with RAID5.

 

If you don't back up your computer frequently and it dies, you lose whatever was on it since the last backup. That's why I back up every day at about 1am.

 

DROBO does support RAID 5, as my main NAS these days is an 8-bay DROBO and it's running 4 drives (4TB each) and is running RAID 5.

 

 

Just to give you an idea, I've been "doing" computer stuff since 1979, and used to back up stuff on floppy, then CD, then DVD, then Blue-Ray and finally now RAID storage devices. My system has a 4TB data drive that is backed up daily to a server with the same single 4TB drive, then backed up to an LG NAS with 4x2TB drives and finally backed up to a DROBO with 4x4TB drives. Yes, I have THREE backups done every day after midnight.

 

Even with that, if my main computer were to crash at say 7pm, I would lose the day's work. Since backup programs can be run manually, if I were extra cautious, ever time I upload photos to the data drive I *could* run the backup to make sure they were saved. I don't because I'm probably getting complacent as I've not lost photos yet.

 

I have photos backed up including scanned slides from the '70s, so around 90GB total representing about 32000+ photos.

[bvanant 191]

It seems clear from the Drobo literature that they are not doing RAID in the ISO spec sense. You can use multiple drive sizes for example in a single box (not possible with traditional RAID). In any case worth looking at; the 5D version is quite fast.

Bill

[adamhanlon 0]

The problem with Drobo is that in the event of the enclosure failing, the only way the get your data is to put the drives into another Drobo. They use a proprietary file system. RAID (at whatever level) is a standard configuration that will allow the data to be accessible in pretty much any RAID enclosure...

[cneal 9]

As a retired IT guy I would recommend buying a spare drive for a RAID 5 array. When a drive fails a few years down the road locating an identical replacement can be a problem. If a spare is on the shelf it is a quick and painless fix.

[bvanant 191]

Adam: While that is true in theory, i had a couple of Wiebetech enclosures die and no other array could read the drives. Maybe if they are from the same manufacturer with the same firmware it might work.

Bill

[Barmaglot 250]

As a server/infrastructure admin in my day job, I have to stress a point: RAID is not a backup! RAID provides business continuance in the case of a certain class of failures, though it is by no means a guarantee by itself. Furthermore, RAID5 is the least reliable type of RAID besides RAID0 (which, by strict definition, is not RAID at all), especially as your arrays get bigger.

 

To illustrate, imagine an 8-drive RAID5 array where a drive fails - you're immediately in degraded mode, where reads have to be reconstituted from parity data, and any additional drive failure kills your array dead. Then, once you replace the failed drive, or your hot-spare (if you have one) kicks in, the array, while still degraded, is put into maximum stress mode - the rebuild process reads all the data from all the drives to build the blocks on the new drive, and that's where one of three things can happen: (1) everything is fine, the array is rebuilt and you go back into normal mode, (2) another drive fails under stress (and drives from a single batch sometimes fail close together) and your array dies, or (3) as your data is rebuilt, a bad block is discovered on one of the remaining drives, and with redundancy gone, it cannot be rebuilt, resulting in a condition known as 'punctured stripe' - your array is still online, but some data is bad, and normally you will need to destroy the array, rebuild it with good drives and restore the data from backup.

 

RAID6 avoids these two latter scenarios by computing two sets of parity data, which enables it to survive double disk failures - at the cost of higher overhead (lower performance) and lower raw space to usable space ratio.

 

RAID10 (stripe of mirrors) has the highest performance, but also the worst raw to usable space ratio of the common RAID levels (only half of your raw space can be used), and it can survive most, but not all double-disk failures. To illustrate, in the same 8-drive array configured as RAID1, you have drives 1+2, 3+4, 5+6 and 7+8 configured as mirrors of each other, and then your data is striped across these four sets of mirrors. If drive 1 dies, the array can survive the failure of any drive other than drive 2 - better than RAID5 (which can survive no additional failures) but worse than RAID6 (which can survive any one additional failure).

 

An important feature of any RAID system is proper maintenance and monitoring - drives have to be periodically scanned, and an administrator has to be alerted of any errors, in order to take immediate corrective action.

 

In a home user scenario, unless your datasets are very very very large (tens of terabytes and up), there is little use for RAID. Almost inevitably, it will provide a false sense of security, while left unmaintained and will eventually die - taking your data with it. I have seen this scenario play out more than once. If your goal is data integrity, then instead of RAID, invest in a proper backup system, with versioning and off-site storage. Backblaze, for example, will let you back up unlimited data for just $50/machine/year. Unlike RAID, this will also protect you against data corruption, user errors, ransomware infections, hardware theft, site disasters, etc.

[NWDiver 42]

Maybe same issue, maybe different. But opinions welcome

 

Got a deal on a iMac i7 core, 4.2GH, 32GB ram, DDR4, 512GB SSD Drive. Using it mainly to edit 4k video, using FCPX. I have older G-Drive 4TB drives with USB3 connections. Given the "small" hard drive of the iMac I am trying to save to, and edit off an external hard drive. When working in FCPX and try to "play" a 5min video in the timeline it stutters and stops. You can see and hear the G-Drive working away. But as the FCPX timeline runs the stuttering, slow play makes it very hard to sync music or see timing.

 

Would a Raid 5 system like linked below help in SPEED OF EDITING and smoother operation when using FCPX??? Not a pro, not super worried about storage, really looking to make editing in FCPX as fluid as possible. Hope that makes sense. Thanks again!

 

(Price is always an issue)

OWC / Other World Computing ThunderBay 4 8TB (4 x 2TB) Four-Bay Thunderbolt RAID Array

 

https://www.bhphotovideo.com/c/product/1075757-REG/owc_other_world_computing_8tb_4_x_2tb.html

Edited September 20, 2017 by NWDiver

[Barmaglot 250]

The system you linked doesn't do RAID by itself - it's a JBOD, which stands for 'Just a Bunch Of Disks'. When connected, you'll see four separate 1.86TB drives plugged into your system. You can use MacOS Disk Utility to configure it as RAID0 (all space usable, no redundancy - one drive goes and all your data is gone) or RAID10 (half the space is reserved for redundancy). Depending on how that enclosure is built and on MacOS RAID stack, you might or might not see some performance gain over a single drive. Without extra software, you won't be able to do RAID5.

 

However, for $800, you can get a single 2TB external SSD such as a Samsung T5 which is guaranteed to have much better performance than a box full of 7200rpm spinning rust.

[sunnyboy010101 12]

Generally for video performance, a single hi-performance disk is recommended. I suppose a larger capacity SSD would be awesome, but SSD's do have drawbacks.

 

RAID is not a good fit for performance requirements, especially video editing. RAID comes in two flavors, hardware based and software based. A lot of modern RAID is software based, and it is not the best for performance, especially when writing data, as the data has to be recorded in multiple spots or spanned over drives. Some hardware RAID can be really fast, especially if the hardware includes a good processor.

 

Generally, I'd assume unless the device really blows it's horn about hardware speed and RAID performance, it's probably software RAID and may not be fast enough to really support video work.

 

I concur that a single large drive is probably a better choice.

 

Also remember that the type of drive connection can be really limiting... IDE 2 is pretty slow. IDE3 is somewhat supported (it's hit or miss even when it's supposed to be supported by a machine). E-sata external connectors are really the way to go if you want speed for video work from a plug-in drive.

 

Just my $.02

[Barmaglot 250]

Also remember that the type of drive connection can be really limiting... IDE 2 is pretty slow. IDE3 is somewhat supported (it's hit or miss even when it's supposed to be supported by a machine). E-sata external connectors are really the way to go if you want speed for video work from a plug-in drive.

 

There is no such thing as 'IDE2' or 'IDE3'. The term 'IDE' stands for 'Integrated Drive Electronics' - it goes back to 1980s, when PC hard drives had a dedicated controller card which plugged into an ISA bus slot (I'm simplifying here for the sake of brevity) - Western Digital's IDE standard, released in 1986, moved this controller into the drive enclosure, and ran a derivative of the ISA bus protocol over a flexible ribbon cable. Years later, it was standardized as AT Attachment, or ATA. The standard was iterated with higher transfer speeds, growing from 16MB/s to 33MB/s, to 66MB/s, to 100MB/s and finally 133MB/s, until it was superseded by Serial ATA (SATA), which used the same command set, but communicated over a high-speed serial link rather than ATA's 80-wire parallel link. The first iteration of SATA was capable of transfer speeds up to 1.5Gbit/s (up to 150MB/s in practice due to protocol overhead), then SATA2 (2004) increased that to 3Gbit/s, and finally SATA3 (2009) went to 6Gbit/s. Note that even on its best day, a spinning drive is hard pressed to hit 200MB/s in linear transfer rate, and in actual usage, it's generally a tiny fraction of that, so the interface is very rarely a bottleneck for performance. For SSDs, of course, the story is different - modern SSDs can and do bump into the 600MB/s limitation of SATA3, which prompted the development of NVMe.

[NWDiver 42]

Thanks for all the info! I did download Blackmagic's Disk speed ap. On the iMac 512GB SSD drive it shows over 1900mb read/write speeds. On my old 4TB G-Tech drive with a Super Speed USB3.0 cable the read/write speed was about 40mb/s. So maybe a SSD external hard drive is the best route to edit FCPX from. Again all raw data for the projects is 1-2TB.

[sunnyboy010101 12]

Sorry, meant USB 2 and 3. Late, tired, etc.

[NWDiver 42]

Still trying to figure out what to do.

 

Side note; to transfer 1.95TB of 4k video from my G-Tech 4tb external drive to my Lacie Tough Drive took about 13hrs at 42mb/sec.

[NWDiver 42]

In the end decided editing 4k video was the key. Would risk losing data if a drive fails. So went with the following external hard drive to edit on and then will use old hard drives to store final projects on. Thanks to Tamir at B&H to walk me through options.

 

https://www.bhphotovideo.com/c/product/1295928-REG/glyph_technologies_ar2000slv_atom_raid_2tb_ssd.html

[NWDiver 42]

Final follow up...the Glyph Technologies Atom RAID 2TB USB 3.1 Type-C External SSD came in and so far WOW! In FCPX I have a 3x45min 4k, time lines stacked on each other and editing and playback is super smooth. No more stuttering and stoping. From a strictly editing point of view very happy.

[Undertow 31]

I use the OWC Thunderbay2 bundled with Softraid. Two enclosures (8 bays), 6x4tb HGST drives (2 spare bays for backup drives), RAID 5, imac 5k. Runs as fast or faster than any SSD (750+MB/s I think it was). So far its been great. Lots of space (10TB+ photo library) lots of speed. But I'm no IT guru.

[bossygirl 1]

Following this thread. So interesting. Thanks for posting!