Jump to content


Photo
- - - - -

RAID suggestions


  • Please log in to reply
11 replies to this topic

#1 katy-kid

katy-kid

    Lionfish

  • Member
  • PipPip
  • 64 posts
  • Gender:Female
  • Location:SWFL
  • Interests:FLORIDA!

Posted 26 June 2017 - 12:48 PM

Hi there, I have a 2013 macbook pro, and have a seagate hard drive and am looking into a RAID system. From my understanding RAID 5 or 6 is the best choice for image security and speed; does anyone have any brand recommendations they could offer? I'm looking into spending in the $500 range, not sure if this is even viable or if RAID is all it's cracked up to be. Thank you guys for your help, it's always really appreciated.


Katy Danca Galli

Atmospheric UW Photography

http://katydancagalli.com


#2 TimG

TimG

    Sperm Whale

  • Moderator
  • 1816 posts
  • Gender:Male
  • Location:Amsterdam
  • Interests:Sunlight reefs, warm seas, fine wine, beautiful women. And Manchester City Football Club

Posted 27 June 2017 - 11:01 AM

Hi Kay

 

Are you looking for drives that will mirror each other?

 

If so, I bought the G-SAFE system about 4-5 years ago. 2x 3TB drives in one housing which use USB to plug into my iMac and mirror each other. I use Good Sync software to sync my LR photo collection to the drives - which automatically mirror.

 

I also have a  third G-SAFE 3TB drive which I rotate in and out of the G-SAFE housing with the other two drives so having 3 copies plus the original on my iMac. The third copy is kept "off-site" in a Pelican case!

 

I have been really pleased with this system and feel very confident in its backup capabilities. I think, at the time, it cost about £400 and it has proved very reliable.

 

G-Technology is the manufacturer and there is now an updated system.


Tim
(PADI IDC Staff Instructor and former Dive Manager, KBR Lembeh Straits)
Nikon D800, Nikkors 105mm and 16-35mm, Sigma 15mmFE - Subal housing

http://www.timsimages.uk
Latest images: http://www.shutterst...lery_id=1940957


#3 sunnyboy010101

sunnyboy010101

    Triggerfish

  • Member
  • Pip
  • 38 posts
  • Gender:Male
  • Location:Vancouver Island

Posted 27 June 2017 - 12:13 PM

Just my opinion, but I would recommend RAID 5 minimum. The optimum RAID solution these days is any NAS (network attached storage). You might be able to get a package deal, but bought individually you will need to buy the bare NAS box plus 4 identical hard drives. I recommend 2TB or greater for the HDD as these things are getting pretty cheap.

 

Buy the biggest hard drives you can afford. Look for a "hot swap" NAS, and one that you can just plug drives into (or remove) without having to attach carriers.

 

The benefit of RAID 5 is that it will detect one bad drive and let you "hot swap" that drive while the system is running. It will then rebuild the "volume" (all the drives make up a volume) without losing any data.

 

So RAID 5 = single drive failure correction, more than one drive fails it will detect but you lose your data. That's pretty rare.

 

There are other levels of RAID like basic mirroring with 2 drives, or even the super-fancy RAID 10 which is RAID 5 + mirroring (so lots of drives needed).

 

I am running a LG NAS box that's discontinued, but also run a DROBO which I got cheap and love. The advantage of the DROBO if you can afford it is that mine has 8 drive slots, and you can just plug in ANY drives and it will calculate an optimal use and RAID level for them. I started with like 6 scrounged drives, but now have 4 x 4TB drives for lots and lots of RAID 5 storage.

 

Last point - if you don't have a good backup program that will back up all your important stuff EVERY DAY, you will lose data someday. Probably when you really needed it.



#4 katy-kid

katy-kid

    Lionfish

  • Member
  • PipPip
  • 64 posts
  • Gender:Female
  • Location:SWFL
  • Interests:FLORIDA!

Posted 27 June 2017 - 06:59 PM

Hey guys, very solid advice. Tim, that is pretty much what I was looking for I believe, is that considered RAID 5? Also, sunnyboy, is there a particular brand you use for the hard drives? I have used seagate and they seem ok. Also, when you say 'lose data' (*shudders), do you mean if I am not uploading images daily, or even turning on my drives daily, that they may fail? What would be considered a backup program? Thank you both, really appreciate your help.


Katy Danca Galli

Atmospheric UW Photography

http://katydancagalli.com


#5 TimG

TimG

    Sperm Whale

  • Moderator
  • 1816 posts
  • Gender:Male
  • Location:Amsterdam
  • Interests:Sunlight reefs, warm seas, fine wine, beautiful women. And Manchester City Football Club

Posted 28 June 2017 - 08:28 PM

Sorry Katy, I have no idea which RAID it is (!) and I'm not at home at the moment to check.

 

Really useful advice from sunnyboy. 


Tim
(PADI IDC Staff Instructor and former Dive Manager, KBR Lembeh Straits)
Nikon D800, Nikkors 105mm and 16-35mm, Sigma 15mmFE - Subal housing

http://www.timsimages.uk
Latest images: http://www.shutterst...lery_id=1940957


#6 bvanant

bvanant

    Sperm Whale

  • Team Wetpixel
  • 1812 posts
  • Gender:Male
  • Location:Los Angeles (more or less)
  • Interests:Science, photography, travel

Posted 29 June 2017 - 03:11 PM

You might also look at Drobo. Not Raid but it has the advantage of using drives of different sizes. We have 5 of them running continuously; 2 for video, 2 for photos and one backing up general stuff.  For drives for any NAS/Drobo/Raid box make sure you get NAS drives. I have a mix of Hitachi/WD/Seagates in my Drobos.

 

Bill


Bill
Canon 7d, Nauticam, Lots of glass, Olympus OMD-EM5, Nauticam, 60 macro, 45 macro, 8 mm fisheye, Inon, S&S, Athena Strobes plus lots of fiddly bits.
www.blueviews.net


#7 sunnyboy010101

sunnyboy010101

    Triggerfish

  • Member
  • Pip
  • 38 posts
  • Gender:Male
  • Location:Vancouver Island

Posted 30 June 2017 - 07:57 AM

Katy,

 

Yes, RAID 5 is what you want. I've used Seagate and Western Digital drives in my NAS boxes - whatever is cheapest at the time I bought them. I do not like to mix drive brands. I normally like to buy all 4 at one time.

 

As for data protection, 2 weeks ago I had one WD drive go bad in my LG box. I got the yellow light and warning that a drive had failed, so found another identical spec drive on amazon and bought it. Once arrived I just unplugged the dead drive and plugged in the new drive and about a day later the system was rebult and working. NO DATA LOSS. (it does take time to rebuild the volume when a drive is replaced).

 

With RAID 5, if one drive fails and you replace it fairly quickly, you will not lose data. If two drives fail, you WILL lose data, even with RAID5.

 

If you don't back up your computer frequently and it dies, you lose whatever was on it since the last backup. That's why I back up every day at about 1am.

 

DROBO does support RAID 5, as my main NAS these days is an 8-bay DROBO and it's running 4 drives (4TB each) and is running RAID 5.

 

 

Just to give you an idea, I've been "doing" computer stuff since 1979, and used to back up stuff on floppy, then CD, then DVD, then Blue-Ray and finally now RAID storage devices. My system has a 4TB data drive that is backed up daily to a server with the same single 4TB drive, then backed up to an LG NAS with 4x2TB drives and finally backed up to a DROBO with 4x4TB drives. Yes, I have THREE backups done every day after midnight.

 

Even with that, if my main computer were to crash at say 7pm, I would lose the day's work. Since backup programs can be run manually, if I were extra cautious, ever time I upload photos to the data drive I *could* run the backup to make sure they were saved. I don't because I'm probably getting complacent as I've not lost photos yet.

 

I have photos backed up including scanned slides from the '70s, so around 90GB total representing about 32000+ photos.



#8 bvanant

bvanant

    Sperm Whale

  • Team Wetpixel
  • 1812 posts
  • Gender:Male
  • Location:Los Angeles (more or less)
  • Interests:Science, photography, travel

Posted 03 July 2017 - 09:25 AM

It seems clear from the Drobo literature that they are not doing RAID in the ISO spec sense. You can use multiple drive sizes for example in a single box (not possible with traditional RAID).  In any case worth looking at; the 5D version is quite fast.

Bill


Bill
Canon 7d, Nauticam, Lots of glass, Olympus OMD-EM5, Nauticam, 60 macro, 45 macro, 8 mm fisheye, Inon, S&S, Athena Strobes plus lots of fiddly bits.
www.blueviews.net


#9 adamhanlon

adamhanlon

    Harbor Seal

  • Admin
  • 2090 posts
  • Gender:Male
  • Location:Lancaster, UK

Posted 04 July 2017 - 01:05 PM

The problem with Drobo is that in the event of the enclosure failing, the only way the get your data is to put the drives into another Drobo. They use a proprietary file system. RAID (at whatever level) is a standard configuration that will allow the data to be accessible in pretty much any RAID enclosure...

Adam Hanlon-underwater photographer and videographer
Editor-wetpixel
web | Flickr | twitter | Linkedin | Facebook


#10 cneal

cneal

    Lionfish

  • Member
  • PipPip
  • 50 posts
  • Gender:Male
  • Location:Charlotte, NC

Posted 30 July 2017 - 03:13 PM

As a retired IT guy I would recommend buying a spare drive for a RAID 5 array. When a drive fails a few years down the road locating an identical replacement can be a problem. If a spare is on the shelf it is a quick and painless fix.



#11 bvanant

bvanant

    Sperm Whale

  • Team Wetpixel
  • 1812 posts
  • Gender:Male
  • Location:Los Angeles (more or less)
  • Interests:Science, photography, travel

Posted 30 July 2017 - 04:04 PM

Adam: While that is true in theory, i had a couple of Wiebetech enclosures die and no other array could read the drives.  Maybe if they are from the same manufacturer with the same firmware it might work.

Bill


Bill
Canon 7d, Nauticam, Lots of glass, Olympus OMD-EM5, Nauticam, 60 macro, 45 macro, 8 mm fisheye, Inon, S&S, Athena Strobes plus lots of fiddly bits.
www.blueviews.net


#12 Barmaglot

Barmaglot

    Starfish

  • Member
  • Pip
  • 8 posts

Posted 09 September 2017 - 09:11 AM

As a server/infrastructure admin in my day job, I have to stress a point: RAID is not a backup! RAID provides business continuance in the case of a certain class of failures, though it is by no means a guarantee by itself. Furthermore, RAID5 is the least reliable type of RAID besides RAID0 (which, by strict definition, is not RAID at all), especially as your arrays get bigger.

 

To illustrate, imagine an 8-drive RAID5 array where a drive fails - you're immediately in degraded mode, where reads have to be reconstituted from parity data, and any additional drive failure kills your array dead. Then, once you replace the failed drive, or your hot-spare (if you have one) kicks in, the array, while still degraded, is put into maximum stress mode - the rebuild process reads all the data from all the drives to build the blocks on the new drive, and that's where one of three things can happen: (1) everything is fine, the array is rebuilt and you go back into normal mode, (2) another drive fails under stress (and drives from a single batch sometimes fail close together) and your array dies, or (3) as your data is rebuilt, a bad block is discovered on one of the remaining drives, and with redundancy gone, it cannot be rebuilt, resulting in a condition known as 'punctured stripe' - your array is still online, but some data is bad, and normally you will need to destroy the array, rebuild it with good drives and restore the data from backup.

 

RAID6 avoids these two latter scenarios by computing two sets of parity data, which enables it to survive double disk failures - at the cost of higher overhead (lower performance) and lower raw space to usable space ratio.

 

RAID10 (stripe of mirrors) has the highest performance, but also the worst raw to usable space ratio of the common RAID levels (only half of your raw space can be used), and it can survive most, but not all double-disk failures. To illustrate, in the same 8-drive array configured as RAID1, you have drives 1+2, 3+4, 5+6 and 7+8 configured as mirrors of each other, and then your data is striped across these four sets of mirrors. If drive 1 dies, the array can survive the failure of any drive other than drive 2 - better than RAID5 (which can survive no additional failures) but worse than RAID6 (which can survive any one additional failure).

 

An important feature of any RAID system is proper maintenance and monitoring - drives have to be periodically scanned, and an administrator has to be alerted of any errors, in order to take immediate corrective action.

 

In a home user scenario, unless your datasets are very very very large (tens of terabytes and up), there is little use for RAID. Almost inevitably, it will provide a false sense of security, while left unmaintained and will eventually die - taking your data with it. I have seen this scenario play out more than once. If your goal is data integrity, then instead of RAID, invest in a proper backup system, with versioning and off-site storage. Backblaze, for example, will let you back up unlimited data for just $50/machine/year. Unlike RAID, this will also protect you against data corruption, user errors, ransomware infections, hardware theft, site disasters, etc.