Jump to content


Photo
- - - - -

Big storage


  • Please log in to reply
55 replies to this topic

#21 james

james

    The Engineer

  • Super Mod
  • 9969 posts
  • Location:Houston TX

Posted 27 May 2006 - 01:02 AM

I'd be interested in a group buy too Eric. I'm planning to buy the same setup - the NV with either 400 or 500gb drives.

James
Canon 1DsMkIII - Seacam Housing
Dual Ikelite Strobes
Photo site - www.reefpix.org

#22 echeng

echeng

    The Blue

  • Admin
  • 5845 posts
  • Gender:Male
  • Location:San Francisco, CA
  • Interests:photography, ice cream, cello, chamber music, quadcopters

Posted 27 May 2006 - 02:34 AM

Great, guys! I'll work on it.
eric cheng
publisher/editor, wetpixel
www | journal | photos


#23 Alex_Mustard

Alex_Mustard

    The Doctor

  • Super Mod
  • 8597 posts
  • Gender:Male
  • Location:Peterborough

Posted 27 May 2006 - 03:30 AM

OK, but please don't post how little it ends up costing you! I will be green with envy. :-)

Alex

Alexander Mustard - www.amustard.com - www.magic-filters.com
Nikon D5 (Subal housing). Nikon D7200 (Subal housing). Olympus EPL-5 (Nauticam housing).


#24 3@5

3@5

    Eagle Ray

  • Member
  • PipPipPip
  • 385 posts
  • Location:south of france

Posted 27 May 2006 - 04:57 AM

yeah me too...
/paul
Paul Papadopoulos
canon 1dmkII + seacam silver + inon z220
My Webpage

#25 Paul Kay

Paul Kay

    Giant Squid

  • Industry
  • PipPipPipPipPip
  • 1793 posts
  • Gender:Male
  • Location:North Wales, UK

Posted 28 May 2006 - 08:50 AM

I read somewhere that there are two types of hard drive - those that have failed and those that will fail - a bit cynical but...

Seriously though, I have a collection of 12 varying hard drives - with two configured as a RAID set-up. BUT I also back up ALL my Raw files onto DVD too and am about to start adding a second set burned onto Delkin Gold DVDs - not a perfect solution but the best I have been able to come up with so far.

If anyone has spotted a fireproof DVD safe, I'd be interested to hear about it.

Paul Kay,Canon EOS5DII SEACAM c/w S45, 8-15, 24L,35L, 60/2.8 (+Ext12II) & 100/2.8 Macros - Sony A7II SEACAM 28/2 & 50/2.8 Macro - UK/Ireland Seacam Sales -see  marinewildlife


#26 manatee19

manatee19

    Wolf Eel

  • Industry
  • PipPip
  • 195 posts
  • Gender:Male
  • Location:Montreal, QC, Canada
  • Interests:U/W photo columnist and contributor to Diver Magazine (Vancouver) and En Profondeur magazine (Montreal), multimedia production.

Posted 28 May 2006 - 09:13 AM

After working with too many external drives (USB and/or Firewire) and experiencing various problems, we settled on a "pseudo-server". Simply an Antec case which can house many peripherals and has sufficient ventilation. This was an affordable option since we already had the necessary drives.

We bought a proper motherboard, and fitted the "monster" with a total of seven (7) drives and Windows XP. One HD is the C:\ drive and three others are permanently mounted. The last three are mounted in ventilated drawers. Those three drives are actually mirrors of the permanent ones. All but C:\ are 300Gb Seagate (5-year warranty). We did not go RAID for fear of our limited knowledge and total hate of computer problems.

Once a week, we swap the drawers with those we keep off-premises.

We use a freeware called SyncBack to keep all drives synchronised and it works fine.

Photo bank and presentations (Watchout and Powerpoint) are also copied on DVDs.

Sounds like paranoia but we can't affford to loose that stuff.

Not the most sophisticated and expensive proposal but it works... so far.

If my posting works, there should be a picture of the "monster" enclosed.

Enough computer talk... let's go back to u/w photography.

Michel
Michel Gilbert
www.sub-images.com

#27 manatee19

manatee19

    Wolf Eel

  • Industry
  • PipPip
  • 195 posts
  • Gender:Male
  • Location:Montreal, QC, Canada
  • Interests:U/W photo columnist and contributor to Diver Magazine (Vancouver) and En Profondeur magazine (Montreal), multimedia production.

Posted 28 May 2006 - 09:15 AM

Forgot to tell you that there are so many fans in the "monster" that it sounds like a B-17 ready to take off!

Michel
Michel Gilbert
www.sub-images.com

#28 JoelD

JoelD

    Wolf Eel

  • Member
  • PipPip
  • 151 posts
  • Gender:Male
  • Location:SW Misery...Err Missouri

Posted 31 May 2006 - 03:48 PM

Once I got past a compatability issue (it didn't like the maxtor drives I had) I fell in love with my ReadyNAS NV.
Joel D.
http://www.widgetsandmore.com

Nikon D300, D70s & a bunch of Ikelite stuff and all sorts of stuff my wife doesn't know about. Shhhhhh!

Join the National Association of Photoshop Professionals Today

#29 Craig Ruaux

Craig Ruaux

    Great Hammerhead

  • Member
  • PipPipPipPip
  • 788 posts
  • Gender:Male
  • Location:Oregon, USA

Posted 31 May 2006 - 06:51 PM

Count me basically in on a group buy as well, but it will need to be in August (I'm gone much of June/July) for me to participate, so don't wait for me if everyone else is ready to go. I'd be just as happy to buy a barebones system which I'll populate with SATA drives from the local CostCo.
Why would I take a perfectly good camera underwater??
D300, D200, D70, 12-24 f4 AFS DX, 60mm f2.8, 70-200 f2.8 AF-S VR, 105 f2.8 AF-S VR, Tokina Wunderlens.

Photo galleries @ Ruaux.net

#30 echeng

echeng

    The Blue

  • Admin
  • 5845 posts
  • Gender:Male
  • Location:San Francisco, CA
  • Interests:photography, ice cream, cello, chamber music, quadcopters

Posted 01 June 2006 - 11:13 AM

The group buy is underway. If you're interested, please contact me ASAP. Continental USA only.
eric cheng
publisher/editor, wetpixel
www | journal | photos


#31 JackConnick

JackConnick

    Orca

  • Industry
  • PipPipPipPipPip
  • 1397 posts
  • Gender:Male
  • Location:Seattle, WA
  • Interests:Sailing, diving, women, cats

Posted 01 June 2006 - 01:42 PM

I know this may be a low-tech solution, or at least a s/w solution, but are people converting their RAW files to Adobe Digital Negative format (.dng)?

This is by far the best way to recover about half the space your RAW files take and have a format that is much more usable in the future when camera manufacturers "discontinue" their support for old formats.

it is very easy to do on a trip, simple open up all your keepers in CS RAW and save as .dng. Then you can delete the original RAW files (I know it took me courage the first time!). The dng format works exactly like RAW, but no sidecar files are then necessary and I save about half on my huge E900 RAW files which are 18 MB!

Just something that I think should be part of everyone's workflow and saves a lot of space.

Jack

Jack Connick
Optical Ocean Sales.com

Nauticam, Subal, Sea & Sea, Olympus, Seacam, Gates, Ikelite, Zen, Fix, 10Bar, Kraken, Light & Motion, iTorch/I-DAS & Fantasea Line
Cameras, Housings, Strobes, Arms, Trays & Accessories. System Packages. Photo Expeditions.

Blog & Gallery: Optical Ocean Sales Blog - Flickr Galleries: Optical Ocean on Flickr


#32 echeng

echeng

    The Blue

  • Admin
  • 5845 posts
  • Gender:Male
  • Location:San Francisco, CA
  • Interests:photography, ice cream, cello, chamber music, quadcopters

Posted 01 June 2006 - 02:44 PM

You don't actually have to open the RAW files in Camera RAW. Instead, use the DNG converter: http://www.adobe.com/products/dng/

Also, some RAW formats are already compressed. For example, my Canon 1ds Mk II RAW files are smaller than your Oly RAW files.

but certainly, if your camera produces huge raw files, converting to DNG will save a lot of space!
eric cheng
publisher/editor, wetpixel
www | journal | photos


#33 wagsy

wagsy

    Blue Whale

  • Senior Moderator
  • 3849 posts
  • Gender:Male
  • Location:Cairns, Queensland.
  • Interests:Sewing and Knitting......no diving of course :-)

Posted 02 June 2006 - 05:51 PM

You guys have just to much $$$$$. :)

Amphibico Phenom & EVO PRO & Navigator 900
Share Your Underwater Videos | www.flykam.com.au


#34 hoovermd

hoovermd

    Sting Ray

  • Member
  • PipPipPip
  • 273 posts

Posted 02 June 2006 - 08:09 PM

I have a couple of the 500Gb LaCie drives on my G5.
I can attest to the issue of RAID failing and becoming unrecoverable. Happened to me once.
I now have a tape drive that uses LT02 Ultrium tapes (200 Gb native).
http://www.quantum.c...-2HH/Index.aspx

I now use one of the 500Gb drives as my main, with a daily backup to the other 500Gb drive and a weekly backup to the tape. Tapes go off-site to the safe deposit box at the bank.

May seem like overkill but the value really is in the data in the end :)
---------------
Mark

#35 Giles

Giles

    International Supermodel

  • Member
  • PipPipPipPipPipPip
  • 2618 posts
  • Gender:Male
  • Location:Cayman Islands
  • Interests:water and sun my friend, thats what turns me on.

Posted 22 July 2006 - 03:02 PM

I just had my first hard drive failure and so thought I would add onto this post as I am looking at better back up solutions ..

I am thinking about just a couple of USB2 or firewire (I have firewire ports I never use)
I was thinking there must be some software that can smart detect changes to a hard drive and update another drive automatically. (like a mirror I guess)

I don't get what RAID is .. but a lot of systems say they come with it .. but then there's different levels I looked here and got more confused

My current set up is 2x 200 gb Hard Drives.
Drive 1: Programs and settings
Drive 2: My personal files and media .. (almost full .. now dead)

I have a 120gb external USB drive that has Microsoft Back up files on it ..

I would like to have 2 back ups of everything .. That will probably be about 200gb worth of data. I feel getting a seperate box is a little over kill for me, but I don't know.

I think something like The Lacie Brick with SyncBackAuto Sync the drives or perhaps incremental updates ?

However there is the other option of something bigger that seems more prof ?

oh and I saw this other software that looks quite good too

I was hoping to get osme opinions ?

I want it to be a live back up of my current system and files as I change it .. and I want it to be quick and simple and zero work for me .. I am lazy.

me on the web >> journal / flickr / portfolio
i use >> my camera, eye & stunning good looks


#36 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 23 July 2006 - 11:26 PM

I am thinking about just a couple of USB2 or firewire (I have firewire ports I never use)
I was thinking there must be some software that can smart detect changes to a hard drive and update another drive automatically. (like a mirror I guess)


I use the program syncback. But there are countless other like it.

I don't get what RAID is ..

RAID stands for 'Redundant Array of Independant Disks' (or you can replace the I with Inexpensive, depending on who you ask). The commonly used RAID levels (or types) are 0,1,5.

Raid0: you dont want this if you want your data to be safe, unless you dont care about the data. This basically concatenates the disks into 1 large virtual disk. This means any disk failure will destroy the whole virtual disk. 2 500 GB disks give you 1 TB of data.

Raid1: this is sortof what you describe further down. It's also called mirroring. Each disk in a raid1 array has a counter disk, so it's always an even number. 2 500 GB disks gives you 500 GB of useable data, the other 500 is a live mirror. If any disk fails, the array can continue on the other disk while you rebuild the array with a new disk. This is normally all without interruption to the desktop machine.

Raid5: Here you always have whats called a checksum disk. Raid5 can only be done with 3 or more disks. With 2 disks it would just sortof be raid1 with 2 disks. In raid5 the raid software takes each block on all available data harddisks, and produces a checksum which is written on the block on the checksum disk. Simplified you could look at it like this. Say disk1 has the number 5 on block 1, and disk2 has the number 7 on block 1. Then the checksum disk has the number 12. Now if disk1 fails, the software sees 7 on disk2, and 12 on the checksum disk, and knows disk1 had a number 5 there. In case a disk fails in a raid5, the system continues in a degraded (slower) mode because it has to calculate all that data back. So, a raid5 with 3 x 500GB has 1TB of useable data with 1 disk 'lost' to the checksum disk.

In both raid1 and raid5 you can configure a 'spare' disk, which will automatically replace a failed drive. In both cases the raid software will have to rebuild the data, which can take anywhere from hours to days. During that time you are extremely vulnerable. Another disk failure will mean full loss of data. (there are other methods that can survive multiple disk failures but those are normally not for home use)

I think something like The Lacie Brick with SyncBackAuto Sync the drives or perhaps incremental updates ?


That would probably be fine for some time to come. Just make sure you stash one of your 2 backups someplace else :D

I dont think true image is what you want. I would avoid any kind of Windows software based disk mirroring. If such a thing even exists. Things like Syncback are not live mirrors. I run mine once a night automatically.

I want it to be a live back up of my current system and files as I change it .. and I want it to be quick and simple and zero work for me .. I am lazy.


If you absolutely want a live backup you probably need to get a raid1 or raid5 system. Imho you should get 2 though, so 1 raid system is a backup of the other, and then store the backup someplace else. I realise thats a hassle, but theft or fire are much bigger dangers than disk failure.

Ive actually been looking into this for home use recently also. I currently use firewire based external disks. I have 9 external disks now. 3 main disks, 3 backups, and 3 external backups I keep at my office. But this doesnt scale well. If only because windows can only handle so many firewire/USB2 harddisks.

Im now looking into eSATA arrays. Im probably going to buy a disk array that holds 5 eSATA disks. This is in theory much faster than firewire or USB2, as it basically acts as an internal harddisk. All you do is pull the SATA cable outside the desktop case. Unfortunately I'll have to buy 2 arrays, as I will not rely on RAID alone for my backups. Im in the lucky position that I can keep my 2nd array on another system and keep the 2 arrays mirrored over my DSL line. Too bad that ReadyNAS isnt eSATA.

Cor


Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter


#37 james

james

    The Engineer

  • Super Mod
  • 9969 posts
  • Location:Houston TX

Posted 24 July 2006 - 06:08 AM

This is a great thread and still going.

Giles: check out Eric's journal entry about storage, he has a good one.

A "quick and dirty" way to get started is to buy an external HD and have your desktop do a backup to it (incremental) every night.

Next up is to put a RAID card into your computer, but that means you should start from scratch and reinstall windows on that new partition.

I have a RAID 1 setup in my desktop with two partitions (4x 200 gig disks) with one 200g partition for system and one for files. I back that up periodically to a 500gig external Seagate drive.

And now, thanks to Eric and Wetpixel I have a 1.6TB ReadyNAS NV. It has 4 x 400gig drives in it with 1093GB of usable space.

Now I do my backups to that.

Cheers
James
Canon 1DsMkIII - Seacam Housing
Dual Ikelite Strobes
Photo site - www.reefpix.org

#38 Giles

Giles

    International Supermodel

  • Member
  • PipPipPipPipPipPip
  • 2618 posts
  • Gender:Male
  • Location:Cayman Islands
  • Interests:water and sun my friend, thats what turns me on.

Posted 24 July 2006 - 10:46 AM

Cor Thanks a lot .. thats some really usefull information for me .. and most of it phrased in a way a pretend geek can understand it !

Giles: check out Eric's journal entry about storage, he has a good one.

<{POST_SNAPBACK}>

I have done .. I also know that Eric's needs are a lot more demanding than mine and was hoping someone could guide me simpler yet better.

A "quick and dirty" way to get started is to buy an external HD and have your desktop do a backup to it (incremental) every night.

<{POST_SNAPBACK}>

Thats what I meant by better .. I have this already, but due to my often late night working habits or porn searching as I am sure someone will suggest, I often don't like the back up being done while I am using the computer, so I run a full backup once a month and incremental once a week, maybe I should do it once a day as incremental :D

Next up is to put a RAID card into your computer, but that means you should start from scratch and reinstall windows on that new partition. 

<{POST_SNAPBACK}>

I'd like ideally a backup system that is not part of my PC so that It can grow independant of the PC and my PC can grow independant of it. I am thinking a small RAID 1 system and 2 Firewire USB drives (one with nightly incremental and one with weekly FULL backup)

Does that sound feasable and sensible ? I am thinking a 500gb RAID 1 systems and the back up drives are probably 250gb each.

PS ... my old hard drive is fine .. it's the chipset on the drive that is screwed. My last resort is to send it back to the shop i bought the pc from in the UK .. however I was wondering if there was a cheaper solution in the states that anyone knew of to get the harddrive ghosted. Now I am off home to see what my back up contains.

me on the web >> journal / flickr / portfolio
i use >> my camera, eye & stunning good looks


#39 bmyates

bmyates

    Great White

  • Team Wetpixel
  • 977 posts
  • Gender:Male
  • Location:Seattle, WA
  • Interests:UW Photography, motorcycles.

Posted 24 July 2006 - 12:17 PM

...Simplified you could look at it like this. Say disk1 has the number 5 on block 1, and disk2 has the number 7 on block 1. Then the checksum disk has the number 12. Now if disk1 fails, the software sees 7 on disk2, and 12 on the checksum disk, and knows disk1 had a number 5 there...


:D I just found this thread, and most of it (including the above "simplified" explanation) are WAY over my head! :)

A few questions:

1. It sounds like the group purchase has already taken place(?) - is that the case (and I'm too late)?

2. I didn't see any discussion of "online" archival services, where you upload files to a website and pay them to store archives for you, which I assumed would be a good backup resource. Are none of you using anything like that? If you are, which one(s)?

Bruce Yates
www.UnderwaterReflections.com
Lumix GX8 in Nauticam, Canon 5DMkII in Aquatica, 1DsMkII in Seacam, G15 in RecSea...Inon Z240's...too many lenses
"If at first you don't succeed, try, try again. Then quit. No use being a damned fool about it." WC Fields


#40 james

james

    The Engineer

  • Super Mod
  • 9969 posts
  • Location:Houston TX

Posted 24 July 2006 - 12:26 PM

Hi Bruce,

You can probably still get in on the Group Buy. I'd contact Adam Tow for details as he was the organizer. Do a search on his name and you'll find Adam's website in a jiffy.

Cheers
James
Canon 1DsMkIII - Seacam Housing
Dual Ikelite Strobes
Photo site - www.reefpix.org