Jump to content


Photo
- - - - -

Big storage


  • Please log in to reply
55 replies to this topic

#41 buddy

buddy

    Sting Ray

  • Member
  • PipPipPip
  • 292 posts
  • Gender:Male
  • Location:Switzerland
  • Interests:diving, photography, golf

Posted 25 July 2006 - 01:42 AM

I got my ReadyNAS NV with 4x500 GB today (not from the group buy since I am in Europe but got a good pricing anyway). My question to you advanced users is now how do you keep your files stored. Do you organize the files by topics folder and under each folder you keep a subfolder for different image qualities, such as RAW, TIFF, JPEG, etc

example:

UNDERWATER
-Bahamas 2005
--RAW
--TIFF
--JPEG
-Sardine run 2006
--RAW
--TIFF
--JPEG
TOPSIDE
etc.

or do you keep all the different images by quality seperate in different folders, such as

Captures RAW
-UNDERWATER
--Bahamas 2005
--Sardine Run 2006
--etc.
-TOPSIDE

and

Captures TIFF
-UNDERWATER
--Bahamas 2005
--Sardine Run 2006
--etc
-TOPSIDE

Any other good ideas to keep things well organized ?

appreciate any tips. Thanks

Juerg
www.jvpictures.com
Nikon D4s, Nikon 16-35 /f4 VR II, 70-180,105mm f/2.8, Subal ND4 housing and ports, 2 Subtronic Fusion (flash and video lights), Sigma 15mm /f2.8, Subal 4" Minidome


#42 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 25 July 2006 - 01:57 AM

I keep mine sorted like this:

Originals
Trip1
Trip2
......


Cleaned
Trip1
Trip2


Projects
Contest1
Contest2
Article1
Article2

All of this gets added to Extensis Portfolio.

Cor
Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter


#43 Drew

Drew

    The Controller

  • Video Expert
  • 10708 posts
  • Gender:Male
  • Location:GPS is not reliable in South East Asian seas

Posted 25 July 2006 - 05:42 AM

I use iView Media Pro to catalog the files on the server. That way the dates and location are all stored. Even GPS if wanted. I throw them all in Original and Modded folders, with exact names and create a 2nd catalog in iView for the modded pics.

Drew
Moderator
"Journalism is what someone else does not want printed, everything else is public relations."

"I was born not knowing, and have only had a little time to change that here and there.


#44 james

james

    The Engineer

  • Super Mod
  • 9969 posts
  • Location:Houston TX

Posted 25 July 2006 - 05:49 AM

Well, everyone does this differently. I don't use an asset management program, although I should.

Personally, I have a structure like:

Photos:
Underwater
Bali-05-08
RAW
Keepers
Web
Cozumel-05-09
Family
Travel
Nature
Etc

Cheers
James
Canon 1DsMkIII - Seacam Housing
Dual Ikelite Strobes
Photo site - www.reefpix.org

#45 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 25 July 2006 - 06:53 AM

I use iView Media Pro to catalog the files on the server. That way the dates and location are all stored. Even GPS if wanted. I throw them all in Original and Modded folders, with exact names and create a 2nd catalog in iView for the modded pics.

<{POST_SNAPBACK}>


Ive only recently started using Portfolio and use files like:

cor_20060913_3232.nef
cor_20060913_3232_c.tiff
cor_20060913_3232_cts2006.jpg

First one being the original, second one being a high res cleaned version, and third being a version meant for a specific project/article/magazine/customer/whatever. I did that so I could easily search for all versions of a certain image by providing the base filename as a search. It's the same idea, only within the same catalog/database. I refuse to use iview media pro until they finally make all EXIF info searchable.

Cor
Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter


#46 cronix

cronix

    Wolf Eel

  • Member
  • PipPip
  • 142 posts

Posted 15 September 2006 - 03:08 AM

A hot spare is just a hard disk that is unused until one of the hard disks of your raid volume dies. At that time the unused hard disk automatically replaces the faulty disk, which can take several hours. During that time you system will run in a 'degraded' mode, which usually means it runs slower. If you loose a second harddisk while running in degraded mode (and failures do tend to happen in clusters) you will loose everything.

But no matter if you have RAID, or even a hot spare, these systems rely on software and hardware that can, and does fail. It is not uncommon for raid systems (I happen to manage hundreds of them) to just go all screwy on you with no chance of a full recovery.

I would personally not rely on the 'safety' of a raid volume alone to protect what I count as some of my most valued posessions, my photos. Not only do you have the above mentioned problems, but you can have a fire, or even more common, plain old theft. All those shiny buttons look really tempting. No amount of raid is going to get your images back then. Close your eyes, imagine you coming home, and your tv, your dvd player, your computer, and everything around your computer including your raid system is stolen. No more photos.

I would always keep a copy someplace else.
Cor


Man, you know a lot about computers ;) :)

Just wondering how many of you have backups? Yesterday TWO harddisks died in one day. :blink: The first one was active in my raid 5 array. The other one was my hot spare. Image, my hot spare died during the raid recovery. I still have all my data, because of 3 disks still running. I am currently rebuilding the array. I also bought a new computer case with lots of extra fans, you never know

I am seriously thinking about getting two 500GB disks extra, just as an extra backup for my raid array.

I would almost dust off my F5 : :D angry:
D2X, 10.5mm, 12-24mm, 60mm, 105mm, SB800 in Subal housing, Seacam: housing, S45 viewfinder, Fisheye dome, superdome, wet diopters, 250 TTL Digital strobes (2)
The website of André Crone can be found at http://www.elysia.nl

#47 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 15 September 2006 - 04:34 AM

Just wondering how many of you have backups? Yesterday TWO harddisks died in one day. :blink: The first one was active in my raid 5 array. The other one was my hot spare.

This is very common with raid arrays. Failures often have an underlying problem. Maybe heat (fan died?) or power problem or disks are too old (and bought at the same time so all the same age), etc etc. This is the main reason that imho you should not depend on RAID as your only line of defense.

Cor
Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter


#48 cronix

cronix

    Wolf Eel

  • Member
  • PipPip
  • 142 posts

Posted 15 September 2006 - 05:46 AM

This is very common with raid arrays. Failures often have an underlying problem. Maybe heat (fan died?) or power problem or disks are too old (and bought at the same time so all the same age), etc etc. This is the main reason that imho you should not depend on RAID as your only line of defense.

Cor


I agree, you need more that RAID; or just a disk. I also think it's heat. That why I am going to put the stuff in another case (as soon as the raid set is recoverd).

But too old? I have had three disks die in one year; all not even a year old.
D2X, 10.5mm, 12-24mm, 60mm, 105mm, SB800 in Subal housing, Seacam: housing, S45 viewfinder, Fisheye dome, superdome, wet diopters, 250 TTL Digital strobes (2)
The website of André Crone can be found at http://www.elysia.nl

#49 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 15 September 2006 - 07:52 AM

I agree, you need more that RAID; or just a disk. I also think it's heat. That why I am going to put the stuff in another case (as soon as the raid set is recoverd).

But too old? I have had three disks die in one year; all not even a year old.

Ofcourse 1 year shouldnt be a problem. If they're all the same disk, google for that type and see if its got known issues. Some series have problems that cause high failure rates. One year old HDs should be covered by any manufacturer.

It sounds indeed like you have heat problems.

Cor
Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter


#50 echeng

echeng

    The Blue

  • Admin
  • 5845 posts
  • Gender:Male
  • Location:San Francisco, CA
  • Interests:photography, ice cream, cello, chamber music, quadcopters

Posted 15 September 2006 - 08:10 AM

I always have at least two versions of my data spinning, and have RAW files archived to DVD. One version is on RAID 5. That way, I can lose up to two drives at any given time and be back in business, in theory. :blink:

Have 25 drives at the moment between 5 systems and 2 NAS boxes, and tend to lose 1-2 drives per year.

I also avoid Western Digital and Maxtor. I use Seagate and Hitachi only. WDs die all the time, in my experience. :D
eric cheng
publisher/editor, wetpixel
www | journal | photos


#51 cronix

cronix

    Wolf Eel

  • Member
  • PipPip
  • 142 posts

Posted 15 September 2006 - 08:19 AM

I always have at least two versions of my data spinning, and have RAW files archived to DVD. One version is on RAID 5. That way, I can lose up to two drives at any given time and be back in business, in theory. :blink:

Have 25 drives at the moment between 5 systems and 2 NAS boxes, and tend to lose 1-2 drives per year.

I also avoid Western Digital and Maxtor. I use Seagate and Hitachi only. WDs die all the time, in my experience. :D


What happened to a light box and slides......

I do almost the same. Raid5, a backup on a harddrive and 2 DVD backups. I just don't trust DVD's.

My raid 5 is dirt cheap. It's a 250 euro PC filled with disks (still within warranty ;) ) Fedora Core Linux and software RAID. Performance is not an issue my 100mb network is still the bottleneck.
D2X, 10.5mm, 12-24mm, 60mm, 105mm, SB800 in Subal housing, Seacam: housing, S45 viewfinder, Fisheye dome, superdome, wet diopters, 250 TTL Digital strobes (2)
The website of André Crone can be found at http://www.elysia.nl

#52 cronix

cronix

    Wolf Eel

  • Member
  • PipPip
  • 142 posts

Posted 16 September 2006 - 08:15 AM

Everyting is saved even though 2 disks died this week. I just rebuild my raid array. I also rebuild the computer in a rack mount case with 3 extra fans to keep the stuff cool. Main advantage is that this case can easily hold another 5 disks. So it wont be long before I will put in a second raid array.

And I just bought a WD NetCenter network attached 500GB disk to backup my most important image files. I can always buy more of these if I like this one.
D2X, 10.5mm, 12-24mm, 60mm, 105mm, SB800 in Subal housing, Seacam: housing, S45 viewfinder, Fisheye dome, superdome, wet diopters, 250 TTL Digital strobes (2)
The website of André Crone can be found at http://www.elysia.nl

#53 Drew

Drew

    The Controller

  • Video Expert
  • 10708 posts
  • Gender:Male
  • Location:GPS is not reliable in South East Asian seas

Posted 16 September 2006 - 06:58 PM

I always have at least two versions of my data spinning, and have RAW files archived to DVD. One version is on RAID 5. That way, I can lose up to two drives at any given time and be back in business, in theory. :blink:

Have 25 drives at the moment between 5 systems and 2 NAS boxes, and tend to lose 1-2 drives per year.

I also avoid Western Digital and Maxtor. I use Seagate and Hitachi only. WDs die all the time, in my experience. :D


Wow Eric, my experience is entirely opposite. My Seagates have failed miserably (even my old UWSCSI ones (yes I still have them)) over the years. I finally stopped using them for backup. WD and Hitachi have been ok with a bi-annual failure rate. but my most stable drives have been the Maxline Maxtor 3 years and still running!
Temperature wise, I've found the Maxtors and WD running hotter than the others by 10F, esp the SATA II drives. My seagate HD in my laptop runs relatively cooler vs the hitachi it replaced.

Drew
Moderator
"Journalism is what someone else does not want printed, everything else is public relations."

"I was born not knowing, and have only had a little time to change that here and there.


#54 photovan

photovan

    Great White

  • Moderator
  • 983 posts
  • Gender:Male
  • Location:Brisbane, Australia

Posted 16 September 2006 - 09:12 PM

A question for you guys loosing drives; out here in Aus we seem to be able to get two standards of drive from WD and Seagate; 2 year warranty or 5 year warranty. The 5 year warranty ones have 16MB cache and seem to be the ones being spec'd into RAID boxes. Do you get different models with different warranties, and if so which are you using?

Darren Jew  |  Australia  |  darrenjew.com  |  fotofrenzy.com.au

Canon EOS1Dx   |   EOSM   |   Nauticam  |   Inon Z240


#55 photovan

photovan

    Great White

  • Moderator
  • 983 posts
  • Gender:Male
  • Location:Brisbane, Australia

Posted 16 September 2006 - 09:23 PM

And anecdotal evidence only, but since we put the office on its own dedicated electrical circuit (seperate from the house) we have had a lot less trouble with drive failures. I might be miles off the mark, but having its own circuit seems to result in no instances of power fluctuations in the office - when the washing machine/pool filter/aircon switch on and off in the house, it does not effect the office. We live in a house that was originally built and wired for the 1950's; now it's 21st-century-ready, power-wise anyway.

Darren Jew  |  Australia  |  darrenjew.com  |  fotofrenzy.com.au

Canon EOS1Dx   |   EOSM   |   Nauticam  |   Inon Z240


#56 cor

cor

    The Hacker

  • Admin
  • 1994 posts
  • Gender:Male
  • Location:Amsterdam, Netherlands

Posted 16 September 2006 - 11:58 PM

Wow Eric, my experience is entirely opposite.

Every brand has had bad series. So it is not surprising that people have bad experiences with brand 'X'. If you try the same brand a few months later they're usually ok again.

Cor
Cor Bosman - Nikon D2X Subal ND2 - Nikon D7000 Subal ND7000
website | tripreports/journal | facebook | wetpixel map | twitter