Jump to content
underwatercolours

Hitachi 1TB Hard Drive (and RAID discussion)

Recommended Posts

I have found that backup solutions on the Mac are lacking. SuperDuper is great (I use it) but it doesn't version my files. I want somethinglike Retrospect that can version my files, protecting me from corruption and from myself.

 

All of the backup software I've tried on the Mac that support versioning are terrible. Maybe time machine will just solve all my problems.

 

Oh, and about the partioning thing: I like having a separate system partition so I can reinstall from scratch without touching my data. On Windows, this was an absolute requirement. Maybe I am thinking with a PC brain...

Share this post


Link to post
Share on other sites
Have any of you given thought to Time Machine yet? It seems like a compelling feature although I don't know how effective it will be for databases and I'm a bit concerned with stability. How will you set up your drives to use it?

 

 

I am not sure quite how I will use it yet.

 

Usually though with any updates to OS or other software I take a very conservative approach in bring the items on line. First I go by the principle if it ain't broke don't fix it - meaning unless there is something that has developed as a conflict or will not run on my production machine, the machine does not get touched.

 

Then I start putting things on my other machines, including an old iBook, tiBook or MacBook Pro to see how it works and get the feel for the new software for a bit and see what issues exist.

 

Then I will put in another hard drive or boot external boot drive for the main machine I use. My main boot drive on my tower is still 10.4.5 and I have no real compelling reason to change it. I have 10.3.9 and 10.4.9 on another drive on the machine when I want to jump to an older OS or run the new FCP Studio 2. (As an aside I also do what Eric mentioned, I make a disk image of my clean installs in case I ever need it. There is a back up image of 10.4.9 with FCP Studio and nothing else on it.)

 

At this point though I guess I am pretty much Tiger and have not gone back to Panther in any form for quite awhile.

 

As to the Time Machine feature, I will go with the wait and see tatic for a bit. I am used to doing what I do now in terms of back-up and it has served me well. I walk away, watch some tv or something and things are taken care of and it is been reliable.

 

Guess that is a long way of saying I am scared of all these new fangled thingies :rolleyes: It is has been years since I have had an issue (a Jaz Drive and some LaCie Drives) so alot of what I do is just by rote and what I feel comfortable with.

 

Eric have you tried Retrospect for the Mac yet? It is pretty good. The one thing I also do is label each file/project/version like 20070825_Film so I can do a search and know I have different versions.

Share this post


Link to post
Share on other sites

Wow, it must be nice to not have problems. Out of 6 750GB's I've added to my Mac Pro, one is already dead after a month. I've mailed one MBP in once and another 4 times in the last year, replaced two failed boot drives in additional systems and lost a notebook DIMM. I'm on my 3rd router in the last year as well. I don't consider this year to be much different than others (except that it's mostly Mac now). Failures are common for me.

 

What does versioning within a backup application offer a home or small business user?

Share this post


Link to post
Share on other sites
Finally, after having been a RAID firmware developer in a previous life, I can honestly say that I'd never use a RAID 3/4/5/6 product unless I absolutely needed to and, these days, drives are so big that most of us really don't need to. RAID 0/1 is relatively easy to implement and is fast. Parity-based RAID is sufficiently hard that most vendors are incapable of doing it right. I know because (1) I was responsible for some eye-opening bugs myself, and (2) I did dozens of competitive evaluations and not one products worked when trouble started. I've owned plenty of RAID products other than my own, and the few times I needed them to work they always failed. No need to use parity drives when the system will lose data in any case.

 

Remember, redundant RAID levels are needed when the system absolutely must continue to run in the event of drive failure. Few of us actually need that provided we have an effective recovery mechanism (which Macs particularly do, I love that!) since all we risk is the work we are currently doing. Considering that many of our systems lack sufficient redundant hardware, we are fooling ourselves in believing that RAID actually buys us much. How many RAID users consider just how many different kinds of failures their systems can actually tolerate? If a drive failure costs you a half day or day to recover but otherwise results in no data loss, isn't that good enough?

Oh great, now you tell me my RAID card is just sucking electricity for nothing. :P Ah well

Actually my workflow uses RAID 1. My boot partition is RAID 0 which has the apps partition, All my apps write data to the RAID 1 partition (whatever app files so it's backed up). The boot partition is hardly over 100GB with various software.

I hate that VM files fragment the disk (even if OSX defragments the files) so that is why I asked about the SWAP partition. On the G4/5 computers, there were ways to set up a SWAP partition so VM files are written to that partition only. The drives stay less fragmented vs a single boot partition. Last I looked MacIntels have a different boot system that precludes this but if Eric has found a way, he MUST SHARE! ;)

Craig, even if you don't run a business, you do spend a lot money getting those images and in the unfortunate event of your on-site drives being lost for whatever reason, an offsite system is a great safety tool which saves the day. Maybe it's a Californian "watch out for the big earthquake" mentality, but off site is a good buffer. My offsite is on another continent.

As for your hardware failures, there's been a belief that some people have the inate ability to cause electrical failures more than usual. Something to do with the energy field they emit :rolleyes: Maybe you work em too hard too. :)

For using rsync to simulate Time Machine, this guy has a script.. I haven't tried it yet because I only backup once a week for the boot partition since the data files are already backed up.

Are you all using Leopard already? I didn't find it too stable and decided to can it a few months back. But my build was at least 7 months ago. Too lazy to get updated. Anything new?

Share this post


Link to post
Share on other sites
Wow, it must be nice to not have problems. Out of 6 750GB's I've added to my Mac Pro, one is already dead after a month. I've mailed one MBP in once and another 4 times in the last year, replaced two failed boot drives in additional systems and lost a notebook DIMM. I'm on my 3rd router in the last year as well. I don't consider this year to be much different than others (except that it's mostly Mac now). Failures are common for me.

 

What does versioning within a backup application offer a home or small business user?

 

 

Sounds like you got hit with a rash of things. I have had some things go wrong of course over time. One iBook issue, one G4 processor and couple of other things. On the hard drive side of things have been pretty lucky (except for Lacie Drives) and one or two casings which had bad connectors/wires and one that had an intermitent power supply issue (that one was really weird). But no data loss (other than the Lacie but they were also backed up.) But all of these were over the last six years or so. The power adapter and the loose port are the only two things I can think of in the last year.

 

I have had pretty good success using Seagate Drives the last few years, have been reliable.

 

Was not sure if you were speaking to what I described in versioning 20070827_V1_Film etc. but with many things I am adding and modifying as I go along. Depending on what it is (documents, photos, film) I need to either specifically refer back to other versions and also once in awhile a file will become corrupted. So saving periodic versions can save a lot of time if I have a file go bad. Fortunately that has not happened in quite awhile. Also for some things I set up a "base" version of a project then try alternate versions or ideas which may or may not be what I want down the road. So starting from a base that is 70% of the final idea as oppossed to rolling back a project can be helpful. And sorting by thiss naming convention helps in the Finder.

 

Again alot of this depends on what I am doing.

Share this post


Link to post
Share on other sites

I only use Seagate and Hitachi now. Although.. a friend ordered 4 1TB Hitachi drives, and three of the four were DOA (!).

 

Re: file versioning: I've had files get corrupted and had to go back to retrieve the ones that were still intact. Because I hate Retrospect for the Mac (see below), I no longer have versioning. It's been scary. Who knows what has been corrupted and backed up? You can't test all your files.

 

As for Retrospect on the Mac, the reason I don't use it is that it's 3x slower than Retrospect for the PC was when copying over the network. I am on gigabit, and my Retrospect backups clocked at 4-6 MB/s. Pathetic! When copying in the Finder, I get 30+ MB/s.

 

Also, Retrospect for the Mac is a pathetic excuse for an application. The application itself is a piece of crap, even though it probably works.

 

Actually, all of my backup programs on the Mac run really slowly when not targeting a local disk. I have tried almost all of the backup programs out there on the Mac, and they're all 2-3x slower than what I used to get using PC-based programs. It's no big deal if you're copying a couple hundred GB, but when you get up in the TB range, a slow network is death.

 

Those of you on Windows should check out SyncBackSE from 2BrigthSparks. A software developer friend whom I trust swears by it.

Share this post


Link to post
Share on other sites

Actually, I was only speaking of parity RAID, RAID 0 and 1 are fine. Thing is, if you aren't doing parity you don't need RAID hardware. If you have some much data and performance requirements that dedicated hardware is needed, then that's another matter. Sometimes storage appliances make sense.

 

As a RAID engineer for almost a decade, what I learned is that parity RAID is the work of the devil. Actually, what I learned is that you don't want to do RAID at all, the best place for it is in the filesystem. End users can't influence that though, so RAID is what we get. The last place I worked was a startup intended to replace RAID with something much better. It tied its fortunes to Infiniband and became something I didn't want so I backed out. There are much better ways to store data than RAID!

 

I didn't mean to denigrate offsite backups. A compelling case can be made for those, I just don't do them. :-(

 

No, I'm not running Leopard. I'm too new to OS X and not sufficiently curious. I believe Time Machine could be interesting and useful though. Oddly, back in the VMS days I found automatic file versions to be an annoyance. Now I want it.

 

I wouldn't say I've had a rash of failures. Hard drives fail for me all the time. My original MBP was an early model with lots of issues. A later MBP was shipped to me defective. The rest of the failures are hard drives and memory, though out of the 8 Mac products I've owned 4 of them have had failures that required warranty service withing the first month. I'm not convinced that the hardware reliability of Macs is worse than PCs but I'm not convinced it's better either. I'm still happy with the conversion.

 

Regarding Macs and network performance, I suspect trouble there. Panther's network performance was terrible and Tiger's Samba performance is too (surprising since Samba generally works well). My Macs share pretty well over wired but mounting filesystem images over wireless seems to corrupt them. Copying data from a Mac to a PC over gigabit (via Samba) yielded only 8 MB/sec for me. Whenever I get a lot of network traffic going my Mac Pro beachballs like mad despite virtually no CPU usage. It used to be that OS X had only a few spinlocks for the entire kernel. That's supposed to be improved though how much I don't know. On my old dual G5 running panther, two simultaneous network copies would lock out the entire machines. I eBay'ed it. I wonder if backup performance over a network isn't Apple's fault especially if the network devices aren't also running OS X.

Share this post


Link to post
Share on other sites

I wrote a lot about slow network performance between Macs and Windows. Here's how I fixed it:

 

http://echeng.com/journal/2006/09/13/mac-o...-to-windows-xp/

 

I find fast traditional copies over finder / explorer, but really slow performance when going through every program out there that does file compares / copies.

Share this post


Link to post
Share on other sites

Do you believe then that slow network performance within apps is another problem or simply an indicator of an incomplete solution? Do you get slow app performance with networks FS's other than SMB? Considering that Finder and Explorer are just apps I'm intrigued. I was discouraged by the slow performance between OS X and Windows and it delayed my adoption. Once I replaced my HTPC's with Minis things got a lot better. I was dumbfounded that GigE seemed useless.

 

It's easy enough to disable IPv6 in Windows without using the command line. Why did you find that necessary?

 

Anyway, I've included the changes you've recommended even though I have little need at this point. It would be nice to get better than 8GB/sec next time I copy my 200+ GB iTunes library to someone's Windows box. :-)

Share this post


Link to post
Share on other sites

Hi Mods,

 

This is an interesting thread with heaps of good pointers about networking and RAID - any chance of splitting it into a new topic?

Share this post


Link to post
Share on other sites
As for Retrospect on the Mac, the reason I don't use it is that it's 3x slower than Retrospect for the PC was when copying over the network. I am on gigabit, and my Retrospect backups clocked at 4-6 MB/s. Pathetic! When copying in the Finder, I get 30+ MB/s.

 

Also, Retrospect for the Mac is a pathetic excuse for an application. The application itself is a piece of crap, even though it probably works.

 

 

Yeah it can be slow even on local drives, since using eSATA not quite as painful, and have gotten fairly decent speeds, though I do not have the backups via a network. Guess I am used to backing up the Macs this way so overlook the issues to some degree. I actually have a copy of ProSoft Data Backup that came with a recent hard drive purchase, but misplaced the disc. Was going to try it out and see how it goes (If I find the disc). Will let you know how that goes (unless you have tried it and can warn me not to bother :rolleyes: )

Share this post


Link to post
Share on other sites

You beat me. I'm doing the same but I have to migrate drives in and out of the chassis so it's taking me more steps. Eventually I'll have an 8x750 arrangement.

 

Being relatively new to Macs, I have a couple of requirements. First, I have sensitive files that I want password-protected beyond login since I have guests who frequently help themselves to my machine when they come over and I'm not good at logging out. Second, I keep applications that I've downloaded so that I can install them on new machines and I don't want OS X to find them and install them in context menus (pinwheel city). My solution is to use mountable files for both those problems. That facility provides encryption plus I can leave them unmounted. Trouble is, OS X is constantly corrupting those files (I think because I access them over wireless). Until I figure out how to stop that, I can't commit to RAID 1 since corrupting one of those files means losing everything in them. I don't have a separate backup scheme nor do I want to have to use one every month to compensate for OS X bugs.

 

How do you guys archive downloaded apps and installers. How do you use encrypted filesystems? Anyone having these corruption issues other than me?

Share this post


Link to post
Share on other sites

Hey, Craig. I just tell spotlight not to index my downloads directory. It is automatically backed up via Chronosync to a directory on a NAS box, which is mounts and unmounts as necessary.

 

My sensitive files are located in an encrypted, password-protected sparse image that lives in my filesystem. I haven't had a problem yet with corruption, and it is on a striped volume.

 

Btw, I did it so fast because i have superduper backups on external drives. Luckily, I didn't have to swap anything in and out, but I did have to go to the store to replace a 1TB drive that was DOA. :rolleyes:

 

Um. did we lose everyone else? i'll bet most people are saying... "WHAAAAAAAAAAAAAAAAAAAAAA? I just want to take pictures!" :P

Share this post


Link to post
Share on other sites
Um. did we lose everyone else? i'll bet most people are saying... "WHAAAAAAAAAAAAAAAAAAAAAA? I just want to take pictures!" ;)

 

Eric, EXACTLY......., but I was thinking I just want to be underwater taking pictures!!!!!! :rolleyes:

 

Unfortunately we can't set this stuff up underwater..........

 

Carol :P

Share this post


Link to post
Share on other sites

I'm swapping out Seagate and WD drives and moving stuff around so I'm having to do multiple SuperDuper operations. Mine probably won't finish until tomorrow. I've also got to replace the drive that's been slowly failing. I've got a plan...

 

I've decided that it's not a good idea to mount a file image over the network and doubly bad to do it over wireless. I've switched to sparse images, and that would probably help, but I still think my corruption problem is caused by network latencies and timeouts. It's pretty clear that my problems only start when I open a 20+GB file image over wireless. Takes forever too. :-(

 

I've tried the spotlight exclusion trick but I'll try it again. Sometimes I've told Spotlight to ignore a whole drive and it just ignores me instead. That's why I went to the unmount trick. If that works then I'll get rid of the disk images except for the encrypted one and leave my backup volume mounted. :-)

 

Thanks.

 

P.S. I can't imagine doing this on Windows. :-p

Share this post


Link to post
Share on other sites

Craig

Try PGP. I use it for work files. I block off an entire folder with all my work in it. It's mandatory for me now.

Share this post


Link to post
Share on other sites

OK, this may be a little off topic, but hopefully someone can answer this. I have an Iomega NAS200d in my network, which has two 160GB Hitachi drives, and would love to upgrade to larger drives like the 1TB Hitachi mentioned in this thread, but the manual says this is not possible. Is there a way to do this? Iomega will say no, since they want to sell me a larger NAS. Anyone have an idea what would happen if I try this? Any thoughts and suggestions would be appreciated, since the current drives are full, and I'm always having to delete old files, or move them to off network backups.

 

Thanks,

Vic

Share this post


Link to post
Share on other sites

Texas diver,

 

The answer is likely "no", not easily anyway. The problem is that you would need a way to partition/format/install the Iomega system with the new hard drives, and I don't believe the Iomega unit supports this.

 

I have an Infrant produced unit that you can actually take 1 drive out, put in a bigger 1...wait for "RAID" rebuild to complete and then do the next drive. After replacing all of the drives you would have the larger capacity. Likewise, I could just take all of the drives out and install larger ones and setup the system from scratch.

 

The downfall of almost all small/entry NAS systems that any of us would consider is performance. The Infrant I have maxes out at 23MB/s on a good day, if you have parallel/concurrent writes the performance drops...and this is over GigE.

 

The home office/media center storage market is still not addressed. There are no affordable, reliable and decent performing NAS systems on the market. Its ridiculous, since any reasonable PC hardware can run Linux with software RAID and such to do it. I'm just too lazy, I want someone else to set it up for me. I want something that just works, which we are a Mac household (other than work required VMs with Windows). I will likely go the OpenFiler path, unless I can find a bargain on a StoreVault or some other entry business class NAS.

 

The other option would be to look at RAID based USB/FireWire enclosures. There are a few out there, prices vary greatly...most are RAID1 or RAID0 with 2 drives only, there are a few that offer RAID5 or RAID1+0 but these go up in price significantly.

Share this post


Link to post
Share on other sites

The only thing about RAID0/1/5 is power surge. Power surge within your computer can fry all your HDs. SO it is still prudent to backup to DVD or an external RAID Array.

Share this post


Link to post
Share on other sites
Um. did we lose everyone else? i'll bet most people are saying... "WHAAAAAAAAAAAAAAAAAAAAAA? I just want to take pictures!" :)

This RAID stuff is way over my head, and I suspect, way too complex for my needs.

I have a Mac Pro with two 500GB drives, each about half filled, and presently using the second drive for my Time Machine Backup. Critical files in my house are Aperture Vault, iPhoto Libraries, and some MYOB and Quicken files. Contacts, Calendar etc are backed up automatically to .Mac.

Any reason I cannot install a 1TB Ethernet or wireless drive and just target the Time Machine for all family computers in the household to this drive. Sounds simple and fairly reliable for my needs. Anyone disagree?

Edited by loftus

Share this post


Link to post
Share on other sites

That sounds like a good idea Jeff. It could take a long time to backup over gigabit ethernet, so schedule your backups for night time :-)

 

It also couldn't hurt to store that backup at your office - i.e. in a different location from your main machine.

 

Cheers

James

Share this post


Link to post
Share on other sites
This RAID stuff is way over my head, and I suspect, way too complex for my needs.

I have a Mac Pro with two 500GB drives, each about half filled, and presently using the second drive for my Time Machine Backup. Critical files in my house are Aperture Vault, iPhoto Libraries, and some MYOB and Quicken files. Contacts, Calendar etc are backed up automatically to .Mac.

Any reason I cannot install a 1TB Ethernet or wireless drive and just target the Time Machine for all family computers in the household to this drive. Sounds simple and fairly reliable for my needs. Anyone disagree?

 

 

Here is a great site to show how raid works in Pictures...

 

http://www.acnc.com/04_01_00.html

 

The most USEFUL RAID Array for photogs are RAID 0, RAID 1 or RAID 5. You can see RAID in action by choosing the # on top of the diagram and clicking the diagram to start the animation. Enjoy!

Share this post


Link to post
Share on other sites

The purpose of RAID (ignoring RAID 0) is to reduce or eliminate downtime. It can also provide performance advantages, though no RAID configuration can outperform RAID 0. Keep in mind that RAID 0 is not actually a RAID level and offers no redundancy.

 

The problem is that people mistakenly believe that RAID can take the place of backups. It cannot. ChrisJ points out that there are problems that effect the data on multiple drives simultaneously and those are not limited to power spikes. The simplest examples are OS failures and user errors. There's a whole category of problems which RAID is vulnerable to. Being and ex-RAID developer I can tell a lot of stories.

 

I believe that most photographers, of which I count myself as one, do not need reduced downtime due to drive failure. Some do. For that reason, most photographers don't need RAID at all. All photographers need a true backup system.

 

My backup system consists of a separate set of disk drives residing on independent controllers and implemented using asyncronous duplication. That eliminates almost all possibility of corruption of both stores simultaneously. If my data were more critical I would use remote backups like James suggests and some here implement. I do not use RAID. I do use data striping (RAID 0). If file systems were smarter, RAID 0 wouldn't be attractive either.

Share this post


Link to post
Share on other sites
Here is a great site to show how raid works in Pictures...

 

http://www.acnc.com/04_01_00.html

 

The most USEFUL RAID Array for photogs are RAID 0, RAID 1 or RAID 5. You can see RAID in action by choosing the # on top of the diagram and clicking the diagram to start the animation. Enjoy!

Very helpful; also explains why (as Craig says) I do not need it. I think Time Machine with it's ongoing incremental backups of my whole machine on a 'remote' hard drive, and in addition backups to my second drive on my MacPro of Aperture vault should work well resulting in 2 backups of my photographs on separate drives.

Share this post


Link to post
Share on other sites

Sponsors

Advertisements



×
×
  • Create New...