31 replies to this topic

[Paul Kay]

Looking through the posts on wetpixel it soon becomes clear that the difficult subject of lens/port combination and image 'quality' repeats itself again and again. Given wetpixel's poster's experience and knowledge base I wonder if it would be possible for us to come up with a more standardised methodology for testing lens/port combinations which could be used to build up a database of recommended solutions and quality assessments? Unlike on land where MTF data is often available and testing is relatively easy, little data for underwater is available - other than that supplied by the manufacturers which does not always prove to be optimal. Add to this that underwater tests are time consuming and difficult to be precise over and it is clear that some defined standard (possibly using a downloadable 'target') is starting to be needed. Since people are testing and posting their observations it makes sense that such tests should be comparable.

Anyway to start such a discussion I would firstly recommend that test distances are established and I'm going to suggest that for wide-angle lens/port combinations, test distances should include 1m (as being 'the' prime test distance), perhaps 3m and 0.3m (or minimum focus distance) as measured by a physical method - ie tape measure - from the sensor plane. Targets should be some form of varying line test chart and should be used centrally, on the centre edge and in the corner.

Is this an idea which is of interest and if so has anyone got any suggestions especially regarding a suitable test chart or method of assessing field of view angle?

[photovan]

I'd second the idea. There was some discussion along these lines about 12 months ago but it seemed to fade. some standard test charts would be great, if we can come up with a design available as a download that would be good, so people can print their own or I'd be happy to print for something near cost.

Then a documented method explaining all the critical aspects that need to be covered ... image and subject planes parralell etc.

the more considered the input the better.

[Paul Kay]

OK. I'm a great believer in simplicity so I'll suggest that we look for a simple A4 test chart which can be downloaded, printed and encapsulated. Simplest method could be to use 3 x prints of test chart. I'd actually suggest shooting the test charts labeled up as follows; one in centre of image ©, 1 at centre left of image (ie on the edge)(CL) and one at bottom left hand corner of frame(BL); first test should be at 1m (1m) then at 0.3m(0.3m), then at 3m (3m). This should give a fairly good spread of info and would yield 9 x images (C1m, CL1m, BL1m, C0.3m, CL0.3m. BL0.3m, C3m, CL3m & BL3m). The test chart images could be posted as crops which might be at 1024 x 768 pixels as an initial 'finding' with hi res crops available upon request. Of course lighting and visibility have a role to play, but for simple 'lens/dome placement optimisation' such a test should show the best combinations. And also should indicate the expected 'resultant quality' (rather nebulous I know - say an indication of image before/after optimisation), of varying lens/dome/extender variations.

Now some form of chart needs suggesting.

[loftus]

Must have been my test of the Nikon 17-35 that got you thinking eh Paul.
I was in on the discussion with Darren a while back and of course I agree.
For myself I try to do the test with the chart you see in my test thread at the shallow end of my pool, about 1m shooting down onto the bottom of the pool. Leaves out the variable of having someone holding the chart, and easy to duplicate the distance.
My only comment on your above suggestion Paul is make it even simpler, and leave out the bottom left hand corner and possibly simplify the distances. Maybe just use 1m.
I think this would make it easier and less tedious for folks to do by themselves, and in a shorter time period.
Another suggestion I would have is that these tests are then posted to possibly a new section where they can be easily found.
The chart I use is a focus testing chart I downloaded, they can be helpful if one is trying to look at field curvature at the corners. They are good for studying under the loupe at 100% with lines numbers and text that one can look at.
http://regex.info/bl...ch/focus-chart/

Edited by loftus, 16 April 2010 - 03:50 AM.

[Paul Kay]

Must have been my test of the Nikon 17-35 that got you thinking eh Paul.

My only comment on your above suggestion Paul is make it even simpler, and leave out the bottom left hand corner and possibly simplify the distances. Maybe just use 1m.
I think this would make it easier and less tedious for folks to do by themselves, and in a shorter time period.
Another suggestion I would have is that these tests are then posted to possibly a new section where they can be easily found.
The chart I use is a focus testing chart I downloaded, they can be helpful if one is trying to look at field curvature at the corners.
http://regex.info/bl...ch/focus-chart/

Well it was this test and others!

I think that you mean leave out the (less important than corners) Centre Left test chart - which actually sounds fine and simpler - I'd also agree that as a simplest test then just 1m would be OK, but the 0.3m is quite important to people shooting CFWA so is probably relevant where possible, whilst the 3m is less so. Not sure about the chart you use - corners are tricky but to compare them a simpler line chart would be probably better as it may show degradation effects more effectively. I'll search for something that I think might work and link to illustrate if/when I find something.

[loftus]

Well it was this test and others!

I think that you mean leave out the (less important than corners) Centre Left test chart - which actually sounds fine and simpler - I'd also agree that as a simplest test then just 1m would be OK, but the 0.3m is quite important to people shooting CFWA so is probably relevant where possible, whilst the 3m is less so. Not sure about the chart you use - corners are tricky but to compare them a simpler line chart would be probably better as it may show degradation effects more effectively. I'll search for something that I think might work and link to illustrate if/when I find something.

Once you find a chart, maybe some type of composite chart can be made up. I would suggest keeping at least a piece of the chart I use as the markings which are numbered can be very useful if one wants to tilt the chart to demonstrate DOF. I see this as having potential value for evaluating differences between rectilinear lenses and fisheye lenses and curvature of the plane of focus behind a dome. Particularly important at close focus.
Setting up the test so that it can be easily done by one person is also important, none of my family want to get involved. That's why shooting down at the pool bottom at the shallow end works for me.
Once a chart is decided on, would be nice if it could be downloaded from WP maybe.
Another suggestion I would have is that the test images be kept on a WP server.

Edited by loftus, 16 April 2010 - 04:16 AM.

[Steve Williams]

When you guys figure something out let me know. I've been using this chart from Tim Jackson to check the lens before getting in the water with a housing. I have been using this chart for underwater tests. You can read the links about the specifics about the details here. I have no idea if it's the best way to approach getting the answers you're looking for Paul but it's been helpful to me. I've got to say I'm not too concerned with getting the last pixel of definition out of the system. I'm just trying to understand the best camera settings I'm capable of then go from there. I'm not going to necessarily shoot at the "optimum" setting. But it sure helps to know where it is. On the other side of the coin I'm pretty sure too much pixel peeping can affect your Vision (capital V intended)

The suggestion of using a common distance is intriguing, I've been using relative distances depending on the subjects and lenses I've been concerned about. For example when I was trying to see how to optimize the 17-40L for dolphins I was testing at around 2 m because that's how close I thought I could get. Now I'm looking at the Zen dome/ Tokina combination and I'm trying to understand the effects at distances more like 30 cm.

Jeff, I like taping my chart to the wall of the pool or using the test frame I rigged up so that I can do the tests on snorkel. I found it was much easier to vary the subject distance that way. I like the idea of standardizing the way we test these things. I'm trying to shoot so that I fill the frame with the test chart in a corner. It's key that we shoot with the detail right in the corner of the frame. If you miss and go a little wide the images just aren't comparable.

I also think Stephen Frinks method with the movable stand and sign works well. His method is much more condusive to looking at depth of field issues. I do think it's important to post a crop of the corner (if that's what we're comparing) so everyone can judge for themselves.

Cheers,
Steve

[jcclink]

If you can set up a test chart or something with a lot of fine print (especially in the corners) in a large tub of water (12-18" deep), you don't even have to get wet. Just need to have the dome u/w. If you get good results up close, then more distant shots will also be good. This method has worked well for me setting up a couple of my lens/dome combinations. This also makes it a lot quicker & easier to change setups for ports, ext rings, diopters, etc.

Edited by jcclink, 16 April 2010 - 04:41 PM.

[okuma]

I use a metal yard stick (meter stick) and set it on a diagonal angle and make sure it is in the frame corners. This gives you a flat target with 1/8 inch increments across the full shooting plane.

You can hang it off the side, by one end or lay it on the bottom.

Metal is better than wood as it will not warp after multiple uses and the markings are of sharper quality.

Works for both W/A and macro.

Edited by okuma, 16 April 2010 - 07:45 PM.

[Paul Kay]

I have been using this chart for underwater tests. Cheers, Steve

That's the one I had in mind. Initially I'd say that a flat field test would be most useful using this chart in centre and corner at 1m. This should provide a comparison between the potential 'best' (ie central) results and just how poor the corner is. Once this can be quantified then other tests might be useful but this is IMHO the most useful initial test.

Whilst on the topic of lens testing the following link might prove to be of interest http://www.l-camera-...ens-design.html.

[greedo5678]

This seems like a good plan, esp for someone like me about to invest in whole packages and needing to compare lenses.

I suggest using a test card like steves, and having seen his test with the zen 4" dome, it was nice to see the difference between the 10 and 17 ends of the tokina.

I would suggest that the card be taped on the pool wall and the camera moved until the frame is exactly full with the card, and shots taken at either end of the zoom spectrum at the distance card to camera (plane or tip of lens) measured and stated.

Of course then vary the F-stops too for comparison.

This would allow a new buyer, i.e. me, to compare say a 10-17tok fisheye vs a 10-22 rectilinear, being able to compare both warp, bend/fisheye effect, the minimum shooting distances, the distances of shots at either end of their scale and edge clarity on a set subject.

another big discussion at the moment is with the nauticam 7d, tok 10-17, 8.5" dome and the use of the 20mm extension and edge sharpness. Filling the frame with the card would allow precise and direct comparison between the 2 setups i.e with and without the extension. In this instance the card to camera distances would remain the same in both tests, because it would be the same lens, but would the extension and moving the dome away from the lens increase clarity?

With macro shots, it could be "fill the screen" and "minimum distance" ?

Olly

[PeteAtkinson]

I have been using a piece of garden mesh about 1 1/2m wide (that hard plastic kind with holes about 18mm) attached to a PVC pipe frame with cable ties. I tie something to the middle to aid focusing. I shoot so it fills the frame. For me, it's the comparison of the middle with the corners which is most informative.

Attached Images

• 

[H2Oplanet]

Love the idea...Might also consider expanding to include a protocol for testing strobes and conveying results consistent with pool/field test of others...???

Cheers,

Scott

[JohnLiddiard]

As well as sharpness etc. at standard distances, I would also be interested in absolute minimum distance of lens/port/housing combinations and also a practical real measurement of angle of coverage.

[whaleshark]

This is a great idea. How are you planning to laminate these charts and do they hold up underwater? Can you print them to about 1 meter square? I was thinking of using several plastic labels from bleach bottles or something like that. Still trying to find something that will work.

After my first experience with the Olympus E-330 and the UWA 7-14 mm f/4 lens behind the Ikelite 8" Modular Dome Port in Cozumel last week, I was dissapointed in how the combination worked at 7 mm. The lens is sharp corner to corner in air. UW the lack of sharpness was unacceptable all around the center. At 9 mm it was better and at 14 mm it was sharp all around. The guys at Ikelite had told me the combination was optimized for one focal length, but they did not say which. I need to test this UWA lens at various focal lengths and find the sharpest FL that it's optimized for.

I'll need a large test chart that fills the frame at 14 mm equivalent focal length. Any comments?

Dave

[Steve Williams]

Hi Dave,
I printed my chart using my Epson 1900 and normal photopaper. The biggest I can do is 13" x 19" which worked OK for me. I just printed a few up and took them down to the local office supply place and had them laminated. They did it while I was waiting for just a few dollars. Seems to work very well. No leaks yet and I play with them a few hours a week. You can see around the borders of the chart where the laminations are sealed and as long as you don't trim into this area it stays sealed. In other words you want to leave an inch or two of laminate border around the chart.

Steve

[whaleshark]

Thanks Steve,
I will check with the office supply store here and see how large they can laminate. I need a big focus target for the UWA lens. I can easily build the PVC frame you and several people have posted to attach them to.
Dave

Edited by whaleshark, 28 May 2010 - 07:49 AM.

[John Bantin]

From the many photographs I see submitted to the office of the mag I work for, I have deduced that a lot of underwater photgraphers don't care that the sharpness in the corners of their pictures are nothing more than mush.

[stewsmith]

From the many photographs I see submitted to the office of the mag I work for, I have deduced that a lot of underwater photgraphers don't care that the sharpness in the corners of their pictures are nothing more than mush.

They will be mine that you are referring to John

Sorry

Stew

[Steve Williams]

Thanks Steve,
I will check with the office supply store here and see how large they can laminate. I need a big focus target for the UWA lens. I can easily build the PVC frame you and several people have posted to attach them to.
Dave

Dave, If you print a large ref chart it may be as simple as taping it to the side of the pool. You might try just leaving enough extra material so you can tape it above the water line. Depends on the lens and distance your trying to test.

John, I can't decide if your last comment was the scurilous hack talking or the intrepid reviewer. Just curious, what percentage of the images that you get sent to you would you consider good enough to print?

Cheers,
Steve