Jump to content
Interceptor121

Side effects of Mirrorless on sensor phase detect autofocus

Recommended Posts

During the last months I have acquired my first camera with on sensor phase detect. This was mostly to keep up with birds in continuous autofocus and it has been pretty successful.

I then tried to consolidate my cameras into this one as I still shoot a dedicated camera for video that happens to be the one I have a housing for underwater use

Stressing the set up in various situation including backlit shots, sunsets and sunrise, trees, wide field astro photography I have noted down at least 3 side effects of OSPDAF

1. Lens internal reflections are aggravated and take a grid shape where the AF pixels are located

2. In certain situations you have PDAF striping where you can see stripes in the frame aligned with the AF pixels

3. In other cases of low light you also have banding

In essence the AF create patterns and increase fixed pattern noise this generates banding artefacts and more

This issue is common to different level of gravity among all cameras that use a Sony sensor and therefore includes Sony, Nikon and Olympus cameras. Other cameras that only have contrast detect and use same sensor like Panasonic are immune.

Looking at Nikon Z6/Z7 those are the worst offenders as apparently Nikon tried to fix the striping issue and doing that further deteriorates performance increasing the probability of banding

As of today it looks like Canon dual pixel AF implemented also in selected Sony model with different technology is exempt

What does it means for underwater use? Due to the issues with blue water and potential banding I envisage underwater photography will expose those weaknesses although those may take different shape like for example blotchy shadows in addition to patterning and banding.

Obviously traditional DSLR are exempt from this issue but this is a further challenge with mirrorless camera that until now I had not thought of

I would be curious to see some sample images with those cameras where those effects are apparent as obviously this does not always happen otherwise nobody would buy those cameras 

Share this post


Link to post
Share on other sites
23 minutes ago, PhotoJunkie said:

Could you show a few examples?

They are not underwater shots don't have a housing for the Olympus

There are many examples around just google Sony PDAF Striping or banding

 

SensorReflection-20201010.jpg

  • Thanks 1

Share this post


Link to post
Share on other sites

Thanks for the Pictures.  That really is distracting.

Share this post


Link to post
Share on other sites
Thanks for the Pictures.  That really is distracting.
Obviously that 'effect' only comes out at small apertures so in landscape photography you can avoid it
Underwater however you do shoot small apertures so I think sunburst may trigger that issue and others
But again I am just making hypotheses here
The banding issue is more subtle doesnt depend on aperture and is present at any ISO speed

Sent from my SM-A505FN using Tapatalk

Share this post


Link to post
Share on other sites

First time I heard of this issue. But the image shows it clearly. Maybe Panasonic got it right after all. Now their focus implementation just needs to work as well.

Share this post


Link to post
Share on other sites
First time I heard of this issue. But the image shows it clearly. Maybe Panasonic got it right after all. Now their focus implementation just needs to work as well.
Well I posted this issue in a photo science forum
One guy came forward who had developed an algorithm to correct AF AF pixel interpolation in post processing essentially an alternative method of image reconstruction
What is evident is that not only this is widespread but also that DSLR autofocus systems may produce even more flare issues (not banding tho)
All Nikon D750 D850 canon 6D may be affected you can test this quite easily shooting a light bulb
Obviously this guy is totally OCD but essentially says the only full frame camera immune by artefacts is the Sony A7R original version with no phase detect
Panasonic cameras are also immune as well as professional video cameras on contrast detect (none of Vatican arri red have phase detect)
I think while the reflection issue is limited to sinburst6it is obvious that most PDAF cameras have high level of black disuniformity and as such may generate banding or shadows artefacts sometimes confused with read noise
So when you hear someone saying my camera has noise already at ISO 800 and blotches in the shadows it is very possible that this is AF pixel generated fixed pattern noise
Obviously I could devise a test quite easily to demonstrate this however my camera has only contrast detect so those issues don't manifest at all
I can certainly say I do note issues in blacks in relatively straightforward land shots
This definitely put me off buying a housing for the olympus and as I had to go back to the GH5 for HDR sunrise and sunset shots. It's a shame as the Olympus has other great features

Sent from my SM-A505FN using Tapatalk

Share this post


Link to post
Share on other sites

Coincidentally, I took a very similar shot of the setting sun on Saturday with my EM-1 mk II.  These were test shots as I was setting up for sunset.  I can see no evidence of striping or banding in the first JPG or when pixel-peeping the RAW file, though the highlights are certainly blown which explains the flare on the black coastline at the left.  The bright specs that might suggest evidence are actually whitecaps, it was a windy evening.  The second JPG shows a very uniform black silhouette. 

I don't think I have ever seen evidence of striping or banding from points of light above ground or underwater, even when lifting shadows further than they should be.  I'd be curious to hear what specific combination of factors seem to aggravate this newly discovered issue. 

 

50607193878_aec3992e9f_o.jpg

 

50607215348_be97bfb75b_o.jpg

Share this post


Link to post
Share on other sites

Is it possible that some of the more extreme examples are lens-dependent?    Olympus cameras have a different UV cut-off on the sensor filter, which leads to issues like purple blobs from the Panasonic 7-14.  There's also an issue with banding from the P20 1.7, but I believe that has a separate root cause than the filter...

Share this post


Link to post
Share on other sites
2 hours ago, troporobo said:

Coincidentally, I took a very similar shot of the setting sun on Saturday with my EM-1 mk II.  These were test shots as I was setting up for sunset.  I can see no evidence of striping or banding in the first JPG or when pixel-peeping the RAW file, though the highlights are certainly blown which explains the flare on the black coastline at the left.  The bright specs that might suggest evidence are actually whitecaps, it was a windy evening.  The second JPG shows a very uniform black silhouette. 

I don't think I have ever seen evidence of striping or banding from points of light above ground or underwater, even when lifting shadows further than they should be.  I'd be curious to hear what specific combination of factors seem to aggravate this newly discovered issue. 

 

50607193878_aec3992e9f_o.jpg

 

50607215348_be97bfb75b_o.jpg

 

Normally you will see ghosting and lens flare more or less controlled. You also have ghosting which your image is an example of.

However if you close the aperture to say f/11 and smaller which you do with a sunburst you see some red dots appearing this issue does not occur at wide apertures.

Why does it happen? Is a combination of lens flare AND bouncing on the UV coating of the sensor that is made of highly reflective material. The AF pixels make this more apparent  however if you push really hard you may see some artefacts on contrast detection too. I have not yet been able to recreate the issue on my Panasonic cameras but I have seen it demonstrated on Olympus cameras with contrast detection. 

Note this issue applies also to Fuji, Sony APSC for what I know. I have not seen examples on full frame mirrorless however those have other issues

 

Share this post


Link to post
Share on other sites
1 hour ago, Wapiti said:

Is it possible that some of the more extreme examples are lens-dependent?    Olympus cameras have a different UV cut-off on the sensor filter, which leads to issues like purple blobs from the Panasonic 7-14.  There's also an issue with banding from the P20 1.7, but I believe that has a separate root cause than the filter...

The 3 issues reported here striping, banding and sensor flare are unrelated to purple fringing which is what you mention.

On that specific note this is not longer a specific Olympus Body/ Panasonic lens issue since the OM-D EM1MKII the Olympus and Panasonic both exhibit some purple fringing in certain conditions with any lens with the Olympus bodies producing a more pronounced effect

This includes not just purple fringe that can be attributed to UV filter, but also mild green fringing that currently goes unexplained. Generally speaking Olympus sensor coating makes it more prone to chromatic aberrations but this does not mean that the issue goes away on Olympus lenses and is also not true that Olympus lenses have UV coating to eliminate this issue they produce it anyway in certain conditions. For sure some specific lenses have the issue more than others this may be due to an underlying weakness of the lens that is exacerbated by the camera. 

Purple and green fringing can be corrected in RAW files so I have not specifically focussed on this issue as it can be resolved with some post processing effort relatively easily

Share this post


Link to post
Share on other sites
On 11/14/2020 at 8:53 PM, Interceptor121 said:

They are not underwater shots don't have a housing for the Olympus

There are many examples around just google Sony PDAF Striping or banding

 

SensorReflection-20201010.jpg

Hi Massimo,

Your picture looks interesting. I did never observe such a dot pattern in my images from EM1II. I believe your exposure conditions must have been special. Could you please specify the EXIF data, lens and how you postprocessed?

I count (roughly) 9*8= 72 of the dots, but the EM1II sensor has 121 PDAF pixels. How do these numbers correlate and how do the dots correlate with the location of PDAF pixels?

 

Wolfgang

Share this post


Link to post
Share on other sites

I'm unclear how phase detect in an SLR with a seperate phase detect sensor (i.e. not on-sensor) would produce the result?

The AF sensor data is separate from the imaging sensor...the only way it could happen would be if the processor is somehow combining the two sensors into the camera's output?

Even in Live View, the Nikon SLRs use only contrast detect.

Adam

 

 

Share this post


Link to post
Share on other sites
29 minutes ago, adamhanlon said:

I'm unclear how phase detect in an SLR with a seperate phase detect sensor (i.e. not on-sensor) would produce the result?

The AF sensor data is separate from the imaging sensor...the only way it could happen would be if the processor is somehow combining the two sensors into the camera's output?

Even in Live View, the Nikon SLRs use only contrast detect.

Adam

 

 

Does not produce banding or striping those occur because of near pixel interpolation for on sensor phase detect only so this is not generally related to DSLR perhaps I was not clear. So it is only issue number 3 reflections and not necessarily you see pixel shape.

I say generally because some DSLR like canon use PDAF on sensor as well not a separate circuit etc

In terms of flare we are talking about internal reflections well explained here but you can also google Nikon D750 flare or reflections. I guess it has to do with bouncing on the mirror. 

 

The DSLR part was not part of my investigation but when I said DSLR are immune this guy posted some examples saying there is even worse when it comes to reflections.

My approach is as long as I don't see them in the image I don't care but now I have been able to reproduce at least a few consistently during day shooting so it is getting annoying. On the other hand no mirrorless camera is exempt

More examples on mirrorless can be found with google PDAF striping or PDAF banding Nikon comes up a lot as they have attempted to correct striping and ended up with problem of banding

 

Edited by Interceptor121

Share this post


Link to post
Share on other sites
3 hours ago, Architeuthis said:

Hi Massimo,

Your picture looks interesting. I did never observe such a dot pattern in my images from EM1II. I believe your exposure conditions must have been special. Could you please specify the EXIF data, lens and how you postprocessed?

I count (roughly) 9*8= 72 of the dots, but the EM1II sensor has 121 PDAF pixels. How do these numbers correlate and how do the dots correlate with the location of PDAF pixels?

 

Wolfgang

Nothing special is f/13 the issue starts appearing from around f/8 wider you only see a bit of lens flare

You don't see all the AF pixels reflected in the lens depending on the incident angle where the lens flare came from 

Look closer at your sunburst f/9 posted on dpreview https://www.dpreview.com/forums/post/64485695?image=5

I can see few artefacts there. If you push that to f/11 and smaller the bubble may indeed appear

Edited by Interceptor121

Share this post


Link to post
Share on other sites

Could the patterning be something to do with the way the camera debayers during processing? These ones in the image above are definitely color artifacts, which would prompt me to think this is a color issue rather than a phase detect one?

Banding etc. can all be caused in the algorithm too

Share this post


Link to post
Share on other sites
11 minutes ago, adamhanlon said:

Could the patterning be something to do with the way the camera debayers during processing? These ones in the image above are definitely color artifacts, which would prompt me to think this is a color issue rather than a phase detect one?

Banding etc. can all be caused in the algorithm too

The issue in my image is number 3 sensor reflections (of AF pixels) this is not the same as banding or striping and occurs in a specific scenario with all cameras. PDAF pixels only aggravate it

For what concerns the banding and striping

This thread explains the specific of Nikon Z7 https://petapixel.com/2018/10/01/nikon-z7s-banding-makes-it-fall-short-of-d850s-dynamic-range-report/

My understanding is as follows: AF pixels do not produce an image so the image has to be interpolated from near pixels. This is done before the data is de-bayered and saved as RAW. The process of replacing those pixels can create artefacts that in certain conditions lead to striping or banding.

If you increase the number of AF pixels and there is more interpolation to be done so smaller sensor like MFT or APSC are more prone to problems. 

I have recently done a collaboration with Bill Claff the owner of the website Photons for Photos that has the largest collection of sensor data you can find on the internet. I submitted data for both my camera on the research for fixed pattern noise https://en.wikipedia.org/wiki/Fixed-pattern_noise#:~:text=Fixed-pattern noise (FPN),above the general background noise.

Fixed patterns means in short the chance you will see banding as it means the sensor is prone to create patterns in absence of light 

I have submitted data for my two cameras that share the same sensor with the only difference being PDAF.

The measurement is conducted at low and high gain that is different for each manufacturer and camera.

So to give an example at ISO 200 which is low gain for both cameras the camera with contrast detect scored a value of 6% DSNU which is a measure of non uniformity so low values are good

The camera with PDAF scored 22% which is a fairly high value.

At high gain the camera with contrast detect scored 10% and the one with PDAF scored 14% which is a lower value of the low gain band.

In the specific case the AF pixels are 'hot' compared to the neighbour pixels at low gain when the high gain circuit kicks in the AF pixels remain as they are but the neighbour are also hot so the effect reduces and so does fixed pattern noise.

So as a paradox the impact is higher at low ISO and as this specific camera has high gain at 2000 it means really for all normal situations you have higher risk than at high gain. This is also confirmed by my experience I get sometimes pretty decent shots at ISO 4000 while I may have low quality shadows at ISO 400.

When you look at Nikon D850 vs Z7 the situation is almost similar at low gain the Z7 has 15% DSNU vs 8% of the D850 when they are both at high gain that kicks in pretty early at 400 the DSNU is the same. So this confirms what you read on petapixel this banding actually happens at low ISO which for most people is completely counterintuitive.

Nikon in addition has implemented a specific technique to minimise the striping that can actually create banding itself so if you look at Sony A7R series for example A7R3 you see that fixed pattern noise is lower an incredible 3% DSNU at low gain and dropping to 16% at high gain but here high gain is 640. From what I can see also canon dual pixel is effected as fixed pattern is noticeably worse in the EOS R than it was with traditional DSLR.

To what extent this is a problem is always the question and my view at current state of data is as follows:

1. Smaller sensor MFT/APSC with large number of AF pixels have pretty bad performance (Olympus OM-D EM1, Canon 90D, Fujifilm XT-3) in terms of fixed patter noise due to the density of AF pixels compared to the area. However the effect is stable or even reduced at high ISO

2. Larger sensors are also affected (Nikon Z, Canon R) to what extent it depends on the technology and interpolation but the trend is stable or getting worse at high ISO 

3. Contrast detect sensors or traditional DLSR without AF pixels are generally better especially when high dynamic range is required at low ISO

The issue has not been investigated as a trend at this stage and it needs significant effort and data however I can quite clearly see the trend. When I submitted this problem to Bill Claff he wrote me 

The PDAF pixels are different and certainly can cause more noise particularly Fixed Pattern Noise (FPN). I think there's a limit to how well any firmware can "fix" the effect.

I am not particularly interested in investigating the global effect of this issue however I do see a potential problem with underwater photography and banding as this is a problem that does occur at times in water. I also see an issue with dark shadow banding as that occurs a lot with flash photography. Historically banding has been linked to lack of bit depth however it seems much more interesting to look at fixed pattern noise as you can get banding also at high bit depth. When you don't get banding you get other effects generically described as 'noise in the shadows' but what really means is noise is not fine grain but clusters of patterns that look blotchy

I would be interested looking at low ISO shots from any of the candidate cameras to see to what extent this happens. I do not have any of those cameras nor am planning to buy them but that is unrelated to this issue.

 

Share this post


Link to post
Share on other sites
Quote

I would be interested looking at low ISO shots from any of the candidate cameras to see to what extent this happens. I do not have any of those cameras nor am planning to buy them but that is unrelated to this issue.

I clearly don't understand everything in this topic but own an A7III if I can help?

Edited by waterpixel

Share this post


Link to post
Share on other sites

Sorry I have been  off this thread

This is the detailed information on Olympus PDAF @Architeuthis may want to look at this

https://www.dpreview.com/forums/thread/4108353

Bill says that the effect only shot in stacking images low light like astrophotography however there are other situations as explained and in some cases you get other artifacts striping and banding

there is no solution all PDAF methods have this problem one way or another

Share this post


Link to post
Share on other sites

Sponsors

Advertisements



×
×
  • Create New...