ISO – Don't hold your breath

Midweek mobile 15

Modern phone cameras get sharp and bright images with awful lenses and jokes of sensors. The most important aspect of the images is that they are usually viewed on the small screen of a phone. A quick search led me to an estimate that people take 4.7 billion photos every day. Be suspicious of such facile estimates. But it is clear that far less than a percent of a percent would be viewed on a large screen, where defects can show.

I stress tested my phone camera in exactly this way. My phone has a sensor with 4608 x 3456 pixels. I reduced it to 1667 x 1250 pixels for the leader photo. That looks good. But I asked what it I looked at it pixel for pixel: one pixel of the sensor for every pixel on the screen (1:1). I did that in the most detailed photo in the slideshow above. The next one compressed 4 pixels of the photo into one on the display (4:1), the next 16 pixels of the photo for one on the display (16:1), and the next (the featured photo) is shown 32 pixels per pixel of display (32:1). But for the post I compressed these views a little more; the closest is at 9:1, the rest are 36:1, 144:1 and 288:1. The result begins to show digital artifacts in the 9:1 view, although they are not overwhelming (at 1:1 they are unmistakable). Of course, I can’t predict what screen you’ll see them on, but if you have a choice looking at them on the biggest screen you have would be interesting.

On a whim I took a photo of a beetle and gave it the same treatment. Here you see the views in the ratios 9:1 (nine pixels of the photo to one of the display), with the successive frames showing 36:1, 144:1 and 288:1 compressions. It is only the last which looks sharp. On my phone the display is even smaller, so the image looks much sharper. But why this big difference between flora and fauna? I compared the exposure first. The flowers are taken with an equivalent exposure of 1/100 seconds and ISO of 100; the beetle with 1/50 seconds and ISO of 223. This means that the number of frames which are superimposed to give the final image is twice as many in the second. Slight hand movements could create the effect that you see, but the phone must compensate for that. But the ISO is also a factor; you can see more “grain” in the image of the beetle. I think another important factor must be the contrast between the object and background. That’s much smaller in the second photo. I’ll try to explore this further.

If you want a moral, I would say “Don’t look a gift horse in the mouth.” Your phone does not replace a good DSLR in image quality. Be happy with what it shows on its small display.

Phone photography changes our expectation of the interaction of camera hardware and image so dramatically that it is worth rethinking what photography means. I intend to explore this a bit in this series.

Midweek Mobile 13

More lessons can be learnt from the experiment I reported last week: push the performance of my phone camera to an extreme by doing very low-light photography. The camera spews out 64 Megapixel images (9248 x 6936 pixels for each photo). I took a segment which was 3300 pixels on the long side of a 4:3 aspect ratio and reduced it to a 1250 x 938 size for use in this blog. (All photos here use this aspect ratio without further comment, and I quote only the pixels on the long side.) That’s the featured image. We were looking for owls in a dark woodland using a flashlight on a new moon day, and the only lighting on the subject was its reflection from leaves. Not a bad photo given that, you can see the photographer, his shirt, the camera, and his hat. The amazing thing about the photo is its ISO of 17996! That’s the only way that the phone has of getting an image using a 1/10 s exposure with a lens that’s less than 5 mm across.

The photos that you see above comes from zooms to 830 pixel wide areas, subsequently reduced to 640 pixels across for use in the blog. The lighter image is taken from near the collar and arm of the shirt in the featured photo, and the darker shows the barrel of the camera. I’m not surprised by the lack of detail, the colour aberrations, and the enormous amount of digital noise in the photo. There was hardly any light at all to begin with. How did the camera actually manage to get anything useful with that incredible ISO?

Part of the answer is the Sony IMX471 CMOS sensor that’s used by my phone. The sensor has 4608 x 3456 pixels, with each pixel being 1 micron in size. Amazingly, this pixel size is about the minimum that you can achieve in visible light. The reason that the phone produced an image at all was due to the large number of sensor pixels that it could play with. The rest was the kind of statistical guesswork that is today called artificial intelligence or machine learning.

Midweek mobile 12

Push anything to extremes and flaws will begin to show up. Cell phone cameras now boast photos with around 100 million pixels, but taken with toy lenses a few millimeters across. The images that the phone gives you are the result of statistical computations based on fuzzy data. The enormous amount of computation needed for building an image (yes, images are now built, not captured) drains your phone’s battery faster than other uses would. How do you actually get to see the flaws of the optics? One way is to push the camera to extremes. Here I look at low-light photography, so that the camera’s ISO boost begins to amplify the problems that the image always has.

The featured photo used the 4.7 mm lens at the back of my phone, and used an exposure of 1/13 seconds (this means averaging over an enormous number of captures). The original image had 9248 pixels on the long side. When I compress it down to 1250 pixels for wordpress, the result is a crisp picture. Examine it at larger scales though, and flaws emerge. The detail shown in the above photo takes a segment which is 830 pixels on the long side and compresses it to 640 for wordpress. The camera chose an ISO of 15047, and there is quite a bit of noise in the detail. You can see lens flare below the arch. Above the arch you can see some of the railing blown out. The pixels are saturated and nothing you do can bring information out of them. Elsewhere, the railings are full of digital artifacts such as aliasing.

In the slideshow above you see an even more extreme case. This is a photo taken in a dark wood on a new moon night looking for owls using flashlights (yes, this was how I spent my diwali). The camera chose an ISO of 17996 and an exposure of 1/10 seconds. In the most contrasty bits of the photo you can easily see the noise in the image even without sliding into the detailed view. The lens flare in the detail looks cloudy; the AI has tried to erase it without success. It has performed surprisingly well in the face. I’m really impressed with the technique of computational super-resolution that it applies.

I close with a less extreme example from earlier in the evening. Here the software chose an ISO of 844 and an exposure of 1/25 seconds. Details are less grainy, as you can see when you zoom into the picture. The road signs are quite clear, if a little too dark to read easily, but the darker areas of the photo have clear digital artifacts, some of which you can see in the zoom. But you can see the liquor shop in its prize location at a crossroad blazing with light, open to its business of making the roads less safe to drive on.