Do you really want to know?

What’s over the hill? That’s a question that keeps us going, isn’t it? But sometimes what’s on this side of the hill is so beautiful that you don’t want to budge. Perpetual youth is the curse of never being curious about what lies over the hill. The rest of us, we love the view here, but we want to plow on and check out the view from the top as well.

Outside the small town of Ghoti on the Mumbai-Nashik road

Sometimes you get a glimpse of it from down at the bottom. Looks like someone’s made a good place for a selfie or two, a share on social media. This climb will be worthwhile, you think as you set off.

Naneghat, the view across the pass

At other times you reach the top, exhausted. To your dismay you find that it’s not the end of the road. There’s the steep downhill bit. It looks quite scary, and the path is wet. Do you really want to do it? Are the distant plains quite as nice as they look from up here?

Crossing Malshej ghat in Maharashtra

Sometimes you wish that someone had made a keyhole in that mountain, so that you can spy on the other side without needing to climb. It does happen, you know! These hills are full of tunnels.

Monsoon waterfall at the top of Malshej ghat

But sometimes,the other side just falls on you. There’s no way you want that. You roll up the windows quickly and get away from it fast, before all that falling stuff drowns you, or washes you down the hillside. Driving in the Sahyadris during the monsoon will give you all these new perspectives on aging and geology. What you make of these lessons is up to you.

Midweek Mobile 6

You can completely lose control of what you are doing when you take a photo with a mobile phone. Sure, there are all those presets which make you think you are in control: portrait mode, night photo, food photo, aspect ratio, quality, HDR. So many settings, so little control. Every setting instructs your resident AI to switch on some effect. That’s why it’s such a mass-market rage: you can get seriously good results with no effort at all. And that’s why serious photographers have still not taken to it. How to do you get the instrument to render your vision?

So here was one little experiment I did. I noticed that it computed the exposure (that’s a loaded word, see my previous field notes on exposure) through centered weighted averaging. The camera estimated the amount of illumination in the scene by taking an average over the frame with more weight given to the center. I took a photo first of the artfully aged wooden table in the dimly lit bar, with only my reading glasses and its case. The camera declared that it had used f/2.8 and an effective exposure of 1/17 second. Then, when my wine was delivered, I plonked it in the center of the same frame and took another photo. Now the AI decided to use a bigger lens from the cluster of lenses that phones are now studded with. It reported an aperture of f/1.7 with the same exposure as before. Notice how the light on the table changed?

I can’t carry red wine with me on every photo shoot (sigh!), but I can point the center of the field at different places in the frame to influence the quality of light. With 65 megapixels to play with, I can often sacrifice part of the frame to get the light that I want in the rest of the frame. A kludge, to be sure, but if you want a modicum of control in a mass-market device that gives you none, this is a workable hack.

Phone photography changes our expectation of the interaction of camera hardware and image so dramatically that it is worth rethinking what photography means. I intend to explore this a bit in this series.

Dialectic photography

Opposites? I decided I would try the dialectic instead. A thesis and its antithesis, brought together in a single photo: the synthesis. I started by focusing on the water which was dripping from the shade above the balcony. It was a couple of tries before I was completely satisfied with the focus. Inside the drops I saw a blur of light: the cloud-covered sky shining through the water! That was sharp focus and blur together in that droplet, just as it starts to break up.

A potted canna in the rain in one corner of the balcony gave me a synthesis between front and back, the two sides of leaves visible simultaneously. Also, sunlight through one leaf, the rain slicking the other. More than one pair of opposites in this photo. After days of limiting myself to using a cell phone, I was happy to sit on the balcony of the hotel with a proper camera, photographing nature in the rain. It is so hard to carry a normal camera outdoor in the monsoon.

A copperpod flower felled by rain on leaves below was a strange mixture of the barren and the fecund. The rain brings new growth, but it can destroy a flower before it fruits. Farmers in this region have had a little too much of this destruction in the last few years: crops destroyed by unseasonal rain, or by rain that refuses to come when it is expected. This year has been cruel. Half of the country flooded, the other half suffering from extreme dryness. Those in the know say they think the price of food could increase sharply during the coming months.

The reflections on these leaves was fascinating. I saw them moving as the trees above them shed large drops of water. The usual trick of photography stops time and motion. Here it has done just that, as it caught the parabolas of the splash arcing away from the point of impact, while the bulk of the water drop flows down the innermost part of the leaf. A modern mobile phone camera fails miserably in catching the fleeting, something that my actual camera does as a matter of course. But then isn’t this what you set out to do when you take up photography? Isn’t the act of taking up a camera a declaration that you will be contrary and try to freeze the flow of time?

Midweek mobile 5

Panoramas have had a cult following since before the early days of photography. Special cameras and odd format film plates were developed for this purpose even in the 19th century CE. I came to the field in the early days of commercial digital photography when enthusiasts would spend days stitching together individual exposures with the early versions of photo-editors. When I tried my hand at it I rapidly realized two important points. First, as you rotate the camera, the illumination changes, and you have to compensate for the different quality of light in the editor. Second, as you image the same object from slightly different angles, its shape on the film changes slightly, and you come across mismatched edges when you try to stitch the images together. You can understand this as a simple problem in perspective, but it was hard to compensate for it with the photo editor tools that were then available.

Now, a smart camera does all this “in the box” for you. On my drive in the Sahyadris, I stopped near a village and took a panorama of a wooded cliff above rice fields. All I had to do was stand firm, hold the phone in my hands and twist my torso smoothly. The demon in the box took care of the rest: the shading of light across the image, and the automatic adjustment of the distortions of objects due to changing angles. The second step, the one which was hard to do by hand, has a simple mathematical representation (three-point perspective) which the AI solves rapidly. The result seems to be good enough that you can wrap it around the walls of your home.

But my phone records the panoramas only in 40 megapixel images, not the 65 megapixels which it uses for other photos. This is due to the physical limitations of the small aperture and sensor which the phone camera uses. I’ve discussed earlier how multiple frames are read out from the CCD and averaged to produce a single image. The same thing is being done for panoramas. But since the camera is moving while the image is synthesized, the number of frames available for a single segment is limited. When you examine a regular image and a panorama at the same scale, you can see this clearly. In the image comparison above, both photos use the same number of pixels from the original image. I can zoom less when I use a panorama. This is the result of using a smaller number of frames for image averaging in the panorama, and also the restriction on computational super-resolution imposed when using the smaller number of frames. So really, you cannot wrap the panorama from a cheap phone camera around the walls of your home. At least not until the sensor and frame-readouts, or the processor and algorithms improve.

Phone photography changes our expectation of the interaction of camera hardware and image so dramatically that it is worth rethinking what photography means. I intend to explore this a bit in this series.

Midweek Mobile 4

Does AI have limits? You come to this question very quickly when you begin to think about phone cameras. They have tiny lenses which would once have limited both the amount of light and the resolution of photos. Once upon a time, the limited aperture would have meant long exposure times (and camera shake). It would also have created a resolution problem: that you could not get distant details with limited aperture (that’s bokeh for you). How does a phone camera wriggle out of this problem and produce photos as sharp as these two?

There are two parts to the answer. One is physics: the chemistry of silver halide crystals is replaced by the electronics of CCDs. The pixels can be made smaller, and there are clever constructions for noise reduction. As a result, you get closer to the ideal of “one photon, one pixel”, although not very close, at least on the cheap phone that I use. The other is mathematics: there is a lot of computation between what you see and what the CCD gives. First there is the subtraction of the background noise. Then there is the AI which begins to make statistical inferences about the image. Earlier I’d mentioned computational super-resolution: the improvement of lens resolution by making assumptions about the nature of the lighting. In both the photos above I looked at another technique that the AI uses: image averaging.

When I looked at this scene of the Sahyadris, there was a wall of dragonflies between me and the far cliffs. Each hovered in the air to find prey, then quickly darted to it. The light was not too bad, and on a normal camera, many would be blurred, but some would perhaps be sharp because they would be caught hovering. I wondered what the 65 megapixels of my phone camera would catch. Surprise! It caught nothing, although the EXIF told me that the exposure was 1/1912 seconds. Nothing at all, as you can see in this full size zoomed segment of the photo. I went over the photo segment by segment at full size. Nothing! What happened?

The phone camera took multiple readouts (frames) from the CCD sensor and then added them together to form the final image. This image averaging give noise reduction: pixels are averaged over frames and random noise is cancelled. But the random darting of the dragonflies also mimicked noise, and was removed. The exposure time written on the EXIF is probably a sum over the exposure times of the frames. The shorter reported exposure perhaps means that a smaller number of frames is averaged over.

Do I have an image that tells me that the camera is doing image averaging? Yes, the image comparator above tells me that. The “original image” (compressed for the purposes of this blog to 640×480 pixels) is shown on the left. The photo was taken from the car as it slowed for a speed bump. The EXIF data tells me that this was shot at f/1.7 with an exposure of 1/2422 seconds. In that time I estimate that the car must would have moved by a hair over 1/2 mm. The original looks sharp here, and looked even sharper on my phone. But the full size zoom shows strange artifacts. The lettering on the signboard is very blurred, as it would be if multiple images were added together. But the narrow leftmost pole supporting the roof of the shack is perfectly clear. Similarly, the edges of the sun umbrella are clean. This is clear evidence that the AI has selectively added parts of images. Even more than image averaging, there is clearly adaptive multiframe image averaging at work.

A 1450×1088 pixel section of two photos reduced to 640X480 pixels are shown here for comparison. The left from a phone camera, the right with a macro lens.

Now let’s get back to the photo of moss on a brick wall to see how much detail I could get from it. It was taken in full sunlight. At f/3.2 my macro lens required an exposure of 1/200 of a second to capture the moss in the comparison photo on the right. The phone camera lens has about 1/25 of the area, so if I’d masked my macro lens to keep only a phone camera sized area free, the exposure would have climbed to 1/8 of a second. But the phone camera reported f/1.7 (the lens is fixed), with an exposure of 1/264 seconds. Yet when I looked at the phone camera output at full size, I saw the blur on the left! Why?

First, keep in mind that the exposure time of the photo of moss implies averaging about 7 times as many frames as that of the cliff. You might expect so much averaging to remove blur. But I suspect that the blur in this second photo is an due to image averaging interacting with computational super-resolution: the improved lens resolution that AI gives. Since the small details in the zoomed view is almost entirely due to computation, little changes in the illumination can change the inferred image. Then averaging over the result can give the blurred details that you see. In the second zoom into the same photo you can see that the deep shadows look more noisy and equally blurred. This is also what you might expect from the averaging over super-resolved frames: CCD noise is removed, but inferred details are blurred by averaging over inferences.

Phone photography changes our expectation of the interaction of camera hardware and image so dramatically that it is worth rethinking what photography means. I intend to explore this a bit in this series.

Midweek mobile 3

Amazement is very natural when you begin to look carefully at photos which have 65 million pixels. The featured photo of a bushbrown butterfly is as good as anything I’ve posted on this blog. I didn’t take this with a zoom. I did not try to creep up on the butterfly. I stood at a distance and took a photo of a meadow with my phone. That’s the photo you can see on the left in the diptych below. You can see a couple of patches which have been converted to monochrome. I could crop the photo to yield two close ups.

The butterfly photo that is featured was cropped as you see in the monochrome block and then reproduced at about 90% of the size of the crop. The result is quite acceptable on a laptop screen, and seems quite nice on my phone. That’s what 65 million pixels, and 20 Megabytes of memory gets you! The ability to crop pretty drastically and still get an acceptable photo.

Let me digress on the word size. You could think of size by the number of pixels in the photo (or equivalently) by the file size of the image. When I say reduction to 90% I’m speaking of the number of pixels. But you could also think of it in terms of the amount of screen space it covers. That depends on the screen. A phone screen has between 720 and 1440 pixels across the width of the screen. A good desktop screen can have a higher pixel density. Since devices differ quite a bit, I will not talk of sizes on the screen. Your perception of a photo will depend very much on it, of course. A million pixels shown in a small area will give a sharper feel than when it is stretched out over a larger area.

Here’s the second image cropped from the same original. In the gallery above you’ll see that this is a cropped out of a smaller area of the original than the butterfly. After the crop I reduced the size to almost 25% of the original. There’s still enough detail to see the plant and its strange flower. I’ll write about the flower later, but for that blog I’ll use photos taken with a macro lens. The reason is that there is a limit to the crop which is essentially set by the limit of resolution of the lenses.

Here is a crop from the second original in the gallery. Again the area which has been cropped is indicated. When I took this photo I thought I hadn’t charged the macro camera I took on a walk with me. So I took this photo of the lovely blue flower with my phone. After I’d gone forward I realized that the other camera was actually charged. But by then I’d forgotten this flower, and I never saw it again. You can see that I got a pretty decent photo of this flower, but it isn’t quite sharp. I think I hadn’t bothered to tell the phone which point of the field of view the focus should be on. Phone cameras are not quite ready to replace macro lenses yet, but maybe in a year or two the situation will change.

Favourite things?

Raindrops on roses

The time has come,’ the Walrus said,
    To talk of many things:

from The Walrus and the Carpenter, by Lewis Carroll

Walks in the Sahyadris during the monsoon count high among my favourite things. This is perhaps the most difficult time of the year for climbers and trekkers, since the rocks are wet and slippery. But I am neither a climber nor a trekker. I walk with my camera and catch the seasonal burgeoning of flowers. Some, like the balsam in the photo (Impatiens balsamina), are common enough across the world, others flower only in special microclimates for a few weeks. It’s a different world, and one I’ve grown fond of visiting every year.

Whiskers on kittens

The jungles of the extreme northeast of India, the region caught between Bangladesh and Myanmar, is not one I’ve really explored. In a two week trip to Tripura many years back, I was lucky to find a clouded leopard (Neofelis nebulosa) in a hidden spot below us in a ravine. It woke from a nap, gave us a glance and went back to sleep.

Bright copper kettles

It took much planning to actually cross the border into Myanmar. Of the many things I enjoyed in that unfortunate country, one was the street food. Here is a photo of a street food stall in Yangon with people at lunch. Everyone has a large kettle full of tea on the table in front of them. I think it is refilled for free if you want. The tea habits are similar to those in China, you pay for the leaves, and get endless servings of hot water

Warm woolen mittens

Spring in Bhutan oscillates between warm and cool. In the courtyard of the storied temple of Kyichu Lhakhang in Bhutan a group of older women had gathered for a social prayer in the late morning. They gave us quizzical glances as we walked in. I was warm from a walk, but the women wore warms, and all of them had rosaries in their hands.

Brown paper packages tied up with strings

The sight of luggage being loaded on to aircrafts as I wait for my flight is perhaps my most favourite thing of all. The slight annoyance at the long time I will have to sit still in a chair, and the anticipation of what I might see as I step off the plane at the other end, are what drives this blog. And it all starts with the sight of baggage.

Midweek mobile 2

A mobile camera is not a good camera in ways that photographers were used to thinking of. The lens is a toy. Four centwp-admin/wp-admin/wp-admin/uries worth of lens technology have been junked by two related developments. The most important? That about 95% of the world looks at photos on tiny screens when distributing likes. So you don’t need the sharpness that photographers of old wanted; sell megapixels instead. That translates to about 10 Mbytes for the featured photo when my camera saves it. I know from experience that even on my large screen I can easily compress it down to about 200 kbytes and most people would not be able to tell the difference. That means I can retain only 2% of what is recorded. And on my phone I could easily throw away another 90% of the information (retain just 0.2% of the original) and no one would be able to tell. Then why so many megapixels? Because when you start from a large format photo and compress it down to a small screen, everything looks sharp.

You might remember that when you last changed your phone the picture quality changed a lot. Is that all due to more pixels? In a big part, yes. I dropped my old phone too often and was forced to change it quicker than I normally do. In three years the number of pixels in photo from a less-than-mid-range phone had gone up from around 10 million to about 65 million. Now look at the featured photo. The architectural details look sharp, considering that the subject is more than 300 meters away, and it was taken from a car that was making a sharp turn at a reasonable speed. But look at the near-full size blow-up in the photo above. You can see that at this zoom, details are pretty blurred. I have gained the clarity of the featured photo purely by not looking at it at full scale.

But that’s not the only change when you get a new phone. You also get a different AI translating the sensor output into an image. And this technology, which is a guess at what is being seen, is improving rapidly. As a result, the distortions of a bad lens can still be interpreted better, and result in a reasonable image. Note that this phone can remove digital noise much better than a five years-old phone would have done. The darker areas of the photo are much more clean (the detailed view above has been cropped out in the featured photo). Also, notice that the new generation AI deals with non-white faces better than before, getting an impressive image for the man walking towards the camera. This improvement is a response to accusations of biased training of AI.

But another detail is technically very impressive. Notice the level of detail? I can see very clearly that he is not wearing a mask. This resolution is better than a fundamental limitation which is imposed on lenses due to the wave nature of light (something called Rayleigh’s resolution limit). This computational super-resolution is a statistical trick which improves the image by making a guess about the nature of the ambient light. The down side of all this AI? This much of computation has a carbon cost. When I use my phone only for communication, the batteries last three and a half days. Street photography can drain the charge in a few hours.

Midweek Mobile 1

Night time is the worst time for photography if you have a tiny lens. Anticipating crowds and rain on the eve of Independence Day, I went out to get some street photos, but only took my mobile phone. It does a lot of computation to deduce the shapes and colours of what is recorded. With all that computation that goes on between the light hitting the sensor and an image being saved in memory, newer and faster computational hardware has an advantage.

But did these results actually improve over the physical limitations of the small lens? In one sense they did. If and when the sensor and imaging involve chemistry, a small lens exposes less of the chemical on the film. The result was that photos look dim. We are used to saying under-exposed for such photos. The only way to make the image brighter would then be to expose the photo for longer. But that creates a problem we call motion blur. With computation sandwiched between the sensor and image, there is a third way: the brightness can be amplified. I saw that The Family gets a much brighter image with her phone than I do, because her camera software is set to amplify more. So the problem of under exposure is replaced by that of digital noise: when you amplify, both signal and noise are usually amplified together. Motion blur can still be seen though, in the featured photo, for instance.

In another sense, the limitations of a small camera remain. A lens which is half a centimeter across cannot see details smaller than a couple of millimeters at a distance of ten meters. But this fundamental limit of resolution is reached only when the sensor collects light forever. With limited exposure the resolution drops by a factor of ten or hundred. So the image always has to balance motion blur against lens resolution. You can see this at work (at least on a large screen) in the photo above. The scene was well lit, the camera was not in motion, but the image is not awfully sharp. The computational hardware has prioritized freezing the movement of people by sacrificing the light needed for better resolution.

I suppose these photos look sharp and bright enough on phones and tablets to gather likes on instagram and tiktok. Perhaps you are in a minority if you view them on larger screens. As it turned out, it didn’t rain, so I could have taken a better camera with me. But technique is what you develop when you have limitations. A mobile phone is less obtrusive when you want to take street photos, so it is a good idea to start using it more widely for serious photography.

Hunder sunset

Hunder had one of the most relaxing hotels we’d ever been in. The Family said “Lucky we decided to stay two nights here.” Most people were passing through in one night. A few, old hands, had decided to stay longer. We resolved to come back later just to relax here. But in the meanwhile, there was a garden to relax in, and a sunset to watch.

After that, on the other side of the valley we waited for the moon to clear the line of mountains. Rather than jumping over the ridge, as I might have done, it seemed to creep up the slope. It was a futile attempt, because the clouds rolled in before it could top the ridge line. We decided to go for dinner. Later we saw that the moon was full. Good to have these ancient calendars; on holidays I lose track of days and dates.