Your phone edits all your photos with AI - is it changing your view of reality?

From simple enhancements to hallucinated facial features, modern phones choose how our memories will look. You might love the results, but they could alter how we see the world.

Ever taken a picture of the Moon with your phone? You've probably been disappointed – unless you have a Samsung Galaxy device. These phones have a "100x Space Zoom" feature that captures the Moon with astonishing clarity for camera lenses the size of a fingernail. It beats what you'll get from an Apple iPhone, but there's a catch: Samsung's Moon photos are fake.

A Reddit user famously demonstrated this by holding a Samsung up to a deliberately blurry, pixelated image of the Moon on his computer. Happy to oblige, his phone snapped a nice clear picture, full of craters and shadows which didn't actually appear in the original photo. The company calls this a "detail enhancing function". The reality is that Samsung trained an AI to recognise the Moon and fill in details when the camera can't pick them up.

You won't find something that dramatic turned on by default in every phone. But no matter what device is in your pocket, every tap of the camera button triggers a series of algorithms and AI processing tools that run in the background. They can perform trillions of operations, all before the picture gets saved to your photo roll.

In general, it's all designed for beautiful and (mostly) faithful photography. But at the extremes, some phones include AI enhancements that drift far away from what you'd see with your own eyes. The next time you take a photo, ask yourself, is your camera documenting reality – or negotiating with it?

"It's called computational photography," says Ziv Attar, chief executive of Glass Imaging, who worked on the team that built the iPhone's Portrait Mode. Your phone goes far beyond collecting the light that hits your camera's sensors. It's guessing what the image would look like if the camera was better and then building it for you, he says.

"AI-based features are designed to enhance image quality while preserving authenticity," a Samsung spokesperson says. "Users retain full control with the ability to turn off AI functions based on their personal preferences."

But even with the headline-grabbing AI-powered editing features turned off, algorithms are still whirring away on the pictures you take. 

What happens when you take a picture

"When you click capture on a phone, you don't just capture one image, you actually take anything between four to 10 images usually, in normal lighting," Attar says. Your phone blends these images together for a picture that's supposed to be better than an individual snapshot. Some of those pictures can be duplicates. Others prioritise different parts of the image.

This and other baseline processes tweak the photo to fix problems the average person probably doesn't want to see. Noise reduction, for example, smooths random errors that show up as a grainy texture. Colour correction brings the image closer to what you'd see in real life. Then there's High Dynamic Range, or HDR, which melds several photos taken with more and less light in to preserve deep shadows and bright highlights in the same shot. Your phone also despises all things blurry and launches a multi-pronged assault to fight it.

iPhones, for example, employ a feature called Deep Fusion, using AI trained on millions of images. In addition to handling many of the techniques described above, these neural networks can identify objects in a picture and process them differently, changing individual pixels based on other pictures the AI has seen before. "It's very high-level segmentation," Attar says. 

The result is a photo that looks crisp and clear when conditions are good. But some critics and sharp-eyed amateurs argue that modern phones go too far, producing images that sometimes have a weird, plasticky feel or flat textures that look like watercolour paintings. Phones are working so hard to eliminate problems they can even introduce bizarre distortions that look like AI hallucinations if you zoom in on the fine details. Some people are so dissatisfied with the overly-polished photos of new phones that they're switching back to older models, or carrying around a second phone just to take pictures. 

Serenity Strull/ BBC

"At Apple, our focus has always been to help users capture real moments so that they can revisit memories as they experienced them," an Apple spokesperson says. "While we see AI as a tremendous opportunity, we also have great respect for the tradition of photography and believe it must be handled with care. We remain focused on providing our users a device that captures true and authentic photos that look stunning, while giving them tools to personalise them as they'd like."

There's a positive way to look at this. You could do most of this editing by hand if you were skilled and patient. Now, "instead of fiddling with all these different parameters, we have automation", says Lev Manovich, a professor of digital culture and media at the Graduate Center, City University of New York. "Certain capacities which before were only available to professionals are now available to amateurs."

But at the same time, your phone is often making creative or even artistic decisions about the memories you're capturing. Users may have no idea it's happening – and on some phones, AI is doing a lot more than tweaking parameters.

"I believe smartphone makers really want photographs to reproduce what people capture. They're not trying to create fake images," says Rafał Mantiuk, a professor of graphics and displays at the University of Cambridge in the UK. "But there's a lot of creative control in how you render an image. Every phone has a style, you know. Pixel phones have a style. Apple phones have a style. It's almost like different photographers."

'It's pure hallucination'

Of course, there's an implicit standard hiding in this debate: the idea that a "real" photo should look like it came from the film era. That comparison probably isn't fair. Every camera, since the very beginning, has always involved some baked-in processing decisions. It's easy to hear the word "AI" and assume it means something horrible. In many cases the algorithms are correcting for flaws that are inherent to the tiny lenses and sensors used in phone cameras. 

Some features, however, push the boundaries further. 

For example, phones made for Asian markets, particularly from Chinese brands, often have AI "beauty filters" on by default that automatically smooth or recolour skin and adjust facial features.

"It's pure hallucination," Attar says. "Asian phone models have these aggressive generative AI features on by default. So, they'll detect an eyebrow and literally draw hair on it if there isn't enough resolution to make it out, or they'll add eyeballs looking in random directions to people way in the background." 

Attar and others say this comes down to cultural norms and preferences. You won't find this on phones made by the American tech giants. iPhones don't offer built-in beauty filters, and Google switched them off by default in Pixel devices back in 2020, citing damaging mental health effects.

"Aesthetic refinement is not new," says Manovich. "Retouching photographs has been a major part of photography since the 1850s and it was somewhat similar to what people are doing today, basically beautifying the face, making skin more even. But things like automatically adding new detail, which was just not there in the scene, I feel like that's radically new… In some ways, it's still photography. In some ways, it's something else."

This isn't just a philosophical issue. Research suggests AI-edited photos and videos can plant false memories or change how we think feel our own bodies. 

Even American manufacturers have introduced features that represent remarkable trends in AI photography. Google Pixel phones, for example, have a feature called Best Take. We've all taken a group photo where someone closed their eyes or forgot to smile. (It's usually me, I never learned to pose.) With Google's Best Take, you take multiple pictures and then select the prettiest faces from different shots for one final image.

It may be a nice picture, but it's a photo of a moment that never happened. On the other hand, "that's probably how you want to remember this moment", says Mantiuk. After all, it's a group photo, he says, not evidence for a crime. 

"Authenticity has always been a core principle in how we develop Pixel Camera," a Google spokesperson says. "Ultimately, we're a team of passionate photographers. We focus on making possible what people – including us – have always wanted to do with mobile cameras."

How to see the 'raw' photo

You can deactivate HDR if you don't like it, disable beauty filters and turn off Samsung's Scene Optimizer setting if you want real (but less beautiful) Moon photos. But if you want the pure image coming straight off the camera sensors, completely untouched by all the different forms of AI, you have to take extra steps.

The Pro Mode setting on newer Samsung phones shoots completely unvarnished photos. Despite the name, the iPhone's ProRAW features (only available on Pro models) includes some AI processing. For truly raw iPhone photos, you need special settings in a third-party app. Some popular free options include VSCO Capture and Adobe Lightroom.

More like this

• The number one sign you're watching an AI video

• Is it finally safe to ditch your phone case?

• The 'blackhole' that erased your early 2000s photos

What you get won't be pretty in a traditional sense. It's not that anyone's trying to hide raw images, they're just not as useful for most normal people (though professional photographers use them). There will be a tonne of noise, colours may not look right, it will be softer and out of focus. But raw photos do have a "bad in a retro way" feeling, if that's your thing. 

Still, Manovich is among those who believe a little raw photography is worth your time. "Just to understand what your phone normally does and for the purpose of having a clearer consciousness of what exactly your photos are and what they represent," says Manovich.

-- 

For more technology news and insights, sign up to our Tech Decoded newsletter, while The Essential List delivers a handpicked selection of features and insights to your inbox twice a week.

For more science, technology, environment and health stories from the BBC, follow us on Facebook and Instagram.