Pupils creating AI images to 'bully or ridicule'

Deepfake and indecent images made by children using AI have "risen sharply" on the Isle of Man, police have warned.

The Isle of Man Constabulary said images - some potentially amounting to a sexual offence - were being generated by schoolchildren to ridicule, bully or take revenge on their peers.

Schools Education Officer PC Louise Kennaugh said parents, as well as children, should be reminded of the dangers of AI misuse, adding that parents should be "friendly but intrusive" regarding their children's online habits.

She said the Sexual Offences and Obscene Publications Act 2021 "makes it really clear that people can't be using social media and AI to create images without consequences".

Anyone convicted of making or sharing indecent AI images can be jailed for up to three years.

The Act also updated laws surrounding revenge porn and upskirting offences.

Suspects and defendants in sexual offence cases are granted anonymity prior to any conviction.

"Before [the Act] we didn't have good, strong legislation regarding deepfakes and indecent images made using AI - we couldn't put the message across," said Kennaugh.

She said her team was seeing examples of pupils taking photographs of their peers and then using AI to superimpose another image next to it or make the subjects in the image falsely appear to be doing something.

"They're using it in a way which is cruel and unkind to bully or actually make images which are actually falling foul of the law," said Kennaugh.

"It's become a form of revenge, punishment or threat to share these images."

Kennaugh acknowledged that while AI can be a useful tool useful for everyone, including children, using technology to make or share incedent images or deepfakes is "incredibly harmful".

Her team has worked alongside schools on the island to teach students about the dangers of AI misuse.

The force also urged parents to take action.

A spokesperson said: "Talking openly with young people about responsible use of AI can help prevent mistakes that have real-world consequences.

"It's important to remind [children] that once something is shared online, it can be impossible to take back and images can spread quickly and be saved by others."

Read more stories from the Isle of Man on the BBC, watch BBC North West Tonight on BBC iPlayer and follow BBC Isle of Man on Facebook and X.