The best way to understand ourselves?

While some stories shape political events, and others still resonate millennia after they were created, a few can offer a unique insight into human nature, writes Fiona Macdonald.

A man talking to a skull. A monologue about suicide, at the porous border between sanity and madness. A girl drowning in a brook, grief-stricken for her father. If David Lynch had made Twin Peaks 400 years ago, he might have come up with something similar. But this was William Shakespeare. In 1596, his son Hamnet died at the age of 11 – and five years later, as the playwright was finishing his tragedy Hamlet, Shakespeare’s father suffered a serious illness. He was to die in September 1601.

“Something must have been at work in Shakespeare, something powerful enough to call forth this linguistic explosion,” writes Harvard professor Stephen Greenblatt. “As audiences and readers have long instinctively understood, passionate grief, provoked by the death of a loved one, lies at the heart of Shakespeare’s tragedy.” The death of his son and the impending death of his father “could have caused a psychic disturbance that helps to explain the explosive power and inwardness of Hamlet”. What is revered as one of the greatest works in literature is likely to have sprung from a place of intense emotional suffering.

Alamy

The crucible in which Hamlet was forged might help explain why the tragedy continues to speak to audiences around the world four centuries on. It came eighth in BBC Culture’s Stories that Shaped the World poll, with voters praising its extraordinary insight into human nature. Hamlet is “the play that exemplifies Shakespeare’s profound understanding of the human psyche in so much of its nuanced extremity… our simultaneous blending of genius and self-sabotage, our capacity for love and hate, creativity and destruction,” claims the US poet, novelist and critic Elizabeth Rosner. According to the UK author and critic Adam Thorpe, it’s a story that has “influenced the way we think about our muddled selves. We enter Hamlet’s inner core and emerge rinsed of illusion.” Hamlet reveals how much stories can teach us about ourselves.

As the philosopher Noam Chomsky has said, “we will always learn more about human life and personality from novels than from scientific psychology” – something the critic and author David Lodge has explored. In his 2004 book Consciousness and the Novel, Lodge argues that “literature is a record of human consciousness, the richest and most comprehensive we have… The novel is arguably man’s most successful effort to describe the experience of individual human beings moving through space and time.”

Alamy

On one level, it’s the ability to spy on other people’s thoughts that gives literature this insight. “We do not really know what anybody else is actually thinking at any time – consciousness is a very private thing – and we partly go to literature in its various forms to make up or compensate for the necessary solipsism of our own inner lives,” Lodge tells BBC Culture. “Fundamentally, the reason why we read literary texts is that it gives the impression, if it’s successful, of enabling you to understand how other people think. We know what we feel and what we think, and what we hope for and fear, but we don’t really know how other people process these feelings and observations.”

As Greenblatt argues, in Hamlet, Shakespeare “had perfected the means to represent inwardness… coming in the wake of Hamnet’s death, it expressed Shakespeare’s deepest perception of existence, his understanding of what could be said and what should remain unspoken, his preference for things untidy, damaged, and unresolved over things neatly arranged, well made, and settled”.

Alamy

The critic Harold Bloom has gone so far as to claim that “Shakespeare will go on explaining us, in part because he invented us”. The playwright’s characters, Bloom argues in his 1998 book Shakespeare: The Invention of the Human, “are extraordinary instances not only of how meaning gets started, rather than repeated, but also of how new modes of consciousness come into being… no one, before or since Shakespeare, made so many separate selves”. He credits his own passion for books as stemming from the access they give him into the minds of others: “I am naive enough to read incessantly because I cannot, on my own, get to know enough people profoundly enough.”

Ways of seeing

That impulse can go beyond simply understanding who we are: reading can shape our sense of ourselves. “Whether it’s Dante when I turned 60 or Alice in Wonderland when I was an adolescent, these stories were for me autobiographical,” Alberto Manguel, writer and director of the National Library of Argentina, tells BBC Culture. “I understood perfectly what Alice felt in the world of absurd adults, where she tries as politely as she can to ask intelligent questions, when everything around her seems absurd. That helped me understand the mad world I was in – and later, when I discovered the political world and the Mad Hatter says there is no room at the table, and Alice points out that the table is set for many people and there is lots of room, I felt that was exactly what I was seeing in society, where a group of people like the Mad Hatter were saying to others who were starving that there’s no room at the table.”

Alamy

He insists that reading has given meaning to his life experiences. “I’m certain that if I hadn’t read Alice in Wonderland and Dante, I wouldn’t understand so many aspects of myself.” In his book Curiosity, Manguel claims he might not be able to identify himself in a police line-up: “I’m not sure whether this is because my features age too rapidly and too drastically or because my own self is less grounded in my memory than the printed words I’ve learned by heart.”

Identification with a story can come in unexpected ways. Preti Taneja’s debut novel We That Are Young reimagines King Lear in modern-day Delhi. Studying the play at school in the UK, she felt a profound connection – one that surprised her. “In King Lear I recognised the Indian extended family I was used to visiting in Delhi each summer,” she has written. “Shakespeare somehow recognised the part of my life my English friends had no idea about – the Indian part, in India.”

King Lear helped her to think about the Partition of India. “No one talked about it at school, yet there was this definite sense that there was a huge story that had brought us to this country, that was the reason I’d been born here – and then suddenly there it was, in Shakespeare of all things,” Taneja tells BBC Culture. “This story of partition that leads to civil war, this story of daughters who are forced into performing well for family honour and duty – which is the situation many second-generation immigrant women face.”

Alamy

By turning king into beggar, obedient daughters into villains, loyal daughter into banished exile, legitimate son into outcast and illegitimate son into insider, King Lear messes with ideas of fixed identities. “There’s an aspect in which almost every character is cast into a position of otherness to themselves,” says Taneja. “Everyone gets swapped around. This play really is about alienation from the self, and exploration of the other within the self – how we reconcile those two sides, and come to terms with the fact that we’re all hybrid beings, and that society isn’t a fixed thing.”

Bloom touched on this when he said that “Shakespeare will not make us better and will not make us worse, but he may allow us to overhear ourselves when we talk to ourselves… he may teach us how to accept change in ourselves as in others, and perhaps even the final form of change.” With characters who gauge what they seem like from others’ perspectives, and then adapt their behaviour accordingly, he believes the plays reveal the process of self-revision – the ability “to change by self-overhearing and then by the will to change”.

A light in the dark

But as well as identifying with characters, we read to find out how people profoundly different from us think. “The canon I was introduced to at school included Philip Larkin, JM Coetzee – all these men writing about masculinity and writing about society with a very particular gaze,” says Taneja. “It felt to me like what I was learning was how I was seen. This is what the male characters they are voicing think of me, and of the world of others that I belong to. There were lots of epiphanies in that – Coetzee is one of my favourite writers because his work is so sharp – it taught me about the way that a certain kind of patriarchal masculinity sees me in the world.”

Alamy

Storytelling has an evolutionary role in fostering empathy. As Atticus Finch said in To Kill a Mockingbird, “You never really understand a person until you consider things from his point of view – until you climb into his skin and walk around in it.” Reading encourages us not to reduce others to caricatures. “Unconsciously we begin to accept that the other is always a mystery and that easy characterisations lead nowhere,” says the Greek writer Amanda Michalopoulou. “Literature transforms amorphous fear and pity into individualities. It tells us: the other is not what it seems.”

Fiction can place us in uncomfortable positions too. At BBC Culture’s Stories that Shaped the World event, the author Colm Tóibín argued that it should “show you evil so that you would know it”. In the case of Clytemnestra in the Oresteia, he said: “I need to show you someone who was once good… how easily she could become corrupted, what a monster she could become – where you’re almost following her when she comes to murder her husband Agamemnon, thinking ‘I want you to do this’ – you’re pushing the reader’s imagination into areas where the reader might not want to go.”

And just as stories demonstrate that humans are neither straightforwardly ‘good’ nor ‘evil’, they also remind us how quickly we change. “Our readings are never absolutes: literature disallows dogmatic tendencies,” writes Manguel. “Instead, we shift allegiances… if we recognise ourselves in Cordelia today, we may call Goneril our sister tomorrow, and end up, in days to come, kindred spirits with Lear, a foolish, fond old man. This transmigration of souls is literature’s modest miracle.”

Perhaps most importantly, reading can reaffirm a feeling we all have – that who we are as humans varies from moment to moment. In response to the question posed by the Caterpillar – “Who are you?” – Alice says: “I-I hardly know, Sir, just at present. At least I know who I was when I got up this morning, but I think I must have changed several times since then.”

If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us onTwitter.

And if you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Culture, Capital and Travel, delivered to your inbox every Friday.