Men have a friendship problem. You probably know this already, if only anecdotally – walk into any pub in the land and count the number of blokes sitting there drinking alone. Social scientists know this evidentially. Recent research by the mental health charity Movember, for example, suggests that one in three men have no close friends. And I know this personally – in the summer of 2020, when I was planning to propose to my girlfriend, Naomi, I realised I had no one to call on to be my best man.

Loneliness doesn’t look like me. But there I was – 33 years young, outgoing, and always quick to buy my round – and yet I had no friends. And it made me feel ashamed. Suddenly I was that guy sitting alone in the school lunch hall. I was a Billy No-Mates. This bruising realisation sent me off on a quest, not only to fill a role, but to answer a question: what goes wrong for men like me? And what can we do about it? I discovered that there are three main theories.

“You don’t have friends to call your best man because of the culture in which you were raised. It doesn’t have anything to do with how you are naturally,” says Niobe Way, a psychologist at New York University who has spent her career studying the friendships of boys and men, and one of the many academics I spoke to who pointed the finger at so-called “toxic masculinity”. In her view, men struggle with friendships because they have been socialised into a “man box” of unhelpful gender norms that get in the way of intimacy.

But hang on, I’m not toxic, am I? In my mind, toxic masculinity referred to other men – monsters like Harvey Weinstein, sociopathic tech bros and backwater Tory MPs. Not little old me. “I wouldn’t say you’re toxic,” Naomi reassured. “But you become very different when you’re around guys. Do you know that?”

When I began to examine how I was with the men in my life, I realised that, yeah, maybe I was a bit weird, actually. I experience an irrational tightening of the body and heart at certain moments; a boa constrictor of inherited awkwardness. Whenever a man tries to hug me, I just stand there paralysed and inept like a dog being washed against its will, desperately wishing for it all to be over. The only time I ever tell a male friend that I even so much as even so much as like him is after seven pints or so. And whenever I give a compliment to a guy it’s always paired with a joke. It gives me distance from the emotion implied in it, introducing a degree of ambiguity. It raises the question, what do I really feel?

Ah, yes, feelings. Fredric Rabinowitz, a psychologist at the University of Redlands in California who is renowned for his work with men’s groups, says: “There isn’t any real education or encouragement for guys to develop a vocabulary of intimacy.” He recommended I get a therapist to change that. I swiftly learned I had developed a number of handy tactics to avoid any sort of deep talk with “the boys”. Banter loomed large – that peculiarly male way of relating, that jazz of casual brutality, that belligerent way of occupying space.

About four months into our time together, my therapist told me this hometruth: “You have an aura that you are not open or connected. There is a detachment. A block. You might be a laugh, but you have a vibe where other people aren’t going to share their innermost personal stuff. They get a feeling that you’re not able to give it back. So, perhaps, it’s no wonder you don’t have any close friends.” Sadly, she was right. Laughter had become the only emotional display I felt comfortable with. Banter was everything and everything was banter. The psychologists, it turned out, had a point.

Something was nagging at me, however. Male loneliness is not the contemporary problem we are often led to believe. The data suggests that men have been struggling with their friendships for a long time – as far back as the 70s and 80s, when researchers began to look at these things. If toxic masculinity is the cause of men’s struggles, then surely you’d expect men’s friendships to have improved. After all, it’s hard to argue that the sort of restrictive masculine norms the psychologists told me about haven’t softened since then. Does this not suggest that something else is going on? Something – whisper it quietly – in men’s biology?

Dr Robin Dunbar, leading evolutionary anthropologist and the godfather of friendship research, believes there is. He argues that men don’t become less likely to have intimate friendships; they are born that way. “What’s become very clear in the last decade,” Dunbar told me, “is the completely different way the social world of men and women works.”

READ RELATED: What is the healthiest way to walk? We ask an expert

The contrasting male and female social style is often characterised as “face-to-face” versus “side-by-side”. Women tend to socialise face-to-face with a strong preference for one-to-one interactions, based around talk and intense emotional disclosure. Men, however, tend to socialise side-by-side, preferring to hang out in groups, where intimacy is demonstrated by doing stuff together – playing five-a-side, going fishing, climbing mountains and so on. For men then, activities are the main course of the social feast.

Indeed, in the legendary male friendships of yore – Achilles and Patroclus, Butch Cassidy and the Sundance Kid, Maverick and Goose – it was stoic, shoulder-to-shoulder derring-do that was idolised as peak brotherhood. Not any more, however. Some social scientists argue that intimacy has been redefined in the modern world, to become understood as essentially a synonym for emotional disclosure. Thus, peculiarly “male” forms of closeness have become invisible to us – arguably making assumptions about intimacy men don’t share – or are even viewed as pathological.

Take banter, for example. Yes, men can be absolutely brutal to one another. Yet aggression is often employed not as the opposite of intimacy, but as a strategy to achieve it. While laughter bares teeth, this underestimates the complexity of what’s going on in that moment. It ignores the context – the sacred space of friendship, where there’s a tacit agreement that we don’t actually think or feel what we profess to think or feel. While there is a perpetrator and a victim, everyone is in on the joke. When that’s understood, mordant banter is actually a perverse form of love. It is, in a real sense, intimacy in action, communicating both “I know you” and “I know you trust that I’m not being cruel, that I have permission and that we are playing a game.”

Dunbar’s theory was interesting, because it implied that I had been looking at my best-man challenge from the wrong direction. Rather than focusing on improving the one-to-one friendships I had with the men in my life – as the psychologists had suggested – maybe I should be focusing instead on rebuilding the contexts where male friendships happen. On what we could do together. These habitats had been razed and not replaced in my grownup life. It turned out my best- man quest was actually a rewilding project.

But sharing activities – let alone organising them – takes a lot of time, and this is the third theory on why men have a friendship problem. Time is something that gets scarcer and scarcer as you exit your 20s – your job gets more serious, so does your relationship, maybe kids turn up – and your friends are the first thing pushed off the to-do list. Women are also faced with this same lack of time in middle age, of course. Yet while research shows that the social networks of both men and women wane as they age, men’s shrink much more. Why might that be?

It’s pretty simple, really: women put more effort into maintaining their friendships, while men are apt to let their social circle wilt and co-opt their partner’s instead. As the American standup John Mulaney has quipped: “Men don’t have friends. They have wives whose friends have husbands.” Men treat the women in their lives like their own personal HR department. If guys were honest, they’d introduce their better half at weddings with, “This is Claudia, my wife and director of people operations at Geoff Limited.”

The good news is that, unlike years of conditioning or the fate of our genes, effort is an easy solve. “My mates call me the Sherpa because I organise everything,” a friend of a friend told me one evening when I told him about my mission. “But if I didn’t do that, I’d never see them.” Be the Sherpa – that would become my new motto. A simple approach that helped me reignite my social life. And yes, I found a best man, too – thanks for asking.

“A man, Sir, should keep his friendship in constant repair”, Samuel Johnson once said. The owner’s manual isn’t as complicated as I first thought.

Billy No-Mates: How I Realised Men Have a Friendship Problem by Max Dickins is published by Canongate at £16.99. Buy it for £14.78 at guardianbookshop.com