‘I promise I’m not a sociopath,” I tell them. “But I will be asking a lot of questions that might seem deeply strange or outrageously rude. I’ll want to know what colour the walls were in the consultant’s office when you got the results, I’ll ask what the weather was on the day of the funeral, I’ll want to know about the hotel breakfast on your honeymoon. But it’s because we’re not telling a quick story on a chatshow, we’re writing a book.”

When I meet potential subjects with a view to ghostwriting their story, I always begin with “the talk”, so they understand I have no malevolent intentions. I’ve been doing the job for more than a decade now, and it bears little resemblance to most people’s assumptions.

Often subjects open up about their worst moments, greatest secrets, biggest fears and what they endured to overcome them. On a human level I feel compelled to bear witness to that, giving them the space to speak plainly about their experiences. Listening without judgment is the only way to get to the core of a story.

However, to describe this effectively, this does require a heady cocktail of pastoral care whisked through with interrogation that would look callous to an outsider. After all, it’s often those with the most interesting stories who don’t realise what is actually interesting about them. The first time I gave someone “the talk”, they seemed appalled, but that was because they were already unselfconscious about sharing intimate details.

Others give no detail at all. They’ll share a story of deep trauma, the worst moment in their life, and they’ll be bemused when I respond by asking if I can just double check… “So she was still in the wedding dress for this?” Like I say, rude. But I’m eternally fascinated, not just by the lives people have led, but the colours in which they’ve remembered those lives, and what they – rather than their fans – judge to be exceptional about them. I once watched an exceptionally beautiful face fall when in articulating her past addiction to me; she seemed to realise for the first time just how young she’d been.

Where does this leave the psyche of the ghostwriter? Therapists have extensive training at setting boundaries, but I head home on the bus with my head spinning at the things I’ve just heard and an obligation to follow up with a cheery text about deadlines. And deadlines are an issue – memoir in particular is often written within very tight time frames, leading to days on end spent with a subject, only to hear their voice in my head for further weeks as I transcribe tapes, eventually melting into writing and rewriting their life for months after that. Last summer my son recognised someone I was working with on TV and started talking to her, so familiar was he with her face on my Zoom screen.

It really can mean slowly becoming part of their life, their family, their story, all while trying to remain largely unseen. I feel protective of subjects when stories hit the press that I know they were nervous about. I bear random celebrity grudges because of anecdotes I was told which never made the final draft. I also feel deep loyalty to strangers in industries I will never work in. I have been phoned by subjects in tears in the back of cabs, at 1am on Boxing Day, and when they have just come round after a rhinoplasty.

And, most of the time, only a tiny handful of people ever know I had anything to do with the finished book. An ex once likened me to Harvey Keitel’s Winston Wolf character in Pulp Fiction: turn up, request a coffee, get the job done, leave. “How can you bear it?” is the most frequently asked question. “All that writing and no one even knows you did it?”

But to my surprise as much as theirs, it has turned out that this is the best bit. Without ghostwriting I would almost certainly have given up writing my “own” books by now, slowly losing my mind with loneliness, patrolling the house in pyjamas.

Highs have included genuine friendships made, and an insight into lives that I could never have imagined witnessing at such close quarters otherwise. A personal low was thinking I had a rat in my kitchen, only to discover it was one of my subject’s hair extensions. It made sense that a part of her would be lingering in my kitchen; she’d been lingering in my head all week.

READ RELATED: Portmeirion to Coleg Harlech: an architectural odyssey in north-west Wales

Nevertheless, barely a soul believes me when I try to explain how much pleasure I get from the work. Most people, I suspect, are eternally intrigued by purported secrecy. The fairytale of an undiscovered talent doing the hard graft under someone else’s name still tantalises: a Barton Fink-style secretary covertly entertaining the public while a drunk in the corner takes the credit.

For others, it is the prospect of access to the powerful. Politicians are often still packing for their departure from SW1A when the ink on their publishing contract is drying and the question of who writes those memoirs has long been a source of public fascination – one only compounded by Robert Harris’s political thriller, adapted for screen by Roman Polanski as The Ghost Writer. Well, that and David Cameron’s writing hut.

And the appetite for tales about the writer behind the writer does not seem to be waning: Kate Winslet is set to star in a TV adaptation for HBO of Hernan Diaz’s novel Trust, which focuses on the life of a Wall Street tycoon who hires his own ghostwriter after a thinly veiled version of him appears in a bestselling novel.

At the heart of all this curiosity is the idea that there might be one version of the truth and that it might be being kept from the reader. The title “ghostwriter” – bringing to mind a spectre moving things invisibly – does little to dispel this. The reality is more mundane: most memoirs I have worked on have involved no cover-ups, but have simply needed someone at the helm with a clear idea of storytelling. If you’ve got a book deal, you’ve got a story – you just might not be a writer.

On the whole, the authors who one might expect to be working with a ghostwriter generally are: it was no secret that Rebecca Farnsworth was the discreet but constant hand on the tiller for the glory years of the Katie Price empire. Until her death in 2014, Farnsworth wrote 14 books with Price, each bearing the reality star’s name. Being Jordan, her first autobiography, sold more than 1m copies in 2005 and led to a veritably Pepys-like series of memoirs about her day-to-day life, alongside novels including Crystal, which outsold the entire Booker prize list in 2007.

Usually the writing process really is collaborative. There is always an initial meeting, arranged by agents, which often feels more like a particularly intense blind date than anything. Both the potential subject and I are effectively figuring out whether we can bear to spend up to nine months having incredibly personal conversations with each other. Do they really want me at their kitchen island, drinking their coffee and asking them what losing felt like for hours, days, weeks on end?

And how do I feel about my subjects? Is their life interesting to me? I have only said an outright no to one project and that was because it was going to involve a lot of opinions on vaccines and vitamins that I found… challenging. At times friends have despaired of how happy I am to slip into someone else’s skin for months at a time, before slithering away again unnoticed. “You’re losing yourself in the lives of others!” they’d tell me, encouraging me to finally give fiction a go. But what is novel writing if not slipping into the lives of others for a few months, a combination of storyteller, nosy neighbour and catastrophically bad therapist?

They thought I was wasting my time, but it turns out that ghostwriting was time spent exceptionally well. During years of turbulence in my own life I sometimes felt trapped by my own story, and I was grateful to take refuge for months at a time in the lives of others. Yet, while I thought I was learning about writing, I came to see that I was also learning about living. As I wrote time and again about different sorts of pain, ways of coping, and perceptions of success, I was coming to understand more fully how much life there is to be lived and how many ways there are to do it.

So, at last, I chose to make the most of mine. After years as a ghost, I now feel fully alive: this year I have two books out. One is a novel with my name on the front. And the other has someone else’s.

Under The Same Stars by Alexandra Heminsley is available for £13.04 at guardian bookshop.com