When the four-star general Stanley A McChrystal received an email asking if he might like to spend a month of his extremely busy life being trailed by a Kabul-based freelance journalist called Michael Hastings, he could have been forgiven for issuing a polite but firm reply to the effect that such a project would regrettably not dovetail with his already-packed schedule.

The general was, after all, in the middle of prosecuting the longest-running war in American history. "I was expecting actually no access or perhaps, you know, one or two days or a 45-minute interview," Hastings recalled this week. "Instead, the response was: 'Hey, why don't you come over to Paris when we're going on a Nato trip? You can join us in Paris next week, see the general, meet him, and then come to Kabul a few weeks after that to see him in the war zone.'"

It's impossible to know what exactly persuaded McChrystal's press staff to invite Hastings into their inner sanctum, where he would be privy to a frat-boy atmosphere and culture of contempt for the White House which would ultimately this week force the general to resign from his job as commander of US forces in Afghanistan.

But two words on the initial pitch appear to have tickled their fancy: Rolling Stone. "They wanted to reach a different demographic than they had during the other profiles," Hastings said this week.

In making that fateful calculation, the general's PR staff appear to have figured that they were setting up a journalistic hagiography which would see their boss lobbed softball questions by a star-struck pop journalist, before being showcased in the pages of a fashionable music magazine alongside such glitzy celebrities as Lady GaGa and Russell Brand.

They were quite wrong, of course. And their apparent surprise at the forensic tone of the article reveals a profound ignorance about the nature of Rolling Stone: it may look like a fluffy music magazine, but for more than 40 years it has also been a forum for serious, agenda-setting journalism in politics and popular culture.

The article Hastings produced is part of a tradition of "long-form" reportage which stretches back to the magazine's earliest days, in which writers are instructed to spend long periods with their subjects, building a close relationship in order to gain access and insights that normal journalists would be unable to obtain.

"Some people have expressed surprise that the McChrystal piece appeared in Rolling Stone," says Simon Dumenco, a media columnist for Advertising Age. "But I would say that actually, it could only have appeared in that magazine, which is one of the few publications that still has a commitment to long-form journalism. Reading the piece, you get the impression that its iconic stature in the pop-culture firmament is what gave Hastings access, and that McChrystal's staff were seduced into being indiscreet in front of him, because they grew to feel comfortable with him and thought he worked for a magazine which might make them look cool."

They certainly weren't the first people to make that misjudgement. Rolling Stone's most famous scribe was Hunter S Thompson, who joined the magazine shortly after it was founded, and who, in his devotion to "gonzo" reportage, would often spend weeks consuming drink and drugs with the very people he was supposed to be interviewing. Thompson made waves with famous articles like 'Fear and Loathing in Las Vegas'. Another early staffer, Tom Wolfe, built his reputation on accounts of the early US space programme which would eventually become his book The Right Stuff.

Rolling Stone was founded in San Francisco in November 1967, at the height of the counter-culture movement, by a university drop-out called Jann Wenner, who financed its earliest editions with the help of a $7,500 loan from his family.

"The spirit was that rock 'n' roll was not merely music, but an expression of a new generation," says Victor Navasky, chairman of the Columbia Journalism Review. "People presumed it was a music magazine, but it was far more than that."

In the early years, Wenner cultivated an extraordinarily talented team to achieve his vision in the publication, launching the careers of now-famous writers such as Joe Klein, PJ O'Rourke and Cameron Crowe, and a little-known photographer called Annie Leibovitz. His magazine quickly grew from being the in-house publication of the hippy movement to one of the most fashionable and prestigious titles in the US. The band Dr Hook and the Medicine Show released a hit satirical song poking fun at the desire of supposedly reclusive musicians such as Bob Dylan to bare their souls in order to appear on its cover.

Crucially, Wenner, who is now 64 and remains the title's editor, has managed to keep the title true to its founding principles. In an era when the attention spans of readers are said to be in terminal decline, Rolling Stone still boasts 1.4m readers, almost all subscribers who pay just $30 for two-year subscriptions, and is not saddled with the vast debts of other publications. The big question is of course whether owners of other, threatened highbrow titles will now learn from Rolling Stone's success.

Rolling Stone's greatest hits

Fear and Loathing in Las Vegas (1971)

Hunter S Thompson's most famous work appeared as a serialisation after it was turned down by Sports Illustrated. It defined the "decade of dope".

The CIA and the Media (1977)

After his Watergate exposé with Bob Woodward, Carl Bernstein was half of America's most fêted journalistic duo. His seminal piece detailed how the CIA used a network of 400 journalists to exert influence in the media.

The Bonfire of the Vanities (1984)

In desperate need of discipline for his novel, contributor Tom Wolfe agreed to publish it in fortnightly instalments. The publicity was a huge boon for Wolfe – to say nothing of the $200,000 that Wenner paid him for the rights.

The Make-Believe Maverick (2008)

Tim Dickinson's profile of John McCain before the last presidential election painted McCain as petulant and abusive, and detailed a particularly vile insult he was said to have aimed at his wife.

Great American Bubble Machine (2009)

Matt Taibbi's coruscating rant against Goldman Sachs became one of the definitive accounts of the financial crisis.