Anointed “our pre-eminent lefty” among contemporary American novelists, EL Doctorow, who has died aged 84, was praised as the “epic poet” of the forgotten American left. It was praise that he did not welcome. He proved elusive when dealing with the pigeonholes crafted by reviewers, and not a few readers. In a career spanning five decades, Doctorow feared that tidy labels were a distraction. He lived contentedly within the paradoxes of his career.

He did not want to be called a political novelist. “My premise is that the language of politics can’t accommodate the complexity of fiction, which as a mode of thought is intuitive, metaphysical, mythic.” Although he wrote lovingly of the lost world of the Jewish Bronx in the 1930s, where he grew up, he rejected the idea that he was an autobiographical writer. “Every book is an act of composition,” he remarked in 1989, “and if you happen to use memories or materials from your own mind, they are like any other resource; they have to be composed. And the act of composition has no regard where the material comes from. So when it’s all done it’s all autobiographical and none of it is.”

Doctorow wrote a handful of the most influential historical novels of the past half-century, but was determined not to be known simply as a historical novelist. Praised for having “done his homework” on the American Civil war for The March(2005), he claimed that he did little research, freely inventing when the historical record seemed somehow incomplete. There is a moving letter in The March sent by the Union generalissimo William Tecumseh Sherman to a Confederate general whose son was killed in battle. But no such letter was ever written.

The titillating scene in Ragtime, his international bestseller of 1975, between the anarchist Emma Goldman and Evelyn Nesbit, Stanford White’s squeeze, was wildly out of character for Goldman, as her two-volume autobiography Living My Life made abundantly clear. Doctorow claimed that his portrait of the financier JP Morgan in Ragtime was largely based upon deep study of an Edward Steichen photograph of the formidable money man. (He may not have been fully serious on this point.) Doctorow was attracted by the idea of improving upon the historical record.

Gore Vidal received heavyweight attacks from the leading Abraham Lincoln scholars when he published Lincoln in 1984. Long lists were compiled of his scholarly misapprehensions. Vidal accepted no corrections from the professors. But at heart he sided with the professional historians against writers such as Doctorow, doubting the wisdom of playing fast and loose with the facts. “It is hardly wise, in what looks to be a factual account, to have Harry Houdini chat with Walt Whitman aboard the Titanic, or whatever. Fantasy, as such, must be clearly labelled, even for our few remaining voluntary readers.”

Austere scholars similarly deplored Simon Schama’s use of novelistic devices in Dead Certainties (Unwarranted Speculations), his playful 1991 exploration of the possibilities of “historical novellas”. In 1994 Schama wrote a notably sympathetic review of Doctorow’s The Waterworks. Doctorow felt that history was, in the end, literature. And that it was literature, “intuitive, metaphysical, mythic”, which takes us to the heart of the social reality of American life.

Doctorow grew up in New York city, the grandson of Jewish immigrants who arrived from present-day Belarus in the 1880s. His grandfather, a printer by trade, was a passionate reader of Tolstoy. Everyone in the family read, but they were poor, and books mainly came from the library. Doctorow told in many interviews the story that he was named after Edgar Allan Poe, supposedly his father’s favourite writer. Living in a flat on Eastburn Avenue in the Bronx, north of Manhattan, and with a well-to-do uncle and aunt in the suburb of Pelham Manor in Westchester County, he had a glimpse of a benevolent, calm life, contrasting sharply with the family life of the Doctorows in the Depression, which seemed both quarrelsome and stressful.

Edgar attended the elite Bronx High School of Science, but much preferred reading Kafka to his assigned lab work. He published a short story in the school magazine, an enclave for the literary-minded. As a junior, he enrolled in a journalism class. The class’s first assignment was to conduct an interview. Doctorow turned in a gem, an interview with a German-Jewish refugee who worked as the stage doorman at Carnegie Hall. The teacher was impressed, thought it a terrific piece, and wanted to use a photograph of the doorman to accompany the interview when it was published. Running out of excuses why the doorman could not be photographed, Doctorow finally admitted he had made the whole piece up. A note to his parents and a trip to the principal’s office soon followed. He was unrepentant. An imagined interview was so much more rewarding, more transgressive.His father, David, owned a music shop in midtown Manhattan, only for it to go broke in 1940. Love of music was the glue that held the family together. An older brother played jazz piano. Edgar was allowed free access to his father’s extensive collection of 78 rpm albums. His mother, Rose, played Chopin with passion.

In Reporting the Universe, lectures delivered at Harvard in 2003, Doctorow reflected upon lessons learned. “I believe nothing of any beauty or truth comes of a piece of writing without the author’s thinking he has sinned against something – propriety, custom, faith, privacy, tradition, political orthodoxy, historical fact, literary convention, or indeed, all the prevailing standards together.”

He did his first degree at Kenyon College in Gambier, Ohio, where he studied with the poet and critic John Crowe Ransom, before switching his major to philosophy. In a production of King Lear, he played Edgar. After graduating in 1952, he studied English drama at Columbia University, New York, where he met Helen Setzer, whom he married in 1954. Before he could complete his dissertation at Columbia he was drafted, and spent two years in Europe as a corporal in the US Army signal corps.

Back in New York in 1959, he took a job with Columbia Pictures in their New York office as a script-reader, looking for something with potential. He urged Columbia to pick up their option on Saul Bellow’s Henderson the Rain King, which he read in typescript. The studio decided not to proceed with it. (When he read the published text several months later, he rather regretted that Bellow had “flattened the life out of it”.) The quality of the novels he read, mostly westerns, seemed notably awful.

For virtually the whole of the 60s, Doctorow worked in publishing, initially as an editor at New American Library, and from 1964 as editor-in-chief at the Dial Press. Publishing figures such asNorman Mailer and James Baldwin had its moments, but, he told Adam Begley in the New York Observer, “It turned out to be very useful to see how many really bad books were being published. It was very encouraging.”Welcome to Hard Times was filmed by Burt Kennedy in 1967, with Henry Fonda in a lead role. Doctorow regarded it as the second worst film ever made. It was followed by a sci-fi novel, Big As Life (1966), which Doctorow regarded with deep embarrassment. He subsequently refused to allow it to be republished.

While continuing his day job at the Dial Press, he wrote 150 pages of a straight chronological narrative in the third-person about the case of Julius and Ethel Rosenberg, the great cause célèbre of the American left. They were executed in 1953: their trial, the impassioned campaign against the treason verdict, and the subsequent wrangling over what, actually, the Rosenbergs had done, came through in Doctorow’s narrative. But, as a novel, it was dead. He wanted to throw the manuscript on a fire. Out of frustration he began to tell the story through the eyes of the Rosenberg’s son, Daniel. It was Daniel’s voice, his New Left anger, which brought the book to life. “Not I, but Daniel would write the book.” The Book of Daniel appeared in 1971.

“One of the things I had to learn as a writer was to trust the act of writing. To put myself in the position of writing to find out what I was writing.” The use of Daniel’s voice changed the book, and gave it a new meaning. He was no longer writing about the Rosenbergs (and did not, in any event, draw any conclusions about whether or not they were traitors), but exploited the narrative possibilities of shifting freely between the early 50s, as the net began to close around the Rosenbergs, and the late 60s, with its radically altered voice and sensibility. It was not possible to treat the book as a fictional plea on behalf of the wronged and persecuted Rosenbergs, nor did it take sides.

Some readers were puzzled by where Doctorow stood on the greatest of the Cold War treason trials. It was a test-run for Doctorow, whether it was possible to write a novel that was at once a political novel, a historical novel and a something which used all of the narrative possibilities created by being able to shift voices and times. The Book of Daniel was a powerful and brilliant novel. Giving up the day job seemed a real possibility.

Four years later, with the publication of Ragtime in 1975, Doctorow became one of the great international superstars of literary fiction. He received the National Book Critics Circle Award for it, topping the Publisher’s Weekly annual fiction bestseller chart with hardback sales over 250,000.

Ragtime broke with the literary convention of a consistent and identifiable narrative voice. There is no quoted dialogue, merely an author’s summary which floats confidently between characters. He assumed that readers would grasp who JP Morgan was, and the ensemble of historical characters, led by Goldman, Nesbit, Henry Ford, Booker T Washington and Houdini, flattered his readers. Doctorow did more research for Ragtime than he was prepared to let on, but claimed that he had made them all up, as every character in the novel was similarly made up. But the figure of Coalhouse Walker Jr, searching for justice in a society where a black man could not find justice, was a creation of a stronger and more moving kind.

Ragtime was filmed in 1981 by Miloš Forman, who promptly dropped many of the plot lines woven into the novel, so he could concentrate upon the figure of Coalhouse Walker, played by Howard E Rollins Jr. The author was very disappointed at the loss of so much of the political content of the book.

Doctorow did the screenplay for Daniel, directed by Sidney Lumet in 1983, but the novel, and his screenplay, were lost along the way. It was neither a commercial or a critical success. Doctorow’s opinion about it is unprintable.

At four or five yearly intervals, Doctorow published Loon Lake (1980), World’s Fair (1985), Billy Bathgate (1989) and The Waterworks (1994). Short story collections and essays followed along the way, as did the awards: the National Book award for World’s Fair in 1986, the National Book Critics Circle award and the PEN/Faulkner award for Billy Bathgate in 1989. In 1998 he received the National Humanities Medal from the National Endowment for the Humanities.

In novel after novel he explored the uses of complex narrative devices. City of God, his first novel set in contemporary New York, mixed jazz-like improvisations on song lyrics, biographical sketches, monologues, notebook entries, and stories which, as it turned out, were fabrications – but interesting fabrications, which paradoxically struck many readers as the most engaging part of a novel of heavyweight ideas which Doctorow described as “a big kitchen sink of a book, with a lot of surprises and riffs and all sorts of things”.

It was followed by The March, which received a second National Book Critics Circle Award in 2005, and a second PEN/Faulkner award the following year.Homer & Langley followed in 2009. Andrew’s Brain, with its sharply honed satire of Bush, Cheney and Rumsfeld, appeared in 2014.

Doctorow was inducted into the New York Writers Hall of Fame in 2012, and in 2013 received the Medal for Distinguished Contribution to American Letters from the National Book Foundation, and the Library of Congress Prize for AmericanFiction in 2014.

He held the Loretta and Lewis Glucksman chair of English and American Letters at New York University, and donated his papers to the Fales Library at the university.

Of the writers of his generation, Doctorow reshaped the boundaries of his craft, and with great coherent purpose.

He is survived by Helen, and by three children, Jenny, Caroline and Richard.

• Edgar Lawrence Doctorow, novelist, born 6 January 1931; died 21 July 2015