USADA chief executive Travis Tygart wrote Holder on Jan. 14, urging him to join the civil case and telling Holder that "fraud and other crimes were committed" by Armstrong and other members of his former U.S. Postal Service teams.
It was not clear if Holder responded, but the Justice Department has not yet announced it if will join the lawsuit filed by former Armstrong teammate Floyd Landis.
Landis, who was stripped of the 2006 Tour de France title for doping, accuses Armstrong of committing fraud against the Postal Service, which sponsored most of Armstrong's teams as the star rider used PEDs to win the Tour a record seven times.
Landis also was a key witness in a USADA investigation last year that exposed Armstrong's doping. He stands to collect a portion—potentially millions of dollars—of any possible financial penalties against Armstrong.
The letter is dated the same day Armstrong admitted his drug use to Oprah Winfrey in a television interview that was broadcast on Jan. 17 and Jan. 18. USADA officials had been urging him to speak under oath with its investigators if he hoped to have his lifetime ban reduced.
The letter was first reported Thursday by Velonews.com and The Associated Press independently obtained a copy from a person familiar with the case.
Armstrong and USADA officials talked on and off over a couple of months about the terms of the cyclist sitting down for a long interview to spill all he knows about doping in cycling, but Armstrong said Wednesday he would not cooperate.
A person familiar with discussions between the two sides, who spoke Thursday on condition of anonymity because they were private, said among the topics was how much protection USADA could provide Armstrong in the whistle-blower case and against possible criminal action. The cyclist and his attorneys ultimately were not satisfied with USADA's offer, the person said.
In commenting Wednesday on Armstrong's refusal to talk, Tygart said that, "over the last few weeks he has led us to believe that he wanted to come in and assist USADA, but was worried of potential criminal and civil liability if he did so."
In his letter to Holder, Tygart wrote that USADA "uncovered one of the greatest frauds in the history of sport" but that his agency had reached the end of what it can do to punish Armstrong and other "non-sports" people involved with his teams.
"Fraud and other crimes were committed," Tygart wrote, telling Holder that the case involved drug trafficking, federal witness intimidation and that other federal agencies have gathered more information.
"USADA has already done the work in the sports case and won," Tygart wrote.
Armstrong was the subject of a two-year federal grand jury investigation that was dropped a year ago without an indictment.
Tygart also told Holder that the Justice Department joining the case against Armstrong would be viewed favorably by the public and the media.
The letter noted the federal government recently went after Landis for defrauding donors to his defense fund back when he still denied doping to win the 2006 Tour de France.
Tygart called the doping by Armstrong and the Postal Service teams a "massive economic fraud" that "absolutely dwarfs anything Landis did."