A federal judge in Virginia has rejected President Donald Trump’s former campaign chairman’s move to throw out charges brought by the special counsel in the Russia investigation.

The decision Tuesday was a setback for Paul Manafort in his defense against numerous tax and bank fraud charges. Manafort had argued that special counsel Robert Mueller had exceeded his authority because the case was unrelated to Russian election interference.

U.S. District Judge T.S. Ellis III issued the ruling. Ellis had previously grilled Mueller’s team, questioning whether they brought the case to get Manafort to testify against Trump.

Prosecutors had argued that Mueller was within his authority, citing an August 2017 memo from Deputy Attorney General Rod Rosenstein. The memo shows Rosenstein authorized Mueller to investigate Manafort’s Ukrainian work and related financial crimes.

Manafort was ordered into custody earlier this month after a federal judge revoked his house arrest, citing newly filed obstruction of justice charges. The move by U.S. District Judge Amy Berman Jackson came after prosecutors accused Manafort and a longtime associate of witnesses tampering.

Manafort is the first Trump campaign official to be jailed as part of special counsel Robert Mueller's investigation. His attorneys have argued that Manafort didn't do anything wrong and accused prosecutors of conjuring a "sinister plot" out of "innocuous" contacts with witnesses.

Manafort will remain in jail while he awaits two trials in the next few months. 

Manafort allegedly used encrypted messaging apps and an intermediary to contact two former business associates - asking them to falsely testify.

He has denied any wrongdoing.

Trump previously said he hadn't spoken to Manafort in a while, distancing himself from his former campaign manager. 

"Well, I feel badly about a lot of them, because I think a lot of it is very unfair," Trump said. "I mean, I look at some of them where they go back 12 years. Like Manafort has nothing to do with our campaign. But I feel so -- I tell you, I feel a little badly about it. They went back 12 years to get things that he did 12 years ago?"

"You know, Paul Manafort worked for me for a very short period of time. He worked for Ronald Reagan. He worked for Bob Dole. He worked for John McCain, or his firm did. He worked for many other Republicans. He worked for me, what, for 49 days or something? A very short period of time."

(The Associated Press, CBS News contributed to this report.)