WASHINGTON (AP) — The special counsel in the Russia investigation is accusing former Trump campaign chairman Paul Manafort of violating his plea agreement by repeatedly lying to federal investigators, an allegation that could expose him to a lengthier prison sentence — and potentially more criminal charges.

The torpedoing of Manafort’s plea deal, disclosed in a court filing Monday, also results in special counsel Robert Mueller’s team losing a cooperating witness from the top of Donald Trump’s presidential campaign who was present for several key episodes under investigation. That includes a Trump Tower meeting involving Donald Trump Jr. and a Russian lawyer he was told had derogatory information on Democrat Hillary Clinton.

The move signals a return to the acrimonious relationship Manafort has had with the special counsel’s office since his indictment last year. Before his plea agreement, Manafort aggressively challenged the special counsel’s legitimacy in court, went through a bitter trial and landed himself in jail after prosecutors discovered he had attempted to tamper with witnesses in his case.

In the latest filing, Mueller’s team said Manafort “committed federal crimes” by lying about “a variety of subject matters” even after he agreed to truthfully cooperate with the investigation. Prosecutors said they will detail the “nature of the defendant’s crimes and lies” in writing at a later date to the judge.

Through his attorneys, Manafort denied lying, saying he “believes he provided truthful information” during a series of sessions with Mueller’s investigators. He also disagreed he breached his plea agreement. Still, both sides now agree they can’t resolve the conflict, and U.S. District Judge Amy Berman Jackson should set a date to sentence him.

Manafort, who remains jailed, had been meeting with the special counsel’s office since he pleaded guilty in September to conspiracy against the United States and conspiracy to obstruct justice. He cut that deal to head off a second trial after being convicted last summer of eight felony counts related to millions of dollars he hid from the IRS in offshore accounts.

Both cases stemmed from his Ukrainian political work and undisclosed lobbying work he admitted to carrying out in the U.S. in violation of federal law.

As part of his plea agreement, Manafort pledged to “cooperate fully, truthfully, completely, and forthrightly” with the government “in any and all matters” prosecutors deemed necessary. That included his work on the Trump campaign as well as his Ukrainian political work, which remains under investigation by the U.S. attorney for the Southern District of New York.

Manafort also forfeited many of his rights as well as his ability to withdraw the plea if he broke any of the terms. In return, prosecutors agreed to not bring additional charges and to ask a judge for a reduction of his sentence if he provided “substantial assistance.”