Serial sex offender Stewart Murray Wilson has denied breaching his release conditions at a court appearance on Friday.
The Department of Corrections successfully applied to the Parole Board on Thursday for an interim recall following allegations that Wilson phoned someone he had been directed not to contact.
The 66-year-old served 16 years in jail for multiple rapes of women, assault, drugging, ill-treating children and bestiality, which he denies.
At the Whanganui District Court on Friday morning, Wilson pleaded not guilty to a charge of failing to comply with conditions of release.
He has been sent back to Wanganui Prison and will next appear in court on 13 May for a defended hearing.
The Parole Board will consider whether or not a final order is made to keep Wilson in jail at a hearing before the end of March.
Wilson had been living in a self-care unit on the grounds of the prison after being released on strict conditions in August last year.
His lawyer, Andrew McKenzie, said the Corrections Department has failed to reintegrate his client into the community and Wilson may have made the phone call because he is lonely.
Mr McKenzie said that since Wilson is accompanied by two people whenever he leaves prison grounds and is not allowed anyone at his residence, it is difficult to see what the risk would be of speaking to anyone on the phone.
The lawyer would discuss with Wilson on Friday whether he wants to appeal.