The Refugee Action Coalition says authorities on Manus Island in Papua New Guinea are removing basic rights from refugees in an effort to force them to move out of the detention centre.
A spokesperson, Ian Rintoul, says about 150 people found to be refugees still living in the detention centre are refusing to move to new accommodation in East Lorengau.
He says most do not want to leave because they fear for their safety outside of the centre.
Mr Rintoul says authorities are coercing them to leave by taking away their rights.
"The access to the medical centre will be denied to them, internet and phone will be restricted. They are saying until you move out of the detention centre we will progressively remove your access to activities, to excursions, that we will make life intolerable for you."
Ian Rintoul says there are 19 refugees at East Lorengau.
The 110 million US dollar East Lorengau transit centre, built by Australia, is designed as temporary accommodation before refugees are moved to other parts of Papua New Guinea.