The Tourism Industry Association says the spate of tourists having their car keys taken off them by local drivers smacks of racism and vigilante action.
Tourists' keys have been grabbed in at least five incidents recently by people concerned about the visitors' erratic and dangerous driving, and one tourist was punched in the face.
Chris Roberts, the association's chief executive, said about 5 percent of crashes are caused by visitors and that rate has not changed in the past decade years, though the nationality mix of tourists has.
"It is mostly the Asian drivers that are getting targetted by these actions," he said.
"When it was mostly people who looked like us behind the wheel, somehow we were more tolerant."
Vigilante reports worry tourists
Don and Cynthia Sparks, who were visiting from Canada, said they were pleased they decided to catch buses and trains during their New Zealand visit instead of driving.
They said a local snatching their car keys would have ruined their holiday, and called New Zealanders' attitudes towards foreign drivers 'frightening'.
"You're not very comfortable," Don Sparks said.
"You would be very on edge and probably prone to make more mistakes, as opposed to feeling comfortable in what you were doing. I wouldn't [drive]," added Cynthia Sparks.
Lu, a tourist from Beijing, had also chosen not to get behind the wheel during her trip.
She hired a car and driver for her family, and said locals taking car keys off foreign drivers set a dangerous precedent.
She said many Chinese tourists were frightened by the reports of New Zealander's confiscating keys.
"They don't know who they are, because they didn't wear a uniform like a policeman," Lu said.
"They probably thought it was a robbery or something like that. I think they are scared."
University of Auckland academic Changzoo Song said he suspected some of the key taking incidents were really cases of road rage, which had escalated when the driver turned out to be a foreigner.
In the case of the latest incident, involving a foreign driver on the Otago Peninsula, Dr Song said a verbal warning was probably all that was needed to sound the alert about dodgy driving.
"I would have warned the people that were blocking the road rather than taking their key because that's a rather serious action - taking someone's property."
Aucklander Kenneth Leong said New Zealanders needed to step back and take a look at the bigger picture.
He said with so many Chinese people now visiting New Zealand, there were bound to be some who did not drive safely.
In Christchurch alone, he said, about 50,000 people attended Chinese New Year celebrations.
"Statistically the risk of an accident involving a Chinese driver would increase, because of the sheer amount of people around for Chinese New year."
The tourists spoken to by Radio New Zealand all agreed that New Zealand was a beautiful and amazing country but they said issues of poor driving were best left to police and not the locals.