Prime Minister John Key says Housing Minister Phil Heatley has made silly mistakes in the use of his ministerial credit card but won't be standing him down.
Phil Heatley and his Cabinet colleague Gerry Brownlee have apologised for misusing their cards.
Ministerial cards are supposed to be used only for official business expenses and reimbursement for personal expenses is not allowed.
Mr Heatley, the MP for Whangarei, has admitted several breaches of the ministerial credit card policy and came close to tears as he told reporters at Parliament that misusing his card has cost him dearly.
"It doesn't look good at all and I'm terribly embarrassed and I've apologised to the Prime Minister, I apologise to the public, I apologise to the people of Whangarei and I'm just not particularly pleased with myself."
Mr Heatley says the use of the card was unintentional. He has repaid $175 for wine at last year's National Party conference and intends to pay back about $900 spent during a South Island visit, when his family accompanied him while he was on ministerial business.
Mr Key says in some instances Mr Heatley has simply used the wrong card, while one or two were outside the guidelines and have been repaid.
"In my opinion he wasn't attempting to rort the taxpayer, but they are stupid mistakes," he says.
The Prime Minister says Mr Heatley will keep his ministerial portfolios of housing and fisheries.
Gerry Brownlee has cut up his card and repaid $151 he spent on lunch with his electorate staff last year. He says he is deeply embarrassed.
"Look, you don't come into Parliament to try and advance yourself or to ingratiate yourself with a $150 lunch. It was a silly mistake," he said.
The Labour Party says John Key promised high standards for his adminstration and must take a tough line with ministers who have misused their credit cards.
Labour leader Phil Goff says the Housing Minister showed poor judgement and New Zealanders who are struggling to make ends meet will be angry about the way the card was used.