China and Russia have banned Fonterra milk products following revelations of contamination, and at least three other countries may follow suit.
Government ministers met with the company on Sunday afternoon to discuss the scare, which involves the potential for some of its whey protein to cause the paralytic illness botulism.
The manufacturer of infant formula, Karicare, has recalled two of its products from New Zealand shelves.
Fonterra's animal feed subsidiary, NZAgbiz, has also issued a recall on a small amount of calf milk replacer in the North Island.
There have also been product recalls in China and Vietnam.
Economic Development Minister Steven Joyce said Fonterra has been on the back foot over the issue, and he's issued them a stern warning.
"The Government's tolerance for any delays in further information is incredibly low, and it was only appropriate that we make it very clear to them today about where people's heads were on these things ... and that they need to communicate very, very clearly to the officials and to the public."
Mr Joyce said he believes Fonterra got the message.
Fonterra's head of NZ milk products Gary Romano said all contaminated product had been recalled or stopped from making it onto the market, or companies affected were satisfied there was no health risk.
Fonterra said its number one focus was public safety, and it would answer questions over possible disciplinary action, and the financial impact to the company, at a later date.
NZ's export markets move against Fonterra products
China has halted the import of all New Zealand milk powder after it was found that products sourced from Fonterra may cause botulism.
Most of China's dairy imports come from New Zealand, which relies on dairy for 25% of its exports.
Russia's ITAR-TASS news agency says that country has suspended imports and circulation of Fonterra products.
There are reports Thailand has ordered a recall of Fonterra products imported since May.
Australia says a contaminated batch on its way there has been located and will be quarantined.
Karicare's manufacturer has recalled two products from the New Zealand market.
A statement on Nutricia's website said safety for customers is its primary concern.
They are Karicare Infant Formula Stage 1 and Karicare Gold+ Follow On Formula Stage 2.
Only the products in New Zealand are being recalled.
Fonterra on Sunday announced that its animal feed subsidiary, NZAgbiz, is recalling a small amount of calf milk replacer sold in the North Island.
Whey protein contaminated last year - Fonterra
Fonterra said on Friday three batches of whey protein have been contaminated and eight companies operating within New Zealand, have been told of the risks of customers developing botulism.
In a statement, the Chinese quality agency said it had contacted the New Zealand Embassy to urge the New Zealand Government to take action in a timely manner.
It also told agencies throughout China to strengthen inspection over milk products from New Zealand.
Trade Minister Tim Groser told Television New Zealand's Q+A programme that the markets to which contaminated whey protein concentrate have been exported are: Australia, China, Malaysia, Saudi Arabia, Thailand and Vietnam.
Mr Groser said New Zealand has notified Infosan, the World Health Organisation's international food safety regulators network.
Fonterra says three batches of whey protein were contaminated with a bacterium last year.
Fonterra has sent chief executive Theo Spierings to China to deal with the problem and overseas governments have also been informed.
The contamination happened last year, but MPI acting director-general Scott Gallagher was not told till Friday.
AAP reports about 38 tonnes of whey protein concentrate manufactured at Fonterra's Hautapu plant in Waikato were contaminated by an unsanitary pipe in May 2012.