The Public Service Association says moves to cut more public service jobs will be recessionary.
Prime Minister John Key says he wants to tackle a bloated public service, trimming back-office jobs and merging some departments to cut costs and improve efficiencies.
In his speech to the first Parliamentary session of the year on Tuesday afternoon Mr Key said there is more to be done to make government bureaucracy smaller and better.
PSA General Secretary Richard Wagstaff says merging some government departments makes sense but the kinds of cuts suggested by Mr Key will ultimately stall the country's nascent recovery.
He says there is a close correlation between spending on public services and strong economic performance.
Mr Wagstaff says taking a knife to the sector is not the best way to get more value for money and cultural change to encourage high performance is more important.
However Business New Zealand chief executive Phil O'Reilly says Mr Key's speech was about value for money and will appeal to voters.
He says he would have liked the speech to have included more on how to tackle youth unemployment, especially in the Maori and Pacific communities.
Mr Key says any cuts to the public service are about reorientating the sector to meet future needs, rather than saving a set amount of money.
He told Morning Report he wants a public service which is more outward looking and electronically driven, which means a logical combination of ministries and crown entities that the public can access more freely.
Mr Key says one merger being considered is the Ministry of Agriculture and Forestry combining with the Ministry of Fisheries, to form a Primary Industries Ministry.
The Labour Party says having fewer government agencies does not mean there will be better public services.
Labour MP Grant Robertson says it is important to look at each case on its merits.
CTU rubbishes figures
The Council of Trade Unions says figures on wage growth included in the Prime Minister's speech do not reflect the struggle by ordinary families to pay bills.
Mr Key said the cost of goods and services has risen by 6% since September 2008 while the average wage has gone up by 16%, meaning workers are better off.
CTU economist Bill Rosenberg says the figures are only half the story because two thirds of adults have incomes that are less than the average wage.
He says last year's rise in GST helped push the cost of living to the point where ordinary families are struggling to pay for food, power, fuel and other bills.
