Only about half of Defence Force staff describe morale as good, and fewer than half think their contribution is valued.
The Defence Force has published its first ever annual engagement survey of navy, army and air force staff, involving two thirds of its military and civilian personnel.
Previous attitude surveys revealed a drop in morale after the government unveiled a plan for big cuts in funding and staff numbers in 2010.
Defence Force chief Lieutenant-General Tim Keating said he had been sent a clear message that the top command level needed to inspire more confidence in its leadership.
He said the force scored well in questions reflecting commitment and comradeship, but he and the organisation's senior leaders and commanders had to improve their communication and engagement with staff.
One area needing change was people's faith in the system. Lieutenant-General Keating said just 22 percent believed that things will change for the better as a result of the survey, and he took that very seriously.
"They don't believe that the CDF [Chief of Defence Forces] reading these results will change and that's an area of concern for me, so I've got to restore that sort of faith that I'm reading this."
Lt-Gen Keating also said only 30 percent of those surveyed believed poor performance was being dealt with effectively.
"At the front end of our organisation we demand excellence on operations, but if we look at the big organisation, they're saying 'too much mediocrity in there'.
"We've got to pursue a performance culture in the organisation - we've got to drive that so what we do is best practice in operations - it has got to be brought back into the way we raise and train our defence force."
Lt-Gen Keating said it was pleasing that the staff level of engagement was found to be on a par with other uniformed government organisations such as the fire service, customs and police.
But not all of those who have been in uniform agree. Retired police inspector Chris Salt was in the army territorials for 10 years as well as being a trainer and strategy manager at the Police College.
He said despite most saying they were proud to work for the Defence Force, only 38.8 percent believed there was high morale in their workplace.
"You can be proud to be in an organisation, but that doesn't mean that you think the organisation is necessarily where it should be."
He said that could be quite difficult in an organisation like the armed services, with a very clear mandate to protect the sovereignty of New Zealand.
"But in fact they haven't been given the tools to do it and the people in the organisations must know that, so that must make a contribution to levels of morale and their sense of worth."
Mr Salt said if the NZDF was not rewarding performance particularly well, it was because people were being taken for granted.
Lance Beath of Victoria University's Centre for Strategic Studies said if the leadership did not deal with poor performers it would have a corrosive effect.
"What this boils down to is the possibility that those who are outstanding performers will be dragged down in their efforts when they look to the side and see poor performers who are allowed to carry on without being sanctioned or fired."
His colleague Robert Ayson agreed that what stood out from the survey was the low percentage of people who believed poor performance was being dealt with effectively - under 30 percent.
"How do you deal with that? That's the real one, that the perception, that's the really interesting one. (But) there are always going to be gaps between perception of your immediate circumstances and those of the organisation as a whole."
However Mr Ayson said job satisfaction surveys for the British and Australian armed forces had similar results in areas such as pride in working for the service and morale. He said the NZDF retention rates had improved and there were "no huge alarm bells ringing".
"From what I can sense, very anecdotally, defence is in a better position than it was a few years ago when it was going through these rather drastic changes, and some of the bounce back from that may still be yet to come."
The Defence Force admitted in September it needed to lift its game when it came to managing the changes needed for it to be able to better respond to future events.
A review led by Paula Rebstock for the State Services Commission outlined the need for the force to be able to respond to future challenges, and found it did not have a consistent track record of implementing change, and that needed to improve.
Lt-Gen Keating said the Defence Forces had an organisational plan in place that should deal with many of the areas of concern.