Labour's chief whip Chris Hipkins says his party is frustrated with the rulings of new Speaker David Carter.
Mr Hipkins and senior Labour MP Trevor Mallard were ordered out of Parliament after protracted arguments with Mr Carter last week.
Speaking on a TV3 current affairs programme on Saturday, Mr Hipkins said he believes Mr Carter is finding it difficult to change from being a partisan politician to an impartial referee.
"It wasn't a great week for anybody in the Parliament. But I think, ultimately, that comes down to the leadership that's shown by the Speaker and by the Prime Minister (John Key) as well.
"When the Speaker allows the Prime Minister to get away with having digs at the opposition with absolutely no penalty ... and then is very hard on the opposition - turfing opposition members out of the House - that is going to lead to a certain amount of frustration."
Mr Hipkins said it is up to the Speaker to take a lead on setting the right tone for behaviour in Parliament.
David Carter replaced Lockwood Smith, who was appointed High Commissioner to the United Kingdom earlier this year. Both have been National Party MPs.