Research by the Department of Conservation shows there has been a fall of 31% in the number of sea lion pups born this season.
The Royal Forest & Bird Society acknowledges the decline cannot be directly linked to squid fishing, but says the industry is threatening the population's ability to recover.
Fisheries Minister Phil Heatley has raised the number of sea lions that can be killed as a result of squid fishing this season from 81 to 113.
However, he says the industry has voluntarily lowered that number to 95, in response to the latest fall in seal births.
Forest & Bird wants the quota gradually reduced to zero.