Japan struggled to a 2-1 home victory over Yemen in their opening match of the final round of 2011 Asian Cup qualifiers.
The three time Asian champions failed to gain automatic qualification for the 2011 Asian Cup after losing to South Korea in the third-place playoff at the 2007 tournament won by Iraq.
Japan's next game is in Bahrain on January 28.
Hong Kong is the other side in Group A. The top two teams from the five groups advance.