The first group of an expected 10,000 asylum-seekers has been greeted in Munich after an arduous journey through Hungary and Austria.
Germans applauded and offered sweets as about 450 asylum-seekers arrived on a special train service.
Meanwhile, the Hungarian authorities say no more buses will be laid on for those trying to reach the Austrian border.
Hundreds more are now reported to have set off from Budapest on foot.
Those who had already left to walk across the border into Austria, the refugees, many of them fleeing war in Syria, were whisked by train and shuttle bus to Vienna, where many said they would continue on to Germany.
The plight of the asylum-seekers has highlighted the European Union's struggle to deal with a surge of people flooding in from war zones in the Middle East and North Africa.
Last week, there were chaotic scenes in Budapest as Hungary blocked asylum-seekers from travelling by train to Western Europe, saying it was obliged to register them.
Crowds broke through security lines and began walking the 175km to the border with Austria.
Under mounting pressure, Hungary opened its border and provided buses to take the asylum-seekers - mostly Syrians and Iraqis - to Austria, which expects to have received up to 10,000 people by today.
About 3,000 exhausted people crossed the Austrian border yesterday and were received in a Red Cross Centre. They were offered the opportunity to claim asylum in Austria or carry on to Germany.
The move comes as European Union states are struggling to agree on how to deal with an unprecedented surge in refugees.
Hungary's government eased restrictions on transit after many asylum-seekers overwhelmed police cordons and set off towards the border on foot on Friday.
More on the crisis in the Middle East and Europe from RNZ
Buses began picking up people from Keleti station in central Budapest, where thousands had been camped.
Vehicles were also sent to take those who had decided to walk along a motorway to Austria.
On Saturday Hungarian government spokesman Zoltan Kovacs told the BBC there would be no more buses or trains to take the asylum-seekers on to Austria.
He said the transport had been arranged as a one-off, because of fears for people's safety.
When the buses arrived, some of the asylum-seekers argued with officials, fearful they would be arrested rather than sent to Germany, the BBC's Matthew Price reported.
But early on Saturday, groups began crossing the border on foot. Some Austrians displayed welcome signs.
Austrian Red Cross workers at a makeshift centre greeted them with blankets and tea.
"I feel [at] home," said Ayaz Morad, one of the first to arrive. "This is a great land - nice people, nice government."
Mohammad, a Syrian, said he was happy to leave but warned other Syrians against travelling to Hungary because the situation there was "ugly".
Many of the asylum-seekers hoped to travel on to Germany, which has said it expects to take in 800,000 people this year.
Austria's Chancellor Werner Faymann said that after talks with his German counterpart Angela Merkel, the two countries would allow the asylum-seekers in due to the "emergency situation" in Hungary.
But he said he expected Hungary to respect any EU quotas for asylum-seekers - something Hungary, along with the Czech Republic, Poland and Slovakia, has rejected.
Hungary has become a major transit nation for people fleeing the Middle East and Africa, and seeking to reach north and west Europe.
The Hungarian parliament on Friday approved tougher border controls and penalties for refugees, underlining divisions within the EU on how to tackle the crisis.
Hungarian Prime Minister Viktor Orban has said the surge in arrivals was "Germany's problem", since that was where most people wanted to go.
But Chancellor Merkel has called for refugees to be fairly divided among EU members.