Frank Field questioned whether current record levels of migration into Britain were “sustainable”.
And he told the BBC News website the UK was in danger of becoming a “global traffic station” for migrant workers.
He urged politicians on all sides to stop ignoring public concern on the issue before the BNP found a leader with the “talent” to exploit it.
The UK is currently receiving the highest number of immigrants in its history, following a surge in migrant workers from new EU member states.
When the EU expanded to 25 members in 2004 the UK, Ireland and Sweden were the only countries which decided not to restrict people from the new member countries – notably Poland – taking jobs.
At the time the UK government predicted 13,000 workers a year from the new EU member countries would move to the UK for work, but the actual figure of registered workers was about 329,000 in 18 months.
According to the latest available figures, for 2004, the overall migration picture saw 359,000 people leave the UK while 582,000 settled in the UK. The figures do not include illegal migrants.
Tony Blair’s official spokesman distanced Downing Street from Mr Field, saying the MP “speaks for himself” and added that “you have to remember the great economic benefit of migration“.
Mr Field, a former welfare minister, questioned whether this level of immigration was sustainable without “dramatic” changes to the character of the country and hitting poorer areas, which have to absorb migrants.
“This is the most massive transformation of our population. Do we just merely accept this as another form of globalisation? That it doesn’t matter where you are, or that you belong to a country and have roots? That we are all just following the jobs?” Mr Field told the BBC News Website.
People who questioned mass immigration were often accused of “playing the race card” but, Mr Field argued, this was “just another way of closing down debate”.
The Birkenhead MP added: “There will be economic gains [from immigration] but I am just raising whether any country can sustain the rate of immigration we are now suffering.
“If we are not careful, we will be transformed into a global traffic station and that is not what most people mean by being part of a country.”
He added: “It is only because the BNP are so inept that the debate has not taken off.”
He said mainstream politicians had to address immigration “before the BNP stumbles on somebody with talent“.
“We are living on borrowed time. We can not continue on the assumption that the BNP will present leaders which turn off most voters, even if what they are saying is important,” he said.
Tony Blair promised a debate on immigration after last year’s general election but so far this had not materialised, added Mr Field.
And, he said, the Conservatives had stopped talking about the issue because they were trying to improve their image and “show they were up not up to their old tricks, whether the electorate want them to or not”.
The Conservatives were asked to comment on Mr Field’s remarks but have so far failed to respond.
Mr Field told BBC Radio 4′s The World at One it was not just a question of race.
“I think you can make a case that there are many more Africans that buy into the British way of life than people from Eastern Europe,” he said.
Local Government Minister Phil Woolas denied the government was ducking the immigration issue.
“I do not accept the accusation that the government has not discussed race or immigration,” he told BBC Radio 4′s PM.
“We have done. For goodness sake, the general election campaign a year ago featured immigration in a large way.
“I think there is a danger that the myth is perpetuated that politicians are out of touch. We are all of us constituency MPs like Frank and we deal with these problems in our constituencies.”
He stressed any debate on immigration had to be based on facts.
“You have to flush out the truth in areas where there are large numbers of East European workers working in the agriculture industry, because we have high levels of employment, which is obviously a good thing.
“But the government has introduced a worker registration scheme and that is well-controlled and works.”
He added: “Of course there are illegal immigrants in a modern society like ours, but we have to base this debate on the facts and not to do so is irresponsible and actually raises the sort of problems with the far right Frank is warning against.”
The Liberal Democrats also argue Eastern European migration benefits the British economy.
The party has backed the opening of the UK’s borders to migrant workers from Romania and Bulgaria, when they join the EU, provided the impact of such a move is properly monitored.
Responding to Mr Field’s comments, home affairs spokesman Nick Clegg said: “There is a fine line between political candour on the sensitive issue of immigration, and down right scare-mongering.
“In making his remarks in this way, Frank Field failed this test and risks exacerbating precisely those public concerns he is urging to confront.”
BNP spokesman Phil Edwards said the party had been calling for a debate on immigration “for many years” but its views were always misrepresented by the media.
He denied the BNP’s leader Nick Griffin was “inept”, adding he was “every bit as good as any mainstream politician”.
On Mr Field’s comments, he said: “He sounds as though he is ready to join the BNP, not that we would have him.”
The BNP wants a halt to all immigration and a programme of “voluntary repatriation”.
The UK Independence Party said it backed the government’s new quota system for migrant workers but argued it should apply equally to all – including people inside the EU.
“We would apply the points system to all immigrants, regardless of their colour,” he told the BBC News Website.
Its spokesman said he believed there would have been more of public outcry about immigration in the UK if the latest wave of migrants had been from Africa or Asia.
“The government is gambling on the fact because the latest wave of immigration is the same race, that people will not actually notice,” he said.
UKIP favours net zero immigration, with the same number of people entering the UK each year as entering leaving it.
Sir Andrew Green, of pressure group Migration Watch, said: “This rate of migration cannot be maintained without the most profound changes occurring in our society, and particularly, given where we know migrants first live, in our poorest areas
“We have for too long ducked a serious debate on the scale of immigration. The government have done their best to bury the numbers and the Conservatives seem to have lost their nerve.”
He said Mr Field was “astonishingly brave” raising the issue in the way that he had.
On Tuesday, the Local Government Association accused the government of underestimating the true number of immigrants in the UK, leading to under-funding and a burden on services.
Cheryl Coppell, chief executive of Slough Borough Council said migration statistics for the area were “woefully inadequate”.
Ms Coppell also said the influx of newcomers from Poland and other eastern European countries was threatening the social cohesion of the town, which already has 37% ethnic minority residents.
She said the new arrivals were hitting the employment prospects of the existing population and created overcrowding, with as many as 15 people living in one house.
LGA chairman Sir Sandy Bruce-Lockhart said the situation was similar in many parts of the country.
Ministers accept there are problems in some parts of the country but say the government had changed the way population statistics are calculated.
Although there are “local ups and downs”, the system works, they say.