Hall received 19,924 votes compared to Webb's 18,429.
74% of the 66,065 registered voters turned out to the polls in Thornbury and Yate compared to 65% in the 2010 general election.
Webb will now relinquish his five-year role as pensions minister, in which he led the pension freedoms policy.
According to the Bristol Post, Webb was disappointed to have been beaten by some 1,500 votes
He said:
"We get a very positive response from people who often said they would vote for me because of things that I had done for them.
"But occasionally they would say 'I would like to vote for you but...' and that was generally a national issue – sometimes it was about political ties and even with the best campaign you can't resist that.
"When I got up this morning I thought I would have the same job tonight and so I've not planned my next move.
"Representing the people of Thornbury and Yate has been a huge privilege. Nobody ever thought I would win here, no one ever thought I would get re-elected and so to serve 18 years, five years in government and deliver some pension reforms I am proud of and grateful to local people for giving me that chance."
Jon Gwinnett, pensions technical manager at Nucleus commented:
“Steve Webb, while not without fault, has been the most successful pension minister for years. His loss will be keenly felt. So too will that of Gregg McClymont - together they had brought a degree of stability to pension policy outlook in the face of the Treasury’s interventions.
"What will the future hold? Ros Altmann has already been promised a place in Government – would it be too much of a leap to see her take a role formally in the pension ministry? Whoever is chosen, never has a steady hand on the tiller been more important, as we continue to navigate the early surge of the pension freedom sea."
Housing minister Brandon Lewis held his seat in Great Yarmouth, but work and pensions minister Esther McVey was defeated in Wirral West.
