He had commissioned polling in the harbour-side electorate that suggested a woman would be better placed to win.
Pundits believe a strong independent candidate could succeed meaning the coalition would lose its majority in the federal parliament.
"You'd be foolish to underestimate the significance of the climate issue in relation to that constituency," former Wentworth MP and one-time Liberal leader John Hewson said on Wednesday.
"It's not going to be too difficult for a significant named independent, with strong credentials on climate, for example, to take that seat in a by-election where there's already going to be a very substantial protest vote against the government."
City of Sydney councillor Kerryn Phelps is considering whether to stand as an independent. She has a high profile as a past president of the Australian Medical Association.
Independent Licia Heath has declared her hand and she's backed by state MP Alex Greenwich who on Wednesday told AAP the investment manager was "committed to action on climate change".
Liberal powerbrokers in Canberra are pressuring Wentworth preselectors to pick a woman on Thursday night after Mr Bragg dropped out of the race.
The polling revealed the Liberal primary vote has fallen to 39 per cent from 62 per cent when Mr Turnbull was the local member.
Woollahra councillor Mary-Lou Jarvis and Sydney East Business Chamber chair Katherine O'Regan now stand the best chance of being preselected when 210 Liberal members meet at Easts Rugby Union Club.
Fellow candidate Carrington Brigham this week sent an email to some members asking them to vote on merit and not gender.
"If you believe I am the right person for the job, don't mark me down for my gender," Mr Brigham said in Tuesday's email.
Tim Murray is Labor's candidate for the Wentworth by-election. He's being supported by Mr Turnbull's son Alex Turnbull.
AAP, Fairfax Media