Organisers of the World Congress of Families regional conference are "getting desperate" to find a location for the Melbourne event after the Australian Senate passed a motion calling on politicians to boycott the group.
One of the organisers, who wanted to remain anonymous, told BuzzFeed it is looking at three more locations - one in Melbourne and two in regional Victoria - before it considers canceling the event.
"We're getting pretty desperate at the moment... and who knows, we may not have it," said the organiser.
BuzzFeed understands the conference's favoured venue is a regional centre 45 minutes drive from Melbourne's CBD.
The Endeavor Forum is organising the event for the World Congress of Families which is coming to Australia to honour Social Services Minister Kevin Andrews with one of its top international awards, "Natural Family Man of the Year".
Four locations have already pulled out following media controversy over some of its pro-life, anti-gay marriage views, including a highly publicised moment when Coalition Minister Eric Abetz was caught defending a link between abortions and breast cancer.
Greens Senator Larissa Waters successfully passed a Senate motion on Thursday morning, calling for all Australian politicians to boycott the weekend conference, including Mr Andrews who is due to open the event.
"The World Congress of Families campaigns internationally for the criminalisation of LGBTIQ people and against women having abortions, even when their lives are at risk," said Senator Waters.
She was joined by Labor MP Andrew Leigh and pro-gay marriage group, The Vocal Majority (pictured), on the lawns of Parliament House to symbolically present a photo album of pictures of diverse families to Mr Andrews.
"Today Minister Andrews refused an invitation to accept a large photo album of diverse family photos submitted by hundreds of Australians, opposed to the Congress' discriminatory view of what a family should look like."
According to the Guardian Australia, protestors are vowing to picket the conference if it is held this Saturday.
Mr Andrews' office has not returned calls for comment.