An evacuation centre was established at Richmond Town Hall, where about 12 people slept overnight as others spent the night in hotels or stayed with friends and family.

Tram routes 48 and 75 have being diverted due to road closures, with replacement buses brought in to try to minimise disruption.

Richmond resident Ian Dalton was asleep with his wife on Monday night when they heard loud knocking about 1.30am. They got up to find three or four police officers evacuating everyone from the street.

"We asked what was going on, and they said you are going to have to leave, we are evacuating Mulberry and Union streets because the crane is in a precarious position and it may fall over," he said.

"We had no choice whether to stay or not, we have to leave. We own two houses, the other is an Airbnb and we had some guests from Israel there so we had to wake them up and try to explain to them what was happening."

The couple headed to the evacuation centre with their border collie, Wally, while also assisting several elderly neighbours. But Mr Dalton did not end up staying at the centre, he spent the night in hospital instead, due to some concerns about pre-existing heart issues.

"I had a quadruple bypass 18 months ago and I'm pretty fit and healthy but I just had this slightly pounding heart for 24 hours before it happened. I suppose because I was a bit tired and with the anxiety I was concerned about it so I spoke to the ambulance people there and they took it pretty seriously and took me to Epworth [Hospital] so I got a good bed.

"My wife got a rubber mat and slept on the floor in the council so I might be in the bad books today," he said with a laugh.

Some residents have been told it could be at least 24 hours and possibly 48 hours until they are allowed back to their properties, however the Daltons hoped they may be able to get back into Mulberry Street on Tuesday.

"If the crane falls it will be pretty dramatic. It's very heavy and it could cause huge destruction and it would be very, very expensive. This is prime property in Melbourne and you flatten a few houses there, you are talking about tens of millions of dollars," he said.

The manager of the CFMEU's occupational health and safety unit Gerry Ayers said he had been on site on Tuesday morning as emergency services, the engineers, crane company and building company worked on a plan to dismantle the crane.

"The sticking point is the huge gusts of wind. We can't send someone up in a man box attached to a crane, which is what we need to do, it's far too dangerous for them," he said.

"But the engineer is confident the crane is OK as it is, it's stable, and the weather prediction is that winds will die down at 4pm this afternoon."

He said they planned to set up two mobile cranes in Lennox Street to assist in the retrieval of the dangerous crane. Setting up the two additional cranes could take five or six hours.

"The retrieval process will commence tomorrow. Tomorrow afternoon they should be able to come back in and relocate their homes unless something goes terribly wrong, obviously we are planning for that not to occur," said Mr Ayers.

A spokeswoman for Yarra City Council said that as well as the 12 people in the Town Hall, the council had provided alternative accommodation for about 30 others.

The owner of the company that provides the crane, Michael Clark from Clark Cranes, told radio station 3AW it seemed the jib of the crane - the boom or arm of the crane - had not been left in the correct upright position.

"It needs to be left down at a certain radius... so the wind can grab it and move the crane around of a night-time," he said.

"But if left up too high, the wind will grab the boom and push it backwards," he said.

He said engineers were on site working to secure the crane. No injuries have been reported and Worksafe has been notified, according to a police spokesman.

A severe weather warning for damaging winds remains in place on Tuesday.

Hundreds of homes across the state were left without power after a night of powerful winds.

In the outer eastern suburbs of Melbourne, 217 homes in Emerald were experiencing power outages on Tuesday morning, as well as 252 residences in Upwey.

In Altona North, in the city's west, just under 40 homes were experiencing blackouts, as well as 128 properties in Thornbury in the inner north.

Out of Melbourne, about 120 homes near Bendigo were affected, and in the suburbs near Warburton, over 200 were still without power.

Properties on the Mornington Peninsula were also affected by blackouts overnight but most had been reconnected to power on Tuesday morning.

The Bureau of Meteorology expects winds from the west will hit the state prior to a cold front, which will arrive in Melbourne during the afternoon.

"We are expecting winds to pick up again from lunch time, we will see winds gusts of up to 90km/h so not as strong as they were last night and a band of gusty showers as well with that change," said bureau senior forecaster Stephen King.

Winds will ease on Tuesday night. The city could see a couple of millimetres of rain in the afternoon.

It comes after wild winds ripped roofs off houses and brought down trees on Monday night, damaging buildings across Melbourne.

Winds of 113km/h were recorded over Port Phillip Bay about 7pm while gusts at Moorabbin Airport, in the city's south-east, reached 95km/h shortly after.

The SES Footscray unit was called to Fitzroy Gardens in East Melbourne just after 10pm on Monday night after a "massive 100-year-old tree" feel across the pedestrian path. It remained there, surrounded by tape, on Tuesday morning.

Pakenham was the worst-affected region, followed by Brighton. Most of those calls were for trees down and damaged buildings.

SES workers were also called to the Richmond crane incident, and a separate case of a timber beam falling from scaffolding onto a neighbouring building site, a spokeswoman said.

Last week, winds of more than 140km/h moved across Victoria and the SES receive more than 800 calls for help.