Federal police and parliament's speaker are refusing to say why a prominent anti-Beijing activist was ordered to leave a public section of the building or risk facing arrest.

Human rights campaigner Drew Pavlou had been due to meet the head of Parliament's intelligence committee on Wednesday but claims AFP officers told him he'd been deemed a "high risk individual" before escorting him out.

"They just said sorry mate we don't want to have to do this, but our higher ups insist you have to leave now because you've been flagged as a high-risk individual," Mr Pavlou claims.

In July the 23-year-old was also detained by British police for over twenty hours, which he claims occurred after the Chinese embassy in London framed him with a hoax bomb threat.

"Because of the way I was targeted by Chinese authorities and how they've tried to smear me as this terrorist bomb-threat guy," Mr Pavlou told the ABC.

On Wednesday morning the campaigner had met with Liberal Senator James Paterson inside Parliament, to discuss what he describes as a campaign of harassment and intimidation directed towards him by pro-Beijing activists. 

At lunchtime he was approached by AFP officers as he dined in the building's public cafe ahead of another meeting scheduled for that evening with Labor MP Peter Khalil, the head of Parliament's powerful Joint Committee on Intelligence and Security.

Shortly after the incident Mr Khalil appeared on the ABC's Afternoon Briefing program and indicated it was being investigated by Parliament's Speaker and the President of the Senate.

"I haven't met with him yet, but I think this is really a matter for the presiding officers to determine what's going on," Mr Khalil said.

"I think it's with them at the moment, the Speaker and the President who have control of the security or preside over the security of the House and the Parliament and the Senate".

A spokesperson for Parliamentary Speaker Milton Dick said the directive to remove Mr Pavlou did not come from his office and was a matter for the AFP.

In a statement the AFP confirmed it had "engaged" with a man at Parliament House who then "departed the building voluntarily" and said, "no further comment will be made on the matter".

Liberal Senator James Paterson, the Opposition spokesperson for countering foreign interference, says he's concerned by the events.

"Australian citizens who protest frequently visit Parliament House for meetings, including union officials and environmental activists, without being asked to leave by the AFP."