Scotland Yard said the unnamed man in his 50s was also detained on suspicion of indecent assault, abuse of position of trust and misconduct in public office.
Tory chief whip Chris Heaton-Harris urged the MP to stay away from Parliament but has not suspended the whip, meaning the suspect remains a member of the parliamentary party.
The Metropolitan Police said officers received a report in January 2020 relating to alleged sexual offences committed in London between 2002 and 2009.
The investigation is ongoing, led by officers from the Met’s central specialist crime team.
It was understood the Conservatives will not make a decision on whether the suspect has the whip suspended until after the police investigation is concluded.
Foreign Secretary Liz Truss said she was “very, very concerned” that a Tory colleague had been arrested.
Asked how she felt when she heard the news, she told Sky News: “I’m very, very concerned about the reports that I’ve heard. Clearly this is a matter for the police.
“The police are currently working on the case but it is worrying that we are seeing these appalling … accusations, again, about a parliamentarian.”
Asked if the MP should have been suspended or had the whip withdrawn, she said: “I think it’s a matter for the police about whether that individual is named.
“I don’t know who it is. I know that the police are looking at it and investigating it and I think we need to wait for the police to conduct those investigations.”
The Prospect union, which represents hundreds of staff working in the Houses of Parliament, said firmer action is needed.
Deputy general secretary Garry Graham said: “What will it take for Parliament to finally take its responsibility to its staff and visitors seriously and suspend access to the estate for parliamentarians under investigation for sexual offences?
“Voluntary agreements to stay away do not work, as demonstrated by Imran Ahmad Khan’s attendance at Westminster whilst investigations were ongoing, despite agreeing to stay away.
“Parliament has the same responsibilities towards its staff as any other workplace and it must live up to them.”
The arrest comes amid renewed scrutiny of sleaze in Westminster following a string of cases.
Former Conservative MP Imran Ahmad Khan quit the Commons earlier this month after being convicted of sexually assaulting a boy.
David Warburton had the Conservative whip withdrawn after allegations of sexual harassment and cocaine use emerged.
Over the past year, around 15 MPs are suspected to have been investigated by Parliament’s Independent Complaints and Grievance Scheme (ICGS) watchdog that examines allegations of bullying, harassment or sexual misconduct.
Neil Parish stood down as the Conservative MP for Tiverton and Honiton earlier this month after admitting watching pornography in the Commons.
The latest arrest also follows a Conservative MP being arrested on suspicion of rape in 2020. He was never identified and police took no further action after an investigation.
Conservative sources declined to identify the individual in the latest case, citing the need to protect the identity of any complainant.
A statement from the Tory whips’ office said: “The chief whip has asked that the MP concerned does not attend the Parliamentary estate while an investigation is ongoing.
“Until the conclusion of the investigation we will not be commenting further.”