Feist will also donate all merchandise sales from her Dublin shows to Women’s Aid
The singer-songwriter said the decision had been "incredibly difficult" and that she had chosen to "distance myself from this tour (but) not this conversation".
Feist had supported Arcade Fire at two Dublin shows at the 3 Arena, and Feist – full name Leslie Feist – said she came to her realisation that she must leave the tour while playing on stage and hearing her songs again over those two nights.
She will also donate all merchandise sales from the shows to Women’s Aid.
It comes after Butler faced accusations of sexual misconduct from four people, which were first reported by US music publication Pitchfork on Saturday.
In two statements to Pitchfork, Butler denied the allegations and said the sexual interactions he had had with the four had been consensual.
Arcade Fire recently began a major European tour, with Feist having supported them on the first two dates in Dublin on August 30 and 31.
But in a lengthy post on social media, Feist - whose full name is Leslie Feist and is known mononymously as Feist-, explained that she was "claiming my responsibility now and going home".
"At a pub in Dublin, after rehearsing with my band, I read the same headline you did," the post read.
"We didn't have any time to prepare for what was coming, let alone a chance to decide not to fly across the ocean into the belly of this situation.
"This has been incredibly difficult for me and I can only imagine how much more difficult it's been for the people who came forward. More than anything I wish healing to those involved."
Feist said that the situation had ignited a conversation that is "bigger than me" or her songs and "certainly bigger than any rock and roll tour".
"To stay on tour would symbolise I was either defending or ignoring the harm caused by Win Butler and to leave would imply I was the judge and jury."
She continued: "The last two nights on stage, my songs made this decision for me.
"I've always written songs to name my own subtle difficulties, aspire to my best self and claim responsibility when I need to.
"And I'm claiming my responsibility now and going home."
Arcade Fire has been contacted for comment.