Four women and four men are in contention for the 2022 award – which is open to fiction, non-fiction and poetry written in English from around the world.
Seven out of the eight writers are based in the UK and Ireland, and their books come from a range of publishers, including two titles each for Penguin General and Vintage.
The shortlist includes Brown’s debut Assembly, which tackles themes including identity, class and race, as well as The Magician by Toibin, a biographical novel about war and family.
Galgut is shortlisted for The Promise, which is set in Pretoria where the author grew up and takes place against the backdrop of South Africa’s transition out of apartheid.
The novel claimed the prestigious Booker Prize for 2021.
In Albert And The Whale, Philip Hoare charts the travels of German painter and printmaker Albrecht Durer, while poet Selima Hill’s Men Who Feed Pigeons contains seven “contrasting but complementary” poem sequences relating to different kinds of women’s relationships with men.
Also featured is China Room by Sunjeev Sahota, which tells the story of a young bride in rural Punjab as she faces love and betrayal.
Chair of the judges Tessa Hadley said: “We’re so excited by our shortlist for the Rathbones Folio Prize this year.
“Our eight books were chosen from a fairly dazzling longlist of 20. The books under consideration are all nominated by writers and so the quality of the work is very high.
“So many good books, prose fiction and poetry and non-fiction – so difficult to weigh one against another. We all brought certain passions to the table when we met.
“There were just a few books that had seized us from the first page and hadn’t let us down until the last, and then seemed even richer and larger on a second reading.”
The winner will be announced during a ceremony at the British Library on Wednesday March 23.
Carmen Maria Machado scooped last year’s Rathbones Folio Prize for her memoir In The Dream House, an account of domestic abuse in a same-sex relationship.