The Swedish Court of Appeal ruled on Monday that John, Mark and Darren O'Connor must serve the three years and eight months sentence handed down last May.
Another brother Edward O'Connor won an appeal against two charges of money laundering and his sentenced was reduced to two years and three months.
The Tipperary men had been jailed in Stockholm over a €4 million tax dodge where they ran a driveway and landscape business, funnelling undeclared cash back to Ireland.
Swedish authorities launched an investigation last year after they were alerted to millions being transferred out of the country through personal bank accounts
The Criminal Assets Bureau in Ireland were drafted in to support the probe while an Israeli security firm helped to crack the brothers iPhone and laptop records.
Authorities in the UK were also contacted in a bid to track down a fifth brother Thomas who eluded police when the others were arrested in October last year.
Ironically, dozens of witnesses who employed the O'Connor brothers, from Carrick-on-Suir, said they were satisfied with how the work was carried out at their properties.
The meticulous investigation by Swedish police however, underlines the difficulty in keeping track of work crews who avoid using permanent addresses and regularly leave the country.
The Swedish Court of Appeal also dismissed three of the brothers' claims that Sweden is their home, in a bid to avoid deportation after their release.
Mark, Edward and Darren (also known as James Kenneally) said they see Sweden as their home and only visit Ireland.
But the court considered their connection to Sweden to be "weak" and said they have property in Ireland, only arrive in Sweden for the landscaping season, haven't learned to speak Swedish and sent cash back to Ireland.
At the time of the police raid on October 20, last year, there was hardly any money in their bank accounts, apart from Edward's. They've never had a permanent residence in Sweden, none of their wives ever worked there and they were vague about how long they spend in Ireland, according to the ruling.
It also rejected the argument that their expulsion would affect their families.
The brothers and two Swedish people were arrested following an investigation into the transfer of large sums of money from Edward O'Connor's bank account.
Much of the cash paid to the brothers through their business ended up being transferred straight back to Ireland.
They had to keep opening new accounts when banks queried the large deposits and transfers being made through their personal accounts. A fifth brother Thomas O´Connor (37), was registered as the chief executive of one of the landscaping firms.
He was found in the same aparthotel as Edward and John when raided by police but escaped before he could be questioned.
"The prosecutor has since searched for him in Sweden, Ireland and the United Kingdom without success," it said.
A Swedish woman who handled their administration, Caroline Lindstrom, was convicted of lesser charges and was not jailed by Stockholm District Court.
However, the Court of Appeal this week imposed a two-year sentence for her role as the administrator.
She had teamed up with John O'Connor in 2010 when he did a job for her father and helped him navigate Swedish red tape.
The district court verdict last May referred to her testimony in which she described John O'Connor as the leader of the group.
"John O'Connor is the head of the family. He can be very dominant. His brothers listen to him and respect him," it said.
The district court case showed how police went to extreme lengths to ensure they could prove their case.
They tracked down dozens of customers to prove the link between cash payments to the brothers' companies which were instead paid through personal accounts.
Surveillance photos of them at work were taken as well as information gleaned from their electronic devices to prove they were working together.
Using software from the controversial firm Cellebrite they were able to piece together how the brothers operated.
Footage from cash machines was even used to show how the brothers used each other's accounts to withdraw money. And texts between John O'Connor and Lindstrom revealed how their professional relationship cracked from investigators.
"One thing you remember for the rest of your life Caroline, I have money my hole life, money will never come between me and my family and real friends," wrote John. But Caroline urged him to leave it to her, telling him: "We need to show them a united front."
But John replied: "Told you don't text me biggest mistake I made was I should of let you in the petrol station in Bromma."
Darren O'Connor, who changed his name to James Kenneally, after running up a previous tax debt in Sweden tried to claim it was a different person.
While being interviewed his wife Chantal said Darren had left her and she had a new boyfriend called James Kenneally.
But when it was put to her in a police interview that Darren and James are the same person she replied: "Yes."