Theresa May won the battle against the immigration tribunal decision that deporting "MA", who cannot be named, would breach human rights.
The man was jailed for 10 years after he attacked a pregnant woman when she was asleep in her own bed in 2006.
He could now have a fresh assessment of his case under immigration rules.
The Court of Appeal judges said he could have this assessment because of the passage of time that has occurred in dealing with his case.
This means he cannot immediately be sent back to Somalia.
Lord Justice Richards, sitting with Lord Justice Ryder and Sir Colin Rimer, said MA originated from northern Somalia and was either 30 or 32 years old.
In 1988 he fled with the rest of his family to Ethiopia.
By 2004, all the family members had come to the UK and gained British citizenship except MA, who had not applied.
Of MA's attack on the pregnant woman, Lord Justice Richards said: "He held a knife to her throat, threatening to kill her, while he raped her and committed a further offence of attempted rape."
MA's appeal against deportation was allowed by an immigration first-tier tribunal on the grounds that removing him would be a breach of his Article 8 right to "private and family life" under the European Convention on Human Rights.
The home secretary's appeal to the upper tribunal was dismissed, but she renewed her challenge in the appeal court.
The judge said the tribunal had failed to acknowledge "the great weight" that had to be attached to the public interest in deporting foreign criminals.