"The Ethiopian government should immediately release 17 prominent Muslim leaders arrested as part of a brutal crackdown on peaceful Muslim protesters in Addis Ababa," HRW said in a statement.
The arrests took place following widespread protests staged by Muslims last month, to dispute what they said was unconstitutional government interference in religious affairs.
According to HRW, the detainees have been "mistreated" in custody and are being held without charge or access to legal advice.
"They have had no access to legal counsel or, in several cases, their relatives," HRW said. "Their lack of access to lawyers while detained in a prison known for torture heightens concerns about their safety."
Muslim groups in the country have staged weekly gatherings since the beginning of the year at two mosques in the Ethiopian capital.
Protesters accuse the government of forcibly imposing the Al-Ahbash branch of Islam, and appointing leaders of the Supreme Council on Islamic Affairs, who are traditionally elected by members of the Muslim community.
Muslims officially make up about 30 percent of the country's 83 million people.
The Ethiopian government denied repression and said the arrests were legal. "There is no crackdown. There are people who are legally apprehended, legally pursued, and this is within the legal bounds, the constitutional bounds, of the government," government spokesman Bereket Simon told AFP.
This month, police said several people had been arrested in connection with the demonstrations for intending "to carry out acts of terrorism".
They said the protests had been carried out illegally and accused the movement of extremism, according to a statement from the Foreign Affairs Ministry.
HRW accused police of illegally harassing, assaulting and arbitrarily arresting protesters and urged the government to engage in peaceful dialogue.
"The Ethiopian government should address the grievances of its Muslim community through dialogue, not violence... the security forces should be upholding the law, not breaking it," HRW Africa researcher Ben Rawlence said.
However, Bereket accused the rights group of meddling and said Ethiopia followed its constitution.
"This is a blatant disregard for the sovereignty of Ethiopia and this country is not led by Human Rights Watch so we go by our constitution, not by the invisible hand of Human Rights Watch," he said.
At least one of the arrested protestors is a journalist, while the bulk of the detained are members of an Islamic Committee created in January to represent the Muslim community when protests kicked off.
Several Muslim leaders have recently gone into hiding, and many refuse to speak publicly, citing fear of arrest.
"They are afraid of going to prison.... The Muslim community is now afraid of the government," said one Muslim living near Awoliya mosque.
Backed by the West, Ethiopia has long been a bulwark against Islamic extremism in the volatile Horn of Africa region.
In 2011, Ethiopian troops and tanks rolled into neighbouring Somalia to battle Al Qaeda-linked Shebab insurgents.
Government interference in religious affairs is illegal, according to the constitution.
In April, Ethiopian Prime Minister Meles Zenawi said freedom of religion is respected under the constitution, but said some Muslims in the country were engaged in subversive acts and had attempted to establish an Islamic state.