Human Rights Watch (HRW) says there is "overwhelming evidence" of torture ordered by George W Bush.
The former president has defended some of the techniques, saying they prevented attacks and saved lives.
The Obama administration has launched inquiries into deaths in CIA custody and other "unauthorised actions".
But HRW argues these inquiries will not cover the activities which were specifically authorised as legal by officials within the Bush administration.
The former president, vice-president, defence secretary and head of the CIA should all be investigated, the group says.
"There are solid grounds to investigate [George] Bush, [former vice-president Dick] Cheney, [former defence secretary Donald] Rumsfeld, and [former CIA director George] Tenet for authorising torture and war crimes," said Kenneth Roth, Human Rights Watch's executive director.
"President Obama has treated torture as an unfortunate policy choice rather than a crime.
"His decision to end abusive interrogation practices will remain easily reversible unless the legal prohibition against torture is clearly re-established."
