But with most of the alleged attacks now more than 10 years in the past, the 77-year-old Emmy and Golden Globe winner -- who has repeatedly denied any wrongdoing -- is unlikely to face criminal charges due to statutes of limitations.
The legendary star of "The Cosby Show" and "I Spy" was the subject of a civil suit brought in 2005 by 15 women, 13 of whom remained anonymous. A settlement was reached in 2006 with one of the plaintiffs.
The following is a look at the allegations against Cosby, and the legal parameters surrounding them:
The allegations against Cosby date back to the 1960s, and the encounters took place in several US states including New York, New Jersey, Pennsylvania and California.
In America, laws vary from state to state in terms of the gravity of the crime, the burden of proof, the length of the statutes of limitations and the possible punishments.
An act of rape "by forcible compulsion" in New York can be considered to be an "illegal sexual relationship" in California, and thus the punishment is less severe.
Most of the allegations go back to the 1970s and 1980s. In the states involved, the statutes of limitations bar charges from being brought, except in Pennsylvania, where an alleged victim has 12 years to file a criminal complaint.
In the East Coast state, Andrea Constand, former director of operations of the women's basketball team at Temple University, has accused Cosby of drugging and sexually assaulting her in 2004.
A year later, prosecutors decided not to file criminal charges against Cosby, citing lack of evidence. Constand then filed a civil complaint accusing Cosby of battery and assault.
With respect to possible civil suits against Cosby, the statutes of limitations have expired everywhere.
However in late 2006, the comic reached an out-of-court settlement with Constand, the terms of which were never disclosed.
Constand had asked for $150,000 in damages plus interest, arguing that Cosby's alleged behavior had caused her to "suffer severe emotional distress, humiliation, embarrassment and financial loss."
If Cosby made any admission in that civil case, that could work against him, attorney Stuart Slotnick told AFP.
But as this case involves a public figure, it is more likely that Cosby simply paid the money to avoid publicity.
In that event, the only thing prosecutors could use is that "he paid money to make a case go away, but that's not evidence he did it."
On the criminal side, no matter what the state, "the legal standard to prove the crime is actually the same everywhere, and that's 'beyond a reasonable doubt' and that's a very high standard," said Slotnick.
That is why the then district attorney in Pennsylvania's Montgomery County, Bruce Castor, chose not to charge Cosby in 2005.
"I didn't say that he didn't commit the crime," Castor told NBC Philadelphia.
"What I said was there was insufficient admissible and reliable evidence upon which to base a conviction beyond a reasonable doubt. That's 'prosecutors speak' for 'I think he did it, but there's just not enough here to prosecute.'"
"Criminal cases don't get better with age -- they usually get worse," said Slotnick. It is difficult, if not impossible, to gather physical evidence and testimony 10 or 20 years after the fact.
Even for the most recent case concerning Cosby, it is too late to gather any physical evidence that would prove rape or the presence of drugs in the woman's system.
"You lose the ability to test for blood or intoxicating agents, whether she was drugged," said Slotnick.
"Years later, the kind of evidence that prosecutors would be able to collect would be the women's testimony," the New York-based attorney said.
"That makes it difficult for a jury, especially when a woman comes forward 10 years later, because that automatically raises questions for the jurors: why did they wait? When there's someone who is a celebrity, there's always the question: do they seek to profit from the criminal prosecution?"
That is the main argument put forward by lawyers for Cosby. The comedian himself has largely stayed quiet, only saying to a Florida newspaper that "a guy doesn't have to answer to innuendos."