One of his horns had been hacked off with a chainsaw, police said.
Zookeepers discovered his carcass Tuesday in the rhinoceros' enclosure.
The raid is said to be the first attack of its kind in Europe.
The world population of southern white rhino, once on the brink of extinction, has in recent times recovered to around 20,000.
Zoo director Thierry Duguet described the attack as "unbelievable" and "extremely shocking".
The poacher, or poachers, might have forced a door open at the back of the zoo before entering a building where the zoo's three rhinos lived.
The victim was Vince, a four-year-old male rhinoceros, and his second horn was partially torn off which indicates that the perpetrators may have been interrupted during the break in.
Two other rhinos in the enclosure ― 5-year-old Bruno and 37-year-old Grace ― were unharmed.
German Town Receives Bomb Threat After Cancelling Turkish Rally
Earlier in the day, Yıldırım said that the decision of German authorities is "very unfortunate". Police had to deal with a bomb alert in one of the places where a rally was cancelled.
"Vince was found this morning by his caretaker, who, being very attached to the animals she cares for, is deeply affected", the zoo wrote.
Despite bans by worldwide convention and French law, the illegal sale of rhino horns persists due to demand for use in traditional Asian medicine.
A police spokesperson told The Guardian that it was possible they had meant to kill all three animals and ran out of time.
A kilo of rhino horn fetched over £44,000 on the black market in 2015, according to Thoiry.
France outlawed the trade in ivory and horns previous year.
Thoiry Zoo, about 50 kilometers (30 miles) west of Paris, features a safari park that visitors can only access in a vehicle.
Soldiers from the Thoiry gendarmerie were today leading the criminal investigation, which is being led by an examining magistrate.