During the Ganesh Festival of 1773 (the actual date was 30 August 1773, the last day of Ganesh Festival i.e. Anant Chaturdashi), several Gardi guards, led by their captain, Sumer Singh Gardi, entered the palace and started creating a commotion. They then charged towards the sleeping Peshwa's chambers, killing a clerk on the way. They intended to release Raghunathrao. Raghunathrao and his wife Anandibai, who were opposed to Narayanrao, had promised the Gardis that they would mediate in their dispute with Narayanrao. The Gardis followed Narayanrao to his uncle's chamber and a menial pulled him while Sumer Singh Gardi cut him down. His corpse was cremated secretly by the river at midnight.
According to popular legend, Raghunathrao had sent a message to Sumer Singh Gardi to fetch Narayanrao using the Marathi word dharaa (धरा) or 'hold' (actual phrase in Marathi - "Narayanrao-ana dhara"). This message was intercepted by his wife Anandibai who changed a single letter to make it read as maaraa or 'kill' . The miscommunication led the Gardis to chase Narayanrao, who, upon hearing them coming, started running towards his uncles' residence screaming, "Kaka! Mala Vachva!!" ("Uncle! Save me!"). But nobody came to help him and he was killed in the presence of his uncle. Rumour has it that Narayanrao's body was hacked into so many pieces that they had to carry the pieces in a pot, henceforth it was taken near the river and cremated at midnight. This act brought ill fame to the Peshwa administration, which was being looked after by the minister Nana Phadnavis. The Chief Justice of the administration, Ram Shastri Prabhune was asked to conduct an investigation into the incident, and Raghunathrao, Anandibai and Sumer Singh Gardi were all prosecuted in absentia. Although Raghunathrao was acquitted, Anandibai was declared an offender and Sumer Singh Gardi the culprit. Sumer Singh Gardi died mysteriously in Patna, Bihar in 1775, and Anandibai performed Hindu rituals to absolve her sins.