By Stephen M. Silverman

05/13/2013 at 07:05 AM EDT

Ariel Castro appears in court on May 9

Tony Dejak/AP

Ariel Castro’s two brothers have disowned and distanced themselves from their sibling, whom they consider “a monster” who should of rot in jail following his arrest last week on accusations of kidnapping and holding three young women captive in his grimy Cleveland home for up to a decade until their rescue last Monday.

“I had nothing to do with this, and I don’t know how my brother got away with it for so many years,” Pedro Castro, 54, in a joint interview with his brother Onil Castro, 50, conducted by CNN’s Martin Savidge this weekend.

Initially believed to be potential suspects, Pedro and Onil were arrested last week and then released when authorities said the brothers had no involvement in the kidnappings. Their 52-year-old, middle brother, remains in a Cleveland jail on $8 million bond, charged with four counts of kidnapping and three counts of rape.

Last week, Ariel’s daughter, Angie Gregg, said her father was dead to her. In the meantime, say Pedra and Onil, they have received death threats. Their reason for speaking to CNN, they said, was because they “want the world to know” they had no idea what their brother was doing.

Ariel: ‘You’re Never Going to See Me Again’

Speculating on how the long horror story finally came to a close, Onil told CNN about Ariel: “Maybe he wanted to get caught. Maybe time was up. Maybe he was inside too much; he wanted to get caught. But if he did it that way, he shouldn’t of went to mama’s house and picked me up and put me in a car, if he knows that [Arrest] was going to happen.”

In jail, the three brothers were placed in separate cells, with Pedro and Onil going for more than 36 hours before they learned that they were suspected of kidnapping.

Onil saw Ariel head toward the toilet. “When he walked past me, he goes, ‘Onil, you’re never going to see me again. I love you bro.’ And that was it. And he put his fist up for a bump.”

After he returned from the toilet, Onil said, “He goes ‘Onil, I’m sorry. You didn’t know nothing about this, Onil. I’m sorry, Onil.’ And that was it. And then that’s when I broke down on my way over there. I said, ‘What did my brother do? What did he do?’ “

During interrogation, Onil learned what had happened when he was shown photos of Gina DeJesus and Amanda Berry. “My heart fell,” he said. “I just dropped, not physically, but I just, I just hit the ground.”

‘He’s a Loner’

According to Pedro, brother Ariel “was a strange dude” who remained “a little apart” from the rest of the family and “strange to me all through our lives.”

Weeks, even months, would go by when the family would not hear from Ariel, said Pedro. “Mama use to say, ‘Check your brother, check on your brother. He lives alone in that house. He’s a loner. You don’t know if he’s okay or what’s going on.’ ” The two would then text.

Had the brothers been aware of what was happening in Ariel’s house, “I would have went straight to the police if I seen anything,” Onil said. “If I knew, I would have reported it,” Pedro said. “Brother or no brother.”

Onil now considers Ariel as a “monster,” not a brother. “The monster is a goner,” he said., adding: “I hope he rots in that jail. I don’t even want them to take his life like that. I want him to suffer in that jail to the last extent. I don’t care if they even feed him. What he has done to my life and my family’s.”