Cassiae semen are dried and ripe seeds of Cassia obtusifolia L. or Cassia tora L. (Fabaceae) and have been made into roasted tea or used as a traditional medicine in Asian countries. However, it was reported to result in liver and renal toxicity. The components of Cassiae semen that induce hepatotoxicity or nephrotoxicity remain unknown. In the present study, we evaluate the potential toxicity of 26 newly isolated compounds from Cassiae semen using quantitative structure-activity relationship (QSAR) methods and co-culture of hepatic and renal cell approaches, and we aim to illustrate the relationship between the structural characteristics and cytotoxicity by general linear models (GLMs). Both the QSAR models and co-culture of hepatic and renal cell systems predicted that 6 compounds were potentially hepatotoxic, 10 compounds were potentially nephrotoxic, and specific anthraquinones and anthraquinone-glucosides were potential toxicants in Cassiae semen. Specific groups such as -OH and -OCH3 at the R1, R2, R3, and R7 positions influenced the cytotoxicity.