Selective estrogen receptor modulators (SERM) including tamoxifen are known to inhibit angiogenesis. However, the underlying mechanism, which is independent of their action on the estrogen receptor (ER), has remained largely unknown. In the present study, we found that tamoxifen and other SERM inhibited cholesterol trafficking in endothelial cells, causing a hyper-accumulation of cholesterol in late endosomes/lysosomes. Inhibition of cholesterol trafficking by tamoxifen was accompanied by abnormal subcellular distribution of vascular endothelial growth factor receptor-2 (VEGFR2) and inhibition of the terminal glycosylation of the receptor. Tamoxifen also caused perinuclear positioning of lysosomes, which in turn trapped the mammalian target of rapamycin (mTOR) in the perinuclear region of endothelial cells. Abnormal distribution of VEGFR2 and mTOR and inhibition of VEGFR2 and mTOR activities by tamoxifen were significantly reversed by addition of cholesterol-cyclodextrin complex to the culture media of endothelial cells. Moreover, high concentrations of tamoxifen inhibited endothelial and breast cancer cell proliferation in a cholesterol-dependent, but ER-independent, manner. Together, these results unraveled a previously unrecognized mechanism of angiogenesis inhibition by tamoxifen and other SERM, implicating cholesterol trafficking as an attractive therapeutic target for cancer treatment.