Fragments of the endoplasmic reticulum (ER) are selectively delivered to the lysosome (mammals) or vacuole (yeast) in response to starvation or the accumulation of misfolded proteins through an autophagic process known as ER-phagy. A screen of the Saccharomyces cerevisiae deletion library identified end3Δ as a candidate knockout strain that is defective in ER-phagy during starvation conditions, but not bulk autophagy. We find that loss of End3 and its stable binding partner Pan1, or inhibition of the Arp2/3 complex that is coupled by the End3-Pan1 complex to endocytic pits, blocks the association of the cortical ER autophagy receptor, Atg40, with the autophagosomal assembly scaffold protein Atg11. The membrane contact site module linking the rim of cortical ER sheets and endocytic pits, consisting of Scs2 or Scs22, Osh2 or Osh3, and Myo3 or Myo5, is also needed for ER-phagy. Both Atg40 and Scs2 are concentrated at the edges of ER sheets and can be cross-linked to each other. Our results are consistent with a model in which actin assembly at sites of contact between the cortical ER and endocytic pits contributes to ER sequestration into autophagosomes.