Patterning of DNA methylation in eukaryotic genomes is controlled by de novo methylation, maintenance mechanisms and demethylation pathways. In Arabidopsis thaliana, DNA demethylation enzymes are clearly important for shaping methylation patterns, but how they are regulated is poorly understood. Here we show that the targeting of histone H3 lysine four trimethylation (H3K4me3) with the catalytic domain of the SDG2 histone methyltransferase potently erased DNA methylation and gene silencing at FWA and also erased CG DNA methylation in many other regions of the Arabidopsis genome. This methylation erasure was completely blocked in the ros1 dml2 dml3 triple mutant lacking DNA demethylation enzymes, showing that H3K4me3 promotes the active removal of DNA methylation. Conversely, we found that the targeted removal of H3K4me3 increased the efficiency of targeted DNA methylation. These results highlight H3K4me3 as a potent anti-DNA methylation mark and also pave the way for development of more powerful epigenome engineering tools.