A major bottleneck in scaling-up COVID-19 testing is the need for sophisticated instruments and well-trained healthcare professionals, which are already overwhelmed due to the pandemic. Moreover, the high-sensitive SARS-CoV-2 diagnostics are contingent on an RNA extraction step, which, in turn, is restricted by constraints in the supply chain. Here, we present CASSPIT (Cas13 Assisted Saliva-based & Smartphone Integrated Testing), which will allow direct use of saliva samples without the need for an extra RNA extraction step for SARS-CoV-2 detection. CASSPIT utilizes CRISPR-Cas13a based SARS-CoV-2 RNA detection, and lateral-flow assay (LFA) readout of the test results. The sample preparation workflow includes an optimized chemical treatment and heat inactivation method, which, when applied to COVID-19 clinical samples, showed a 97% positive agreement with the RNA extraction method. With CASSPIT, LFA based visual limit of detection (LoD) for a given SARS-CoV-2 RNA spiked into the saliva samples was ~200 copies; image analysis-based quantification further improved the analytical sensitivity to ~100 copies. Upon validation of clinical sensitivity on RNA extraction-free saliva samples (n = 76), a 98% agreement between the lateral-flow readout and RT-qPCR data was found (Ct<35). To enable user-friendly test results with provision for data storage and online consultation, we subsequently integrated lateral-flow strips with a smartphone application. We believe CASSPIT will eliminate our reliance on RT-qPCR by providing comparable sensitivity and will be a step toward establishing nucleic acid-based point-of-care (POC) testing for COVID-19.