Long noncoding RNA (lncRNA) PTTG3P has been demonstrated to participate in the development of hepatocellular carcinoma (HCC) by targeting the mRNA PTTG1. The present study aimed to investigate the diagnostic efficacy of serum lncRNA PTTG3P, mRNA PTTG1 and their combination for the diagnosis and prognosis of HCC. A total of 373 participants were enrolled in the present study, including 73 patients with HCC, 100 patients with chronic hepatitis B (CHB), 100 patients with liver cirrhosis (LC) and 100 healthy controls (HCs). The expression levels of serum RNAs were quantified by reverse transcription‑quantitative PCR. The association between serum lncRNA PTTG3P and clinical characteristics was further analyzed. Receiver operating characteristic (ROC) curve and area under curve (AUC) analyses were performed to estimate the diagnostic ability of serum lncRNA PTTG3P, PTTG1 and their combinations with other biomarkers for HCC. The results revealed that the expression levels of lncRNA PTTG3P and mRNA PTTG1 were markedly increased in the serum of patients with HCC and CHB compared with in the serum of HCs. Additionally, the postoperative levels of lncRNA PTTG3P and mRNA PTTG1 were significantly lower than the preoperative concentrations in 36 paired patients with HCC. Spearman's correlation coefficient analysis showed that serum lncRNA PTTG3P was correlated with aspartate transaminase (AST). ROC analysis showed that both lncRNA PTTG3P and mRNA PTTG1 had a significant predictive value for HCC. The AUC values of lncRNA PTTG3P and mRNA PTTG1 alone were 0.636 and 0.634, respectively. Furthermore, combining lncRNA PTTG3P, mRNA PTTG1, α‑fetoprotein (AFP), alanine aminotransferase (ALT), AST, γ‑glutamyl transpeptidase (GGT) and alkaline phosphatase (ALP) significantly increased the AUC value. The best performance was the combination of PTTG3P, PTTG1, AFP, ALT, AST, GGT and ALP with an AUC of 0.959, a sensitivity of 90.4% and a specificity of 98.0%. In conclusion, the combination of serum lncRNA PTTG3P, mRNA PTTG1 and AFP appeared to be a noninvasive biomarker with comparatively high specificity and sensitivity for the diagnosis of HCC.