Assessing the sustainable development of a national research ecosystem: A generative AI-based evaluation of empirical educational research in China (2004-2023)

评估国家科研生态系统的可持续发展:基于生成式人工智能的中国教育实证研究评估(2004-2023)

阅读:1

Abstract

A nation's progress toward Sustainable Development Goal 4 (Quality Education) depends in part on the long-term health of its educational research system, yet systematic, longitudinal assessments of such research ecosystems remain scarce. This study applies a generative artificial intelligence-based framework to evaluate the sustainable development of China's empirical educational research ecosystem from 2004 to 2023. We compiled a dataset of 2,145 empirical studies published in leading Chinese education journals and used GPT-4o to score each paper on 31 quality indicators covering research problem, theoretical framing, design, data collection, analysis, and reporting, using a 1-10 analytic rating scale. Based on the resulting score distributions, we constructed a fuzzy relation matrix and applied a fuzzy comprehensive evaluation method to derive annual and overall sustainability indices, while the Criteria Importance Through Intercriteria Correlation (CRITIC) method was used to determine objective indicator weights. The overall sustainability index of China's empirical educational research ecosystem over the 20-year period is 75.77 on a 100-point scale, with membership degrees concentrated at quality levels 7 (0.328) and 8 (0.435), indicating a generally robust and maturing system. Longitudinal trends reveal three stages of evolution-fluctuating development, rapid growth, and continuous improvement-corresponding to a shift toward more stable high-quality output. At the micro level, the ecosystem shows strong responsiveness to real-world educational problems, with high average scores for the relevance (8.45) and social significance (8.23) of research questions, as well as generally solid research design and data analysis practices. However, relatively lower scores for transparency of data analysis (7.08) and accessibility of raw data (6.46) highlight persistent challenges for reproducibility, open science, and methodological innovation. We conclude that China's empirical educational research ecosystem has reached a relatively high and stable level of performance but faces critical tasks in strengthening data openness, methodological renewal, and AI-augmented governance. The proposed generative AI-based evaluation framework may offer a scalable tool for continuous monitoring and governance of national research ecosystems, while its results should be interpreted as an auxiliary input rather than a substitute for expert peer assessment.

特别声明

1、本页面内容包含部分的内容是基于公开信息的合理引用;引用内容仅为补充信息,不代表本站立场。

2、若认为本页面引用内容涉及侵权,请及时与本站联系,我们将第一时间处理。

3、其他媒体/个人如需使用本页面原创内容,需注明“来源:[生知库]”并获得授权;使用引用内容的,需自行联系原作者获得许可。

4、投稿及合作请联系:info@biocloudy.com。