Research Status and Trends of Science and Technology Evaluation in China—Visual Analysis Based on Knowledge Mapping ()
1. Introduction
Scientific and technological evaluation is a necessary means of modern scientific and technological management and an important basis for scientific decision-making. Scientific, fair and effective scientific and technological evaluation is not only to promote the optimal allocation of scientific and technological resources, to improve the efficiency of resource utilization, to promote the healthy development of scientific and technological undertakings, but also to mobilize the majority of scientific The enthusiasm of workers is of great significance to establish healthy scientific research and academic atmosphere [1].
Science and technology evaluation emerged in the European continent in the 1940s and 1960s, and developed countries such as France, Germany, and Denmark earlier. In the 1980s and 1990s, Switzerland, Sweden, the United Kingdom, Spain and other countries have also developed. In Asia, science and technology evaluation also began in Japan in the 1940s, and the science and technology review agency was Japan’s early science and technology evaluation agency. The evaluation of science and technology in China started in the early 1990s. In 1993, the former National Science and Technology Commission began to introduce scientific and technological evaluation methods into the macroscopic management of science and technology. In 1997, the National Science and Technology Evaluation Center was established on the basis of the former National Science and Technology and Economic Development Research Center [2]. In April 2001, the “Science and Technology Evaluation Norms”, the code of conduct and technical specifications for China’s science and technology evaluation activities, was published publicly, marking that China’s science and technology evaluation activities have officially entered the stage of professionalization [3].
After years of exploration, my country has accumulated a lot of valuable experience in science and technology evaluation, and related scholars and scientific research managers have also carried out many studies on science and technology evaluation and achieved fruitful results. For example, the earliest paper on scientific and technological evaluation research in China was “Science and Technology Early Warning and Evaluation” published by XuLida in “Information Science” in 1980. In 2006, QiuJunping’s “Gold Medal Priority” Rule published in “Library, Information and Knowledge” should be applied to the thinking of scientific research talent evaluation”. It advocates that the best results of scientific research personnel should be used to represent and evaluate the academic level of scientific researchers, and to guide scientific researchers to more Pursue academic quality. In 2010, Yu Liping, Pan Yuntao and Wu Yishan published in “Soft Science” “A New Objective Empowerment Technology Evaluation Method”, which proposed the independent information data fluctuation weighting method, a new objective weighting technology evaluation method but at the same time, problems such as imperfect evaluation system, imperfect rating system, and irregular evaluation methods have also been exposed.
In response to the problems in scientific and technological evaluation, various policies have been issued at the national level. For example, in 1987, the National Science and Technology Commission issued the “Measures for the Appraisal of Scientific and Technological Achievements of the National Science and Technology Commission of the People’s Republic of China”, the Ministry of Science and Technology established the “National Science and Technology Evaluation Center” in 1997, and the “Science and Technology Progress Law of the People’s Republic of China” began on July 1, 2008. Implementation, in 2016, my country issued “Regulations on Technology Evaluation (Trial)” and other policies and regulations [4]. On June 29, 2020, the 14th meeting of the Committee for Comprehensively Deepening Reform of the CPC Central Committee reviewed and approved the “Overall Plan for Deepening Educational Evaluation Reform in the New Era”, which is a programmatic document guiding the deepening of education evaluation reform in the new era. The plan has made a series of new arrangements for breaking “Five Only” and implementing “four evaluations”, which is conducive to fundamentally stimulating the internal driving force of the connotative development of universities.
In view of the existing research results of domestic science and technology evaluation, we need to sort out the current situation and possible future development trends of domestic science and technology evaluation research in order to provide a useful reference for follow-up research. China Knowledge Network (CNKI) contains various journals, papers, patent yearbooks, etc., which are important data sources for literature research and are deeply loved by domestic scholars. However, the quality of the documents included in this database is uneven. Compared with other documents, the documents included in the Chinese Social Science Citation Index Database (CSSCI) have a high academic level and theoretical value. Therefore, this research uses the visual analysis tool CiteSpace to use 445 journal articles included in the source journals of the Chinese Social Science Citation Index from 2009 to 2019 as the data source, and uses literature publication trends, author co-occurrence maps, institutional co-occurrence maps, and research keywords, Research hotspots, research frontier trends, etc. Sort out the development context of China’s science and technology evaluation research field, construct a knowledge map of research progress, research institutions, research hotspots and research trends in this field, and use diversified knowledge maps for scientific and technological evaluation research and analysis. Follow-up research provides reference.
2. Data Sources and Research Methods
2.1. Data Source
The research data of the article comes from the CSSCI database. After many trial inspections, the final search condition is determined as: Keyword = “Science and Technology Evaluation” OR “Science and Technology Evaluation” OR “Scientific Research Evaluation” OR “Scientific Research Performance Evaluation”, and “Science and Technology Evaluation”, “Science and Technology Evaluation”, “Scientific Research Evaluation”, and “Scientific Research Performance Evaluation” are keywords. Since the database has not yet included papers in this field for a whole year, the time span = “1998-2019”, and the retrieval time is September 1, 2020. Matching mode = “exact”, 474 related documents were retrieved. Through systematic filtering of reviews, evaluations and reports, 447 papers were obtained. Through SATI deduplication and cleaning, 445 valid papers were finally obtained as research samples. Export the bibliographic information of research samples in batches and perform format conversion.
2.2. Research Methods
Currently, tools such as CiteSpace, Vosviewer, BibExcel, and Ucitnet are suitable for bibliometric visualization analysis. Among them, CiteSpace has multiple, time-sharing and dynamic functions due to its solid theoretical foundation, a large number of user groups, and rich analysis functions and updates. The advantages such as quickness are currently widely used in various fields. The main inspiration of CiteSpace software comes from the evolution of Kuhn’s scientific structure. The main point of view is that “the focus of scientific research changes over time, sometimes slow, and sometimes more intense”. Scientific development can be traced from the past through its footprint. Extracted from published literature. Based on CitSpace, this paper conducts author co-occurrence map, institution co-occurrence map, keyword co-occurrence map and mutation analysis to analyze the development trend and evolution context of my country’s science and technology evaluation research.
3. Research Results
3.1. Analysis of Literature Publication Trends
Through the changes in the number of publications in the scientific and technological evaluation research field over the years, the development of the research field can be obtained. It can be seen from Figure 1 that the number of papers published in this research field has been on the rise since 1998, and the growth rate has accelerated significantly since 2001. Since 2005, the number of papers published every year, except for a few years, is basically more than 50. This It shows that domestic science and technology evaluation research is a hot field in recent years, with great potential and strong influence. Generally speaking, my country’s science and technology evaluation research has experienced three stages of initiation, development and rapid development, which deserve the attention of scientific researchers in this field.
![]()
Figure 1. Chronological distribution of scientific and technological evaluation research literature.
3.2. Author Co-Occurrence Map Analysis
Co-authored papers are an important form of scientific research cooperation. Through the analysis of the knowledge graph of co-authors, we can understand the core author group of scientific and technological evaluation-related articles and the community structure and organization of scientific research results, and clarify the knowledge interaction of scientific research. Relationship [5]. Figure 2 is a co-occurrence map of authors in the field of science and technology evaluation produced by SATI. In the author’s co-occurrence map, the number of nodes (N) is 49, the number of connections (E) is 25, and the network density (Desity) is 0.0213. In terms of the total number of publications, the top authors in the local area are Yu Liping (25 articles), Wuyishan (15 articles), Pan Yuntao (11 articles), QiuJunping (8 articles), Nie Chao (5 articles), and Zhu Shaoqiang (4 articles) And so on, formed a cooperation network represented by Yu Liping, Wuyishan and Pan Yuntao, but there has not yet been a clear cooperative group of authors in this field in China, and most of them are single or two-person collaboration.
3.3. Institutional Co-Occurrence Map Analysis
Keep other parameters unchanged, select “institution” for the node type, and use the institution of the author of the publication as the node for visual analysis. By analyzing the institutions where the authors of the published documents in the field of domestic science and technology evaluation are located, the status quo of cooperation among institutions in this field in my country is analyzed. As shown in Figure 3, the organization’s co-occurrence map selects organizations with 2 or more articles, the number of nodes is 23, the number of connections is 9, and the network density is 0.0356. Most of the co-occurrence maps of research institutions belong to individual research and do not form obvious institutional centrality. Even if there is cooperation, it basically belongs to cooperation within the
![]()
Figure 2. Co-occurrence map of authors in the field of science and technology evaluation.
![]()
Figure 3. Co-occurrence map of institutions in the field of science and technology evaluation.
same institution, and the status quo of cross-part cooperation has not been realized.
The top five institutions in the field of domestic science and technology evaluation are the Chinese Institute of Science and Technology Information (26 articles), the Institute of Science and Technology Policy and Management Science of the Chinese Academy of Sciences (10 articles), the PLA Artillery Academy (8 articles), and the Chinese Academy of Sciences. Document Information Center (6 articles), Evaluation Research Center of Chinese Academy of Sciences (4 articles). In terms of the total number of published documents, the above-mentioned institutions have a greater advantage in the number of published articles, and represent research institutions in the field of scientific and technological evaluation in my country. However, since the top five institutions only publish 12.1% of the total amount of relevant documents, it shows that there are no research institutions in the field of science and technology evaluation that affect the development of the field, and the cooperation between institutions is still scattered, and it is necessary to strengthen each other. Cooperation.
3.4. Keyword Co-Occurrence Analysis
Keywords are the overall condensed content of the literature and a high degree of generalization of research topics. Keywords with high frequency of occurrence and high betweenness centrality can deeply reflect hot issues in a certain research field [6]. Other parameters remain unchanged, the node type is used as the keyword, and the path-tracking algorithm is selected to generate the keyword co-occurrence map as shown in Figure 4. The keyword co-occurrence map has 65 nodes, 63 connections, and the network density is 0.0303. The keywords scientific research evaluation, scientific and technological evaluation, and scientific and technological evaluation occupy a central position in the network.
![]()
Figure 4. Co-occurrence map of keywords in the field of science and technology evaluation.
Key words such as h-index, bibliometrics, scientific and technological management and peer review, which represent scientific and technological evaluation and specific research methods, appear relatively high, representing the domestic scientific and technological evaluation field. Research hotspots.
Betweenness centrality is a measure of the importance of nodes in a co-occurring network. Nodes with high betweenness centrality are usually the key hubs connecting different hotspots [7], which can be regarded as key points, turning points, and turning points in the research field. Trigger point [8]. In the keyword co-occurrence map, the keyword with the highest betweenness centrality is scientific research evaluation (0.45) (Table 1), followed by scientific and technological evaluation (0.42), scientific and technological evaluation (0.18), SCI (0.14), academic evaluation (0.13), Colleges and universities (0.10), regression analysis (0.08) and impact factor (0.06), bibliometrics (0.05), scientific paper index (0.05), the above 10 keywords have significant influence in the co-occurrence network and are connected to hot spots Important hub.
3.5. Distribution of Research Hot Topics
Keep other parameters unchanged, use the node type as the key word, select the path-based algorithm, and cluster the domestic science and technology evaluation keywords. Based on the network structure and clustering clarity, the Modularity Q (Q value) can be used to measure the stability of the generated clustering network. Generally, a Q value greater than 0.3 indicates that the clustering structure is significant and the effect is good; the average profile the value (Mean Silhouette, S value) can be used to measure the similarity of nodes within the cluster. It is generally believed that S value greater than 0.5 indicates a high degree of matching within the cluster and reasonable clustering [9]. The Q value of clustering result is 0.7007, which is greater than the critical value of 0.3; the S value is 0.9671, which is greater than the critical value of 0.5. In general, the
![]()
Table 1. Hot keywords in the research field of science and technology evaluation.
reliability of the clustering results is high.
From 1998 to 2019, in the 22 years of scientific and technological evaluation research published papers in my country, there were a total of 1784 keywords and a total frequency of 4617 words. High-frequency keywords were scientific research evaluation (188 times), science and technology evaluation (111 times), and h index. (14 times), bibliometrics (9 times), universities (7 times), academic journals (5 times), science and technology management (5 times), etc. In order to analyze the research topics in this field more intuitively, CiteSpace software was used to draw a keyword clustering map of the domestic science and technology evaluation research field. According to the keyword clustering map shown in Figure 5, we can know that in the past 22 years, the high-frequency keywords in this field in the Chinese Social Science Citation Index database are mainly concentrated in 3 clusters.
We extracted keywords with TO50 occurrence frequency, and got a total of 65 keywords that met the threshold. Then the co-occurrence analysis is performed on these 65 keywords, and the original co-word matrix of 65 × 65 is obtained. Further standardize the use of correlation strength of the original co-word matrix. Through the clustering of the common word network, keywords are divided into different groups according to the strength of the relationship.
The clustering results are shown in Table 2. My country’s science and technology evaluation research is mainly divided into three major themes: science and technology evaluation index system, evaluation method and evaluation
![]()
Figure 5. Keyword clustering map of domestic science and technology evaluation research fields.
![]()
Table 2. Keyword clustering and distribution of domestic science and technology evaluation research fields.
system.
1) Research on the scientific and technological evaluation index system.
This research topic includes cluster#0 S & assessment and cluster#1 academic journals, which contain 28 articles in total, and representative keywords include outcome evaluation, peer review, journal evaluation system, scientific research results, management evaluation, academic evaluation, and scientific and technological evaluation Wait. Sun Yao and others explored the growth trend of my country’s international scientific paper output and changes in international status since the reform and opening up, in order to evaluate the ability of existing scientific research institutions and provide an important reference for scientific research evaluation and scientific research decision-making [10]. DuanHongbo et al. proposed that the scientific research evaluation of universities should actively explore the mechanism of quality-oriented and service innovative talent training as the fundamental task, and strengthen the guidance of the timely transformation of scientific research results into teaching results in scientific research evaluation [11]. Ye Lan proposed that it is necessary to attach great importance to the strategic and policy levels, incorporate new measurement indicators and types of results into scientific research evaluation services, and build a scientific research evaluation service system oriented to the life cycle of research influence [12].
2) Research on scientific and technological evaluation methods.
This research topic includes cluster #3 journal citation report and cluster #6 regression analysis. It contains a total of 8 articles. Representative keywords include impact factor, journal citation report, citation frequency, regression analysis, and data envelopment analysis. Shi Wanbing et al. used literature analysis and key performance index (KPI) methods to illustrate the foothold and premise of university teachers’ scientific research performance evaluation methods [13]. Yu Liping, Pan Yuntao and Wuyishan conducted regression analysis on scientific evaluation indicators, and screened out a number of irrelevant and negative correlation indicators. They also measured the scientific rationality of nonlinear technology evaluation methods, and proposed and compared the two methods [14] [15].
3) Research on science and technology evaluation system.
This research topic includes cluster #2 scientific and technological evaluation, which contains a total of 7 documents. The representative keywords are evaluation system construction, promotion strategy, etc. Li Xingong proposed specific measures to gradually establish and improve the national science and technology assessment system from the definition of science and technology assessment scope, the construction of science and technology assessment institutions and the assessment methods of different scientific and technological achievements [16]. Zhao Rongying and others comprehensively combed and summarized the research development of science and technology evaluation in Chinese universities, proposed to expand the breadth and depth of science and technology evaluation research in universities, further improve the management system of science and technology evaluation in universities, and improve the science and technology evaluation system and methods of universities and other strategies [17].
4. Conclusions
With the help of CiteSpace software, the research analyzes the relevant documents of China’s science and technology evaluation from 1998 to 2019 included in the Chinese Social Science Citation Index database, and uses the age distribution of the documents, author co-occurrence maps, institutional co-occurrence maps, keyword co-occurrence maps and keyword aggregation, the class-map sorts out the development status of domestic science and technology evaluation research field. Although this field is developing rapidly in China, and the overall number of publications is on the rise, whether it is author cooperation or institutional cooperation, the breadth and depth of cooperation are not satisfactory. It is necessary to explore multi-disciplinary, multi-field and cross-sectoral cooperation mechanisms. At present, the research hotspots in the field of domestic science and technology evaluation mainly focus on science and technology evaluation and its specific evaluation method h index, bibliometrics, science and technology management and peer review. The themes mainly focus on the three themes of science and technology evaluation index system, evaluation method and evaluation system.
It should be noted that, out of consideration of the representativeness and quality of the data source, the research in this paper sets the data source as the Chinese Social Science Citation Index. Although the sample is satisfied in terms of strong representativeness and high quality, it is filtered A large part of the papers published in the CSSCI journal expansion board and general journals cannot fully present the full picture of my country’s scientific and technological evaluation research. But in general, this article analyzes the current research status of the domestic science and technology evaluation field through the method of document visualization, and provides a useful reference for subsequent research.