Assessing and developing undergraduate students’ critical thinking skills: a longitudinal study using the Watson-Glaser critical thinking appraisal

УДК 378.025.7:159.9

Abstract

This study assesses undergraduate students’ critical thinking skills using the Watson-Glaser Critical Thinking Appraisal (WGCTA), evaluates the need for targeted skill development, and examines whether current instructional design and implementation effectively support the development of these skills. The study involved 706 undergraduate students enrolled in core and free elective courses at the National University of Mongolia. The intervention and assessments were conducted over four academic terms spanning the 2024–2025 and 2025–2026 academic years. The findings indicate uneven progress across the different sub-skills of critical thinking; specifically, several sub-skills showed no statistically significant growth, highlighting areas that require targeted attention in future instructional design. Although overall critical thinking skills showed marginal improvement, further research is needed to identify the factors underlying these outcomes and to select, test, and implement effective pedagogical strategies for enhancing students’ proficiency.

Аннотация

В данном исследовании оценивается уровень развития критического мышления студентов бакалавриата с использованием теста критического мышления Уотсона-Глейзера (Watson-Glaser Critical Thinking Appraisal — WGCTA), определяеся необходимость целевого развития данных навыков, а также анализируется эффективность текущего проектирования и реализации учебного процесса. В исследовании приняли участие 706 студентов бакалавриата, обучающихся по базовым профессиональным и элективным дисциплинам в Монгольском государственном университете. Обучающий эксперимент и оценка результатов проводились в течение четырёх семестров 2024–2025 и 2025–2026 учебных годов. Эмпирические данные свидетельствуют о неравномерной динамике развития отдельных субнавыков критического мышления. В частности, по ряду субнавыков не было обнаружено статистически значимого роста, что указывает на аспекты, требующие особого внимания при последующей разработке учебных программ. Несмотря на то что общие показатели критического мышления продемонстрировали незначительный положительный прирост, необходимы дальнейшие исследования для выявления факторов, обусловивших данный результат, а также для отбора, апробации и внедрения оптимальных педагогических стратегий, направленных на повышение уровня компетенций студентов.

Keywords: 21st century skills; Watson-Glaser Critical Thinking Appraisal; critical thinking sub-skills.

Ключевые слова: Компетенции XXI века; Тест критического мышления Уотсона-Глейзера; Субнавыки критического мышления.

Introduction

In an era of rapid technological change, ongoing transformations across sectors have rendered certain traditional occupations obsolete while giving rise to new professional domains. As a result, workplace demands are evolving rapidly, requiring professionals to possess a diverse and adaptable set of competencies. To remain competitive in this changing environment, individuals must engage in continuous professional development and lifelong learning within their respective fields.

Regardless of their specific job functions, professionals in contemporary workplaces increasingly need to comprehend complex issues accurately, communicate explanations effectively, evaluate evidence, formulate hypotheses, draw logically sound conclusions, and make informed decisions. These competencies are integral components of critical thinking, which is now widely recognized as an essential attribute of 21st-century citizenship.

International organizations, including UNESCO, the European Union, the Organisation for Economic Co-operation and Development (OECD), and the World Economic Forum (WEF), together with independent multidisciplinary research teams, have developed comprehensive frameworks for 21st-century global competencies, resulting in more than ten major models. Across these frameworks, critical thinking is consistently recognized as a foundational competency for effectiveness in both the workplace and daily life. Building on these international frameworks, several countries have developed their own national competency standards. In Mongolia, a research team developed the national competency framework known as the “Toonon Model of 21st Century Mongolian Learner Competencies,” which explicitly includes critical thinking as one of its domains.

Technological progress has enabled instantaneous, borderless communication and information exchange. However, the rapid increase in information flow also creates significant challenges, making it necessary to verify the accuracy, reliability, and evidential basis of information before using it. Consequently, developing critical thinking skills is essential for acquiring new knowledge, revising existing mental models, and making effective decisions. Such skills enable individuals to evaluate claims independently rather than passively accept others’ opinions. Effective decision-making also requires careful reflection and consideration of multiple perspectives to minimize errors. Critical thinking is therefore concerned not simply with reaching decisions, but with exercising sound and justifiable judgment grounded in reasoned belief.

Research Aims and Questions

The primary objective of this study is to assess undergraduate students’ critical thinking skills and evaluate the need for targeted skill development. Accordingly, the study is guided by two research questions:

1. Do undergraduate students currently demonstrate an adequate level of critical thinking skills?

2. Can students’ critical thinking skills be measurably enhanced through an active-learning approach and course design within a single academic course?

Literature Review

Critical thinking encompasses the systematic processes of formulating relevant questions, gathering and classifying pertinent data, integrating new information with existing knowledge, examining underlying assumptions, applying formal logic to establish causal relationships, and reaching objective conclusions. Its core components include analysis, recognition, hypothesis generation, explanation, evaluation, and metacognitive self-regulation [1, p. 242].

Various international organizations, research teams, individual researchers, and educational policymakers have conceptualized critical thinking in different ways, as reflected in international and national educational frameworks.

UNESCO emphasizes critical thinking within the “Learning to Be” pillar of education, stating that individuals should be enabled to develop independent and critical thought and form their own judgments (Delors, et al., 1996, p. 94). Similarly, the World Economic Forum (WEF) defines critical thinking within its 21st-Century Skills framework as the ability to identify, analyze, and evaluate situations, ideas, and information in order to formulate responses and solutions [9, p. 23].

The Partnership for 21st Century Skills (P21) identifies critical thinking, problem-solving, communication, and collaboration as essential skills for success in contemporary society [16, p. 2]. It combines critical thinking and problem-solving and describes them in terms of effective reasoning, systems thinking, judgment and decision-making, and problem-solving [16, p. 4]. The OECD Global Competency Framework defines critical thinking as evaluating the worth, validity, and reliability of material on the basis of its internal consistency and its consistency with evidence and with one’s own knowledge and experience [14, p. 9]. The framework also considers analytical and critical thinking to be intrinsically linked and argues that they should be applied together when analyzing complex intercultural and global issues.

The Council of Europe’s Key Competences Review similarly describes analytical and critical thinking as complementary processes involving the systematic and logical analysis, evaluation, and judgment of different types of material, including texts, arguments, interpretations, issues, events, and experiences [15, p. 63]. A European collection of critical-thinking skills and dispositions required in different professional fields describes critical thinking as purposeful, self-regulatory judgment involving interpretation, analysis, evaluation, inference, and explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations on which that judgment is based (Commission, 2018, p. 9) [14].

In Critical Thinking Competency Standards, Paul and Elder describe critical thinking as the process of analyzing and assessing thinking with the aim of improving it. Their approach emphasizes knowledge of the basic structures of thought and universal intellectual standards. They further argue that the creative aspect of critical thinking involves restructuring thought as a result of analyzing and effectively assessing it [13, p. 7].

The academic literature also offers several complementary definitions. Ennis [6] describes critical thinking as reasonable, reflective thinking focused on deciding what to believe or do. Facione [7, p. 27] emphasizes purposeful, self-regulatory judgment involving interpretation, analysis, evaluation, and inference. Halpern [10, p. 70] defines it in terms of cognitive skills or strategies that increase the likelihood of a desirable outcome. Vardi [17, p. 1] describes critical thinking as a process of reasoning aimed at reaching a sound and justifiable decision, conclusion, or judgment. Other definitions emphasize the objective analysis and evaluation of information while mitigating bias [2, pp. 5–6], as well as the disciplined evaluation of arguments or propositions to guide belief and action [18, p. 9]. Dewey [4] characterized reflective thought as the active, persistent, and careful consideration of beliefs or purported knowledge in light of the grounds that support them. He further argued that scientific thinking serves as an essential instrument for resolving indeterminate problems.

Higher-Order Thinking and Critical Thinking Sub-Skills

Critical thinking is widely classified as a higher-order cognitive process that encompasses complex operations such as analysis, synthesis, evaluation, interpretation, and metacognition. Researchers have proposed various frameworks for categorizing the sub-skills that underline critical thinking:

Krulik and Rudnick [11] propose eight sub-skills, including the comprehensive examination of situations, focused attention on specific elements, data organization, evidence analysis, and reasoned conclusion-drawing. Ennis [6] outlines five components: problem identification, argument presentation, inductive and deductive reasoning, evaluation, and implementation. Facione [7; 8] identifies six core cognitive skills: interpretation, analysis, evaluation, inference, explanation, and self-regulation. Paul and Elder structure critical thinking around eight elements: purpose, question at issue, information, concepts, assumptions, conclusions, point of view, and implications [13].

Taken together, these models suggest that critical thinking is not simply a form of negative or destructive criticism; rather, it is a structured, multi-phase cognitive process oriented toward problem identification, evidence analysis, and reasoned resolution.

Methods

Assessment Instrument

This study employed the Watson-Glaser Critical Thinking Appraisal (WGCTA) [18] to measure longitudinal gains in students’ critical thinking skills. The instrument assesses five distinct sub-skills: (1) Arguments, (2) Assumptions, (3) Deduction, (4) Inferences, and (5) Interpreting Information. It comprises 86 items distributed across 25 scenarios, with a standard administration time of 150 minutes.

Participants and Course Design

The study was conducted over four consecutive academic terms (Autumn and Spring) spanning the 2024–2025 and 2025–2026 academic years. A total of 706 undergraduate students (Years I–V) enrolled in the core and free elective courses at the National University of Mongolia participated in the study.

The course framework incorporated five major assessment tasks across three types of assignments:

Two individual assignments: Reflections on research design, literature selection, methodology, and self-evaluation, submitted in Weeks 6 and 12.

Two team projects: Collaborative empirical research projects involving topic selection, data collection, presentation of analyses, and peer critique.

One synthesis essay: Submitted in Week 15, requiring a comprehensive self-assessment of students’ growth in knowledge, skills, and attitudes.

Data Collection Procedure

A pre-test–post-test design was implemented, with assessments administered in Weeks 1 and 16 of each autumn semester, respectively.

Results and Discussion

To evaluate the impact of active-learning strategies on the development of critical thinking, pre-test and post-test scores from the Autumn 2024–2025 term were analyzed.

Of the 148 students who completed the pre-test assessment, 105 completed the post-test assessment. After invalid and incomplete responses were excluded, the matched sample was used to calculate mean changes and percentage gains in the overall score and in each sub-skill domain.

Table 1.

Pre-test and post-test comparison of critical thinking sub-skills, Autumn 2024 – 2025

The application of active-learning methods produced a statistically significant increase of 0.448 points in the fifth critical thinking sub-skill. However, the corresponding effect size was minimal (Cohen’s d < 0.2). Although the overall critical thinking score increased by 1.0 point, from 40.6 to 41.6, the difference between the pre-test and post-test means was not statistically significant. Accordingly, it cannot be concluded that overall critical thinking skills improved meaningfully during this period.

Based on student feedback collected at the end of the semester, the course’s instructional design and organization were subsequently revised. Assessment of students enrolled in the same course during the following spring term revealed a marginal increase in the mean score, from 41.33 to 41.70, relative to the preceding term. This small improvement may be attributable to curriculum adjustments informed by student feedback.

To evaluate the impact of these pedagogical refinements, pre-test and post-test assessments were again administered during the Autumn 2025–2026 term to students enrolled in the course incorporating active-learning methods. Of the 96 students who completed the baseline assessment, 65 completed the final post-test. After invalid and incomplete responses were excluded, the final sample was used to calculate mean differences.

Table 2.

Pre-test and post-test comparison of critical thinking sub-skills, Autumn 2025–2026

The integration of critical thinking methods into the course produced statistically significant improvements across all individual sub-skills, although the corresponding effect sizes remained small (Cohen’s d < 0.2). Furthermore, the overall critical thinking score increased by 6.1 points, from 44.6 at baseline to 50.7 at post-test, and the difference between the pre-test and post-test means was statistically significant.

Longitudinal changes in post-test scores across the four consecutive academic terms (Autumn and Spring, 2024–2026) are presented in Figure 1. The sample comprised 356 students enrolled during the 2024–2025 academic year and 350 students enrolled during the 2025–2026 academic year, for a total of 706 participants.

Following the initial implementation of the active-learning approach, post-test scores showed a gradual upward trend, culminating in a sharp increase during the third term. However, a decline was observed in the fourth term. The decline in students’ critical thinking performance during the Spring 2025–2026 term may have been influenced by external factors such as course selection, academic workload, and seasonal effects. Further investigation is needed to identify and examine the factors that may have contributed to this decline

Figure 2 compares mean post-test critical thinking scores across all four consecutive terms (Autumn and Spring, 2024–2026), using data from the full cohort of 706 students described above.

As illustrated in Figure 2, the third sub-skill (deduction) and the fifth sub-skill (interpreting information) had the highest scores, whereas the second sub-skill had the lowest score. Because these are post-test scores, the observed pattern may partly reflect students’ experiences with the active-learning methods used in the course. The findings suggest that, although students performed relatively well on the third and fifth sub-skills, greater instructional emphasis may be needed for the second sub-skill. Further empirical research is warranted to characterize the trajectory of critical thinking development more precisely and to determine the specific effects of individual instructional methods on each critical thinking sub-skill.

Conclusion

This study indicates that undergraduate students’ critical thinking skills remain insufficient and require systematic development. Because critical thinking develops over an extended period, an intervention implemented within a single course may yield only modest measurable gains. However, consistently embedding critical-thinking development across the curriculum may produce more substantial gains by the completion of an academic program.

Because critical thinking is an important competency in daily life, regardless of professional field or discipline, student-centered learning approaches that foster critical thinking should be integrated across courses. Attention should also be given to instructors’ own critical thinking skills, as faculty competence plays an important role in fostering these skills among students.

References