LAUNCH OF BHUTAN PISA-D NATIONAL REPORT

29th March 2019

Contact: PISA-D National Project Centre (BCSEA)

The National Project Centre (Bhutan Council for School Examinations and Assessment) in collaboration with Ministry of Education (MoE) launched Bhutan PISA-D National Report on  29th March 2019 at Taj Tashi, Thimphu. The report was on the performance of 15-year-old students in Reading, Mathematical and Scientific Literacy.

The Organization for Economic Corporation and Development (OECD) launched the Programme for International Student Assessment (PISA) in 2000 as a triennial international survey. PISA focuses on the functional skills and knowledge that are essential for full participation in modern societies and assesses 15-year-olds because in most countries including Bhutan these students are near the end of compulsory education. PISA assesses students’ literacy in Reading, Mathematics and Science as they are foundational to a student’s ongoing education.

Similarly, in 2013, OECD launched PISA for Development (PISA-D) as one-off programme with an aim to encourage and facilitate PISA participation by interested and motivated low and middle-income countries. It also contributes to the monitoring and achievement of the Education Sustainable Development Goal (SDG4), which emphasises quality and equity of learning outcomes for children, young people and adults.

PISA-D Assessment Framework is built on the PISA Assessment Framework with few additional features to assessment, that are specifically targeted on lower levels of performance and contextual questionnaires that better reflect the situation of 15-year-olds in middle- and low-income countries. PISA-D has also established a baseline proficiency level 2 on 6 leveled scale as in PISA because at this level students begin to demonstrate the competencies that will enable them to participate effectively and productively in life as continuing students, workers and citizens.

Bhutan has always aspired to participate in international assessments to understand its quality of education. Hence, one of the major recommendations from the Bhutan Education Blueprint 2014-2024 is to partake in international benchmarking systems such as PISA, TIMMS, PIRLS etc. Subsequently, during the 18th National Education Conference held on 12th January, 2017, Bhutan signed the Memorandum of Understanding (MoU) with OECD for its participation in PISA-D to assess the cognitive ability (Strand A) of the students. Despite the actual registration year for the participation in PISA-D was 2015 and the fee being €600,000, OECD agreed upon €50,000 for Bhutan’s participation as it registered only in 2017. Hence, the core team of NPC-Bhutan missed the first two years of technical capacity building programmes offered by the OECD while the other PISA-D participating countries received the complete package.

The main objectives of Bhutan’s participation in PISA-D were to:

  • set a benchmark (baseline) profile of the knowledge, skills and competencies of the 15-year-old
  • students across the three domains;
  • collect evidence about the readiness of the Bhutanese education system for entry into the main PISA in 2021; and
  • to build local and institutional capacity in terms of the standards and structures to implement large scale educational assessments.

The NPC in collaboration with Education Testing Service (ETS, USA), MoE and Royal Education Council (REC) conducted PISA-D in Bhutan from 1st to 15th November 2017. A total of 2457 students from grade VII to XII covering 53 sample schools participated in the 2 hours test. Including Bhutan, 8 other middle- and low-income countries: Cambodia, Ecuador, Guatemala, Honduras, Panama, Paraguay, Senegal and Zambia partook in the assessment. The NPC also coordinated and conducted Data Management and Coding of the assessment materials from 21st December 2017 to 10th January 2018 involving 32 teachers from the PISA-D Sample Schools.

Later, OECD and NPC jointly analysed the assessment data and prepared Bhutan PISA-D National Report in consultation with MoE, REC and Royal University of Bhutan (RUB). 

The major findings were as follows:

  • Students performed slightly better in Reading and Scientific Literacy in comparison to Mathematical Literacy;
  • Girls outperformed boys in Reading Literacy and in Mathematical Literacy, boys performed better than girls while both boys and girls performed almost at par in Scientific Literacy;
  • Students performed better in items requiring lower cognitive skills, however, there is a significant gap in performance in more demanding tasks;
  • Bhutan has significantly higher average percentage of correct answers than the average of all PISA-D countries in all three domains;
  • The findings also demonstrate that the students have relatively broader knowledge and understanding  compared to other PISA-D countries;
  • Students performed at par with top PISA-D countries but significantly below OECD average and of the best education systems in Asia.
    • Reading Literacy percent correct: Ecuador: 52.66, Paraguay: 47.03, Bhutan: 45.34
    • Mathematical Literacy percent correct: Ecuador: 41.69, Bhutan: 38.84, Hondurus: 33.21
    • Scientific Literacy percent correct: Ecuador: 46.98, Bhutan: 45.10, Paraguay: 41.16
  • PISA test is appropriate to measure the performance of our students against established international benchmarks, in the domains of Reading, Mathematics and Science.

Challenges and achievements:

Due to Bhutan’s late registration in PISA-D, Bhutan encountered several challenges. Besides time constraint, there was limited technical capacity in conducting such international large scale assessment for which Bhutan had to coverup the technical capacity building workshops that other PISA-D countries availed for 2 years through OECD. This demanded a lot of effort from the officials of NPC requiring them to go through the documents of PISA and PISA-D to meet the technical standards as specified by the ETS and OECD. Simultaneously a series of communication had to be done with the international partners and PISA-D countries for better understanding of the test administrative processes and the procedures. One of the repercussions of the late registration was that Bhutan could not administer the field trail test which other PISA-D countries administered in 2016.

Despite all the challenges, Bhutan could successfully complete the project on time and it fulfilled the objective in finding out its education standard in comparison to the other PISA-D countries. In addition, the officials involved in the project were able to enhance their knowledge on large scale assessment.

Recommendations:

  • Ministry of Education / Schools / Royal Education Council /Royal University of Bhutan and Bhutan Council for School Examinations and Assessment should
    • strengthen and enhance competency based activities and assessment in curriculum, teachers’ knowledge and skills in competency based teaching and learning and teachers’ knowledge and skills in developing competency based items.
  • Schools / Royal Education Council / Bhutan Council for School Examinations and Assessment  should prioritize on the depth rather than the breadth of learning to avoid superficial learning, incomplete understanding of core concepts and limited ability to transfer and apply knowledge to unfamiliar contexts.
  • Ministry of Education should strengthen the monitoring of quality instructions through the existing school self-assessment tool and school improvement plan.
  • Bhutan Council for School Examinations and Assessment  should strengthen National Education Assessment.
  • Schools should enhance students’ reading and comprehension skills to improve their academic performance across the subjects.
  • Parents and teachers should break the gender stereotypes about Science / Mathematics / Reading related activities and occupations to allow children achieve their full potentials.
  • Bhutan can partake in any future PISA cycles as PISA test is found appropriate in obtaining full insight on the students’ knowledge and skills needed to succeed in future.

Please click on the links given below to view the document: