Development and Validation of Assessment Tool Measuring Knowledge on Sustainable Development - Basis for Instructional Emphasis on Higher Education
DOI:
https://doi.org/10.54536/jtel.v1i3.2127Keywords:
Sustainable Development Goals, Assessment Tool, Multiple-Choice Questions Development, Knowledge-Based Assessment, Prior LearningAbstract
The Sustainable Development Goals are universal and intrinsically global goals for all people. Thus, this study aimed to develop and validate an assessment tool measuring knowledge on sustainable development goals, and it seeks to evaluate the value of multiple-choice questions to build a strong question bank for future usage by validating the questions through item analysis and validation. The study follows a descriptive-quantitative approach utilizing two hundred (200) researcher-made questions validated by experts. Three-hundred twenty-three college students were selected using stratified sampling and participated in the study. The discrimination index, difficulty index, reliability index, and content validity are used to identify the appropriateness of items. Results of the study revealed that out of two hundred (200) items, one hundred twenty-five (125) items were retained, eight (8 ) items were rejected, and sixty-seven (67) were subject to revision following the reliability test. Thus, results showed a reliability index of 97.04% or .97, indicating that the items are standardized and suitable for assessment. After conducting the reliability test, the one hundred twenty-five (125) retained items underwent a content validity test through Lawshe’s method to enhance the construct validity of the developed instrument. It was determined that seventy-two (72) items were to be retained, for it had reached the acceptable level of 0.99. Hence, fifty-three (53) items were to be removed for it has CVR lower than 0.99, which is not acceptable for an item to be retained. Following the result, it is recommended that the seventy-two items (72) retained to be used to assess students’ knowledge of the 17 sustainable development goals. This study also inferred that future assessments should include questions with a medium degree of difficulty and a high ability to distinguish between different functions.
References
Ahamad, N. R., & Ariffin, M. (2018). Assessment of knowledge, attitude and practice towards sustainable consumption among university students in Selangor, Malaysia. Sustainable Production and Consumption, 16, 88–98. https://doi.org/10.1016/j.spc.2018.06.006
Aiken, L. R. (1979). Relationships Between the Item Difficulty and Discrimination Indexes. Educational and Psychological Measurement, 39(4), 821–824. https://doi.org/10.1177/001316447903900415
Alm, K., Melén, M., & Aggestam-Pontoppidan, C. (2021). Advancing SDG competencies in higher education: exploring an interdisciplinary pedagogical approach. International Journal of Sustainability in Higher Education, 22(6), 1450–1466. https://doi.org/10.1108/IJSHE-10-2020-0417
Alvarez-Risco, A., Del-Aguila-Arcentales, S., Rosen, M. A., García-Ibarra, V., Maycotte-Felkel, S., & Martínez-Toro, G. M. (2021). Expectations and Interests of University Students in COVID-19 Times about Sustainable Development Goals: Evidence from Colombia, Ecuador, Mexico, and Peru. Sustainability, 13(6), 3306. https://doi.org/10.3390/su13063306
Ando, Y., Baars, R., & Asari, M. (2019). Questionnaire survey on consciousness and behavior of students to achieve SDGs in Kyoto University. J. Environ. Saf.
Barak, M., Watted, A., & Haick, H. (2020). Establishing the validity and reliability of a modified tool for assessing innovative thinking of engineering students. Assessment & Evaluation in Higher Education, 45(2), 212–223. https://doi.org/10.1080/02602938.2019.1620680
Borges, F. (2019). Knowledge, Attitudes and Behaviours Concerning Sustainable Development: A Study among Prospective Elementary Teachers. Higher Education Studies, 9(2), 22. https://doi.org/10.5539/hes.v9n2p22
Burton, R. F. (2001a). Do Item-discrimination Indices Really Help Us to Improve Our Tests? Assessment & Evaluation in Higher Education, 26(3), 213–220. https://doi.org/10.1080/02602930120052378
Burton, R. F. (2001b). Do Item-discrimination Indices Really Help Us to Improve Our Tests? Assessment & Evaluation in Higher Education, 26(3), 213–220. https://doi.org/10.1080/02602930120052378
Chu, H.-E., & Treagust, D. F. (2014). Secondary Students’ Stable and Unstable Optics Conceptions Using Contextualized Questions. Journal of Science Education and Technology, 23(2), 238–251. https://doi.org/10.1007/s10956-013-9472-6
Clayson, D. E. (2018). Student evaluation of teaching and matters of reliability. Assessment & Evaluation in Higher Education, 43(4), 666–681. https://doi.org/10.1080/02602938.2017.1393495
Considine, J., Botti, M., & Thomas, S. (2005). Design, format, validity and reliability of multiple choice questions for use in nursing research and education. Collegian, 12(1), 19–24. https://doi.org/10.1016/S1322-7696(08)60478-3
Cottafava, D., Cavaglià, G., & Corazza, L. (2019). Education of sustainable development goals through students’ active engagement. Sustainability Accounting, Management and Policy Journal, 10(3), 521–544. https://doi.org/10.1108/SAMPJ-05-2018-0152
D’Sa, J. L., & Visbal-Dionaldo, M. L. (2017). Analysis of Multiple Choice Questions: Item Difficulty, Discrimination Index and Distractor Efficiency. International Journal of Nursing Education, 9(3), 109. https://doi.org/10.5958/0974-9357.2017.00079.4
Ejechi, V. (2018). Awareness And Perception of Sustainable Development Goals (SDGs), among, Library Personnel in Edo state University Library.
Förtsch, C., Werner, S., von Kotzebue, L., & Neuhaus, B. J. (2018). Effects of high-complexity and high-cognitive-level instructional tasks in biology lessons on students’ factual and conceptual knowledge. Research in Science & Technological Education, 36(3), 353–374. https://doi.org/10.1080/02635143.2017.1394286
Khazaee-Pool, M., Naghibi, S. A., Pashaei, T., & Ponnet, K. (2022). Designing and Evaluating Validity and Reliability of the Questionnaire Concerning the Factors Affecting Person’s Intention of COVID-19 Prevention (FAPI-COP). Iranian Journal of Health Sciences. https://doi.org/10.18502/jhs.v10i3.10517
Krathwohl, D. R., & Anderson, L. W. (2010). Merlin C. Wittrock and the Revision of Bloom’s Taxonomy. Educational Psychologist, 45(1), 64–65. https://doi.org/10.1080/00461520903433562
Leite, S. (2022). Using the SDGs for global citizenship education: definitions, challenges, and opportunities. Globalisation, Societies and Education, 20(3), 401–413. https://doi.org/10.1080/14767724.2021.1882957
Leiva-Brondo, M., Lajara-Camilleri, N., Vidal-Meló, A., Atarés, A., & Lull, C. (2022). Spanish University Students’ Awareness and Perception of Sustainable Development Goals and Sustainability Literacy. Sustainability, 14(8), 4552. https://doi.org/10.3390/su14084552
Malone, A. M., Gupta, R. S., Lyttle, C. S., & Weiss, K. B. (2008). Characterizing Community-Based Asthma Knowledge in Chicago and Its High Risk Neighborhoods. Journal of Asthma, 45(4), 313–318. https://doi.org/10.1080/02770900801911202
Martin, M. O., Mulles, I. V. S., & Hooper, M. (Eds). (2016). Methods and Procedures in TIMSS Advanced 2015. Boston College, TIMSS & PIRLS International Study Center Website: Http://Timss.Bc.Edu/Publications/Timss/2015-a-Methods.Html.
Murthy, M. N. (1967). Sampling Theory and Methods. Calcutta-35: Statistical Publishing Society, 204/1, Barrackpore Trunk Road, India.
Nguyen, T. D., Shih, M.-H., Srivastava, D., Tirthapura, S., & Xu, B. (2021). Stratified random sampling from streaming and stored data. Distributed and Parallel Databases, 39(3), 665–710. https://doi.org/10.1007/s10619-020-07315-w
Nhamo, G., & Mjimba, V. (2020). The Context: SDGs and Institutions of Higher Education (pp. 1–13). https://doi.org/10.1007/978-3-030-26157-3_1
Omisore, A. G., Babarinde, G. M., Bakare, D. P., & Asekun-Olarinmoye, E. O. (2017). Awareness and knowledge of the sustainable development goals in a University Community in Southwestern Nigeria. Ethiopian Journal of Health Sciences, 27(6), 669. https://doi.org/10.4314/ejhs.v27i6.12
Organization for Economic Cooperation and Development. (2018). PISA 2018 Results Results Combined Executive Summaries Volume I, II & III. Organization for Economic Cooperation and Development.
Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1301013. https://doi.org/10.1080/2331186X.2017.1301013
Roggenbuck, J. W., Loomis, R. J., & Dagostino, J. (1990). The Learning Benefits of Leisure. Journal of Leisure Research, 22(2), 112–124. https://doi.org/10.1080/00222216.1990.11969819
Sadaf, S., Khan, S., & Ali, S. (2012). Tips for Developing a Valid and Reliable Bank of Multiple Choice Questions (MCQs). Education for Health, 25(3), 195. https://doi.org/10.4103/1357-6283.109786
Schreiber, B., & Torabian, J. (2023). Student Development in Higher Education in the Era of the Agenda 2030 and the UN’s Sustainable Development Goals: Towards SDGs 2.0. Journal of College and Character, 24(1), 61–68. https://doi.org/10.1080/2194587X.2022.2157442
Smith, M. D. (2018). New Multiple-Choice Measures of Historical Thinking: An Investigation of Cognitive Validity. Theory & Research in Social Education, 46(1), 1–34. https://doi.org/10.1080/00933104.2017.1351412
Stiller, J., Hartmann, S., Mathesius, S., Straube, P., Tiemann, R., Nordmeier, V., Krüger, D., & Upmeier zu Belzen, A. (2016). Assessing scientific reasoning: a comprehensive evaluation of item features that affect item difficulty. Assessment & Evaluation in Higher Education, 41(5), 721–732. https://doi.org/10.1080/02602938.2016.1164830
United Nations. (n.d.). Raising Awareness. LOCAL 2030: Local 2030: Localizing The Sdgs Retrieved from Https://Www.Local2030.Org/Library/Tools/Raising-Awareness.
United Nations. (2015). The 17 Goals. United Nations: Department of Economic and Social Affairs.
Wilson, F. R., Pan, W., & Schumsky, D. A. (2012). Recalculation of the Critical Values for Lawshe’s Content Validity Ratio. Measurement and Evaluation in Counseling and Development, 45(3), 197–210. https://doi.org/10.1177/0748175612440286
Wolfe, E. W., & Manalo, J. R. (2004). An Investigation of the Impact of Composition Medium on the Quality of Toefl Writing Scores. Ets ETS Research Report Series, 2004(2), i–58. https://doi.org/10.1002/j.2333-8504.2004.tb01956.x
Yuan, X., Yu, L., & Wu, H. (2021). Awareness of Sustainable Development Goals among Students from a Chinese Senior High School. Education Sciences, 11(9), 458. https://doi.org/10.3390/educsci11090458
Zamora-Polo, F., Sánchez-Martín, J., Corrales-Serrano, M., & Espejo-Antúnez, L. (2019). What Do University Students Know about Sustainable Development Goals? A Realistic Approach to the Reception of this UN Program Amongst the Youth Population. Sustainability, 11(13), 3533. https://doi.org/10.3390/su11133533
Downloads
Published
Issue
Section
License
Copyright (c) 2023 Erica L. Villamor, Eloisa M. Gamayot, Nicagiana R. Baldon, Tomas Jr A. Diquito, Conrado Jr Panerio
This work is licensed under a Creative Commons Attribution 4.0 International License.
