Evaluación de la validez y fiabilidad de un sistema de test basado en el grado de confianza del estudiante

• Valencia Rodríguez, Carlos ^[1] ; Ortega Latorre, Yolanda ; Huerta Cebrián, Paloma
  1. [1] Universidad Autónoma de Madrid

     Universidad Autónoma de Madrid

     Madrid, España

     
• Localización: Revista Iberoamericana de Evaluación Educativa, ISSN-e 1989-0397, Vol. 13, Nº. 1, 2020 (Ejemplar dedicado a: Aprendiendo a Evaluar para Aprender en la Educación Superior), págs. 225-238
• Idioma: español
• DOI: 10.15366/riee2020.13.1.010
• Títulos paralelos:
  • Evaluation of the validity and reliability of a test system based on the degree of confidence of the student
• Enlaces
  • Texto completo (pdf)
• Resumen
  • español

    Los exámenes de tipo test basados en la confianza del alumno son escasamente utilizados en el entorno académico español. Este sistema de evaluación hace énfasis tanto en el conocimiento del alumno como en el grado de seguridad que tiene del mismo. Se analizaron los resultados obtenidos en exámenes con el sistema de test basado en la confianza (TBC). El estudio se realizó en una escuela de Enfermería y Fisioterapia durante los cursos comprendidos de 2009 a 2014. Se compararon la validez y fiabilidad del TBC con las obtenidas con el sistema de test con puntuación tradicional por aciertos. Los alumnos adquieren pronto destreza con el nuevo procedimiento de puntuación, escogiendo de forma racional la opción de seguridad más adecuada acorde con su nivel de conocimientos. La validez del sistema de corrección es superior en el TBC respecto al sistema tradicional de test, con una diferencia estadísticamente significativa. La fiabilidad se analiza en cinco grupos de alumnos mediante el coeficiente α de Cronbach, siendo en cuatro de ellos superior la consistencia interna con el TBC e igual en el restante. El test basado en el grado de seguridad ha mostrado ser un procedimiento que aporta ventajas frente al test tradicional. Proporciona mayor información al profesor, los alumnos se adaptan a él rápidamente y ha demostrado mayor validez e igual fiabilidad a las puntuaciones que el sistema tradicional.

  • English

    Test-type exams based on the student's confidence (CBT) are scarcely used in the Spanish academic environment. This evaluation system emphasizes both the student's knowledge and the level of confidences he or she has of that knowledge. The results obtained in CBT were analyzed. The study was conducted in a school of Nursing and Physiotherapy during the academic years covered from 2009 to 2014. The validity and reliability of the CBT were compared to those obtained with a traditional correct-response grading system. Students acquired dexterity with the new scoring procedure quickly, choosing the most appropriate option for confidence level that was in accordance with their level of knowledge in a rational way. The validity of the correction system was higher in the CBT compared to the traditional test system, with a statistically significant difference. Reliability was analyzed in five groups of students using the Cronbach α coefficient, with four of them having higher reliability with the CBT as compared to the traditional grading system and the fifth having the same level of reliability for the two types of tests. Confidence based testing has shown to be a procedure that provides advantages over the traditional test. CBT provides more information to the teacher, students adapt to it quickly and it has greater validity and at least equal reliability to the traditional grading system.

• Referencias bibliográficas
  • Ahlgren, A. (1970). A hand-scoring system for confidence-weighted scores. Washington, DC: Department of Health, Education & Welfare, Office...
  • Barr, D. A. y Burke, J. R. (2013). Using confidence-based marking in a laboratory setting: A tool for student self-assessment and learning....
  • Ben–Shakhar, G. y Sinai, Y. (1991). Gender differences in multiple-choice tests: The role of differential guessing tendencies. Journal of...
  • Biggs, J. B. (1993). What do inventories of students’ learning process really measure? A theoretical review and clarification. British Journal...
  • Bloom, B. S. (1956). Taxonomy of educational objectives: the classification of educational goal S: Handbook I, cognitive domain. Londres:...
  • Budescu, D. y Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement,...
  • Chang, C. Y. y Cheng, W. Y. (2008). Science achievement and students’ self-confidence and interest in science: A Taiwanese representative...
  • Ebel, R.L. (1968). Valid confidence testing: Demonstration kit. Journal of Educational Measurement, 5(4), 353–354.
  • Entwistle, N., Hanley, M. y Ratcliffe, G. (1979). Approaches to learning and levels of understanding, British Journal of Educational Research,...
  • Frary, R. B. (1982). A simulation study of reliability and validity of multiple–choice test scores under six response–scoring modes. Journal...
  • Gardner, W. C. (1969, septiembre). The use of confidence testing in the Academic Instructor Course. Comunicación presentada en el Annual Conference...
  • Gardner-Medwin, A. R. (1995). Confidence assessment in the teaching of basic science. Association for Learning Technology Journal, 3(1), 80-85....
  • Gardner-Medwin, A. R. (1998). Updating with confidence: Do your students know what they don’t know?. Health Informatics, 4, 45-46.
  • Gardner–Medwin, A. R. (2006). Confidence-based marking towards deeper learning and better exams. En C. Bryan y K. Clegg (Eds.), Innovative...
  • Gardner–Medwin, A. R. (2011). Reasonable doubt: uncertainty in education, science and law. Proceedings of the British Academy, 171, 465-83.
  • Gardner-Medwin, A. R. y Curtin, N. (2007). Certainty-Based Marking (CBM) for reflective learning and proper knowledge assessment. Recuperado...
  • Gardner-Medwin, A. R. y Gahan, M. (2003, julio). Formative and summative confidence-based assessment. Comunicación presentada en el 7th International...
  • Hambleton, R. K., Roberts, D. M. y Traub, R. R. (1970). A comparison of the reliability and validity of two methods for assessing partial...
  • Hassmen, P. y Hunt, D. P. (1994). Human self–assessment in multiple–choice testing. Journal of Educational Measurement, 31(2), 149-60. https://doi.org/10.1111/j.1745-3984.1994.tb00440.x
  • Hevner, K. (1932). A method of correcting for guessing in true–false tests and empirical evidence in support of IT. The Journal of Social...
  • Hunt, D. P. (2003). The concept of knowledge and how to measure it. Journal of Intellectual Capital, 4(1), 100-13. https://doi.org/10.1108/14691930310455414
  • Kansup, W. y Hakstian, A. R. (1975). A comparison of several methods of assessing partial knowledge in multiple choice tests: I. Scoring procedures....
  • Khan, K. S., Davies, D. A. y Gupta, J. K. (2001) Formative self–assessment using multiple true–false questions on the internet: Feedback according...
  • Kleitman, S. y Stankov, L. (2001). Ecological and person-oriented aspects of metacognitive processes in test-taking. Journal of Applied Cognitive...
  • Luetsch, K. y Burrows, J. (2016). Certainty rating in pre-and post-tests of study modules in an online clinical pharmacy course - A pilot...
  • Marton, F. y Säljo, R. (1976a). On qualitative differences of learning (I): outcome and process. British Journal of Educational Psychology,46(1),...
  • Marton, F. y Säljo, R. (1976b). On qualitative differences in learning II: outcome as a function of the learner’s conception of the task....
  • Moore, D.A. y Healy, P.J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502-517.https://doi.org/10. 1037/0033-295X.115.2.502
  • Morales, P. (2006). Las pruebas objetivas: normas, modalidades y cuestiones discutidas. Recuperado dehttp://www.upcomillas. Es/personal/peter/otrosdocumentos...
  • Morales, P. (2008). Estadística aplicada a las Ciencias Sociales. Madrid: Universidad Pontificia Comillas.
  • Morony, S., Kleitman, S., Lee, Y. P. y Stankov, L. (2013). Predicting achievement: confidence versus self-efficacy, anxiety, and self-concept...
  • Panadero, E. y Alonso-Tapia, J. (2014). How do students self-regulate? Review of Zimmerman’s cyclical model of self-regulated learning. Anales...
  • Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching and assessing. Theory into Practice, 41(4), 219-25. https://doi.org/10.1207/s15430421tip4104_3
  • Rippey, R. M. (1978), Interactive confidence test scoring and interpretation. Educational and Psychological Measurement, 38(1), 153-57. https://doi.org/10.1177/001316447803800122
  • Sharma, M. D. y Bewes, J. (2011). Self-monitoring: Confidence, academic achievement and gender differences in Fhysics. Journal of Learning...
  • Stanger-Hall, K.F. (2012). Multiple-choice exams: An obstacle for higher-level thinking in introductory science classes. CBE Life Scientific...
  • Stankov, L. (2000). Complexity, metacognition and fluid intelligence. Intelligence, 28(2), 121-43. https://doi.org/10.1016/S0160-2896(99)00033-1
  • Stankov, L., Lee, J. y Paek, J. (2009). Realism of confidence judgments. European Journal of Psychological Assessment, 25, 123-30. https://doi.org/10.1027/1015-5759.25.2.123
  • Stankov, L., Lee, J., Luo, W. y Hogan, D. J. (2012). Confidence: A better predictor of academic achievement self-efficacy, self-concept and...
  • Thomson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54,837-47.
  • Urosa, B. (1995). La adivinación en las pruebas objetivas: Alternativas a la fórmula clásica de corrección. Tesis Doctoral, Universidad Pontificia...