國小學童資訊素養檔案評量之信度研究

The Reliability of Information Literacy Portfolio Assessment in the Primary Grades

本研究的目的在探討國小資訊素養檔案的分數信度，透過三個面向檢視檔案分數的信度，分別是整體得分的類推性係數與可靠性係數、運用資訊能力歷程向度評分之變異數成分估計和類推性係數以及內外評分者的一致性信度。參與研究的為國小三年級和五年級各兩個班的學生，檔案設計以大六教學法為參考架構，結合語文、自然與生活科技、社會三個學習領域，每位學生在每個領域需完成2－3份作品，研究為期3個學期。研究結果顯示不分年級與學習領域，在僅有一位評分員的條件下，整體評分類推性係數多半可達.75以上，而可靠性係數僅略低於類推性係數，信度良好。歷程向度評分的變異數成分分析顯示，變異的主要來源有四，分別為學生差異、歷程向度、學生與歷程向度的交互作用，以及學生、評分者、和歷程向度的交互作用。歷程向度的類推性係數最佳值在0.5和0.6左右，仍有改善的空間。校內教師間的評分相關普遍高於教師與外部研究助理的相關，校內教師的給分多半較外部評分者寬鬆，不過兩者的差異量並不大。本研究顯示透過適當的檔案設計，任課教師能一致地評量資訊素養檔案。

This article reports on the reliability of information literacy portfolio assessment in the primary grades. Three central questions were examined: (1) how reliable are total scores of information portfolio? (2) how reliable are scores of big six skills? and (3) what are probable sources of unreliability? Do teachers who rate their own students' work systematically score differently than do outside raters? The framework of information portfolio was based on the big six skills approach. This assessment was based on a sample of approximately 70 portfolios integrated three learning areas for students in Grades 3 and 5: language arts, mathematics, and science and technology. The teachers were required to collect each student's portfolio for 3 semesters. The products of portfolios in each learning areas were rated by 3 raters: the student's classroom teacher, a teacher of the same primary school, and an external reviewer. Results indicated that most of the generalizability coefficients for total scores were above 0.75 across learning areas and semesters with a single rater. The dependability coefficients revealed a similar pattern with slightly lower size. Variance in scores of each processing dimension was attributed to individual difference, processing dimension, interaction of individual difference and dimension, and interaction of individual difference, dimension, and rater. The generalizability coefficients for the six-dimensional task were around 0.5 to 0.6. The results indicated that instructional planning based on scores of specific information skill should be questioned. In general, the correlations between classroom teachers were higher than those between classroom teachers and external raters. Classroom teachers tended to rate student portfolios higher than did external raters; however, these differences were small. This study showed that given a solidly structured portfolio, teachers can reliably rate their students' work.