檔案評量信度與效度的分析--以國小寫作檔案為例

Reliability and Validity of Writing Portfolio Assessment

近年來檔案評量為教學與學習開展了無限的契機，但它是否真能不負眾望達到如推動者所宣稱與預期的目的？本研究即在以國小寫作檔案為例。進行檔案評量信度與效度的分析與討論。研究由高雄市七所國小四、五年級共十個班的教師與學生參與，其中六個班為實驗組，實施為期一學期的寫作檔案，四個班為控制組，未接受任何形式的檔案評量。在檔案信度方面，結果顯示透過明確的計分規準及嚴謹的評分者訓練可以提高評分者一致性及信度至相當的水準，但其程度仍受特定評分者及檔案層面的影響。概括性研究結果顯示雖然檔案分數的變異大部分來自受試者的差異，且評分者變異在大多數情形下所佔的比例並不高，但仍有相當比例的變異來自受試者與評分者之交互作用及其它誤差。在檔案效度方面，研究者透過界定明確的檔案目標，再直接根據檔案目標設計檔案活動、檔案項目及計分規準的評量模式能有效的提昇檔案與所測特質間的關聯性。在實施成效上，雖然實驗組的學生在寫作態度上有顯著的提昇，且師生均認為寫作檔案對其教學與學習有正向的影響，但當測驗前測之影響力被排除後，接受與未接受檔案評量的學生在寫作態度及作文表現上均無顯著的差異存在。另外，研究結果大致支持寫作檔案具斂效度與區辨效度及檔案資料符合學生寫作能力之發展理論。最後，本文就檔案的信效度及影響與提高檔案整體效度的一些相關議題與可行方法提出討論。

The major purpose of this study was to examine the reliability and validity of writing portfolios. The students from ten classes of seven elementary schools in Taiwan participated in this study. Six of the ten classes received writing portfolio assessment and four classes the traditional writing assessment.This study shows that with well-defined scoring rubrics and proper rater training， an acceptable level of rater consistency and reliability could be achieved. However， the degree of consistency varied among portfolio dimensions and raters. Results obtained from generalizability study showed that the largest proportion of variation was due to person and， in most cases， only a small amount of variation was due to rater. However， a considerable amount of variance was due to person and rater interaction， indicating that the rankings of individual students were different from rater to rater.In regard to portfolio validity， the degree of content relevance could be increased through defining the portfolio criteria， and then through constructing portfolio tasks and rating rubrics according to the criteria. Although there were significant increases in students’ attitudes towards writing for those in the experimental group， the differences between the experimental group and the control group on both a standardized writing test and attitudes toward writing were not statistically significant after the initial differences had been adjusted. Both students and teachers reported positive effects of writing portfolios on their learning and teaching. The convergent validity of the writing portfolios was generally supported by the moderate correlation between portfolio scores and other writing measures. In addition， the results were also in accordance with the developmental stages of children’s writing ability.