創意產品共識評量之信、效度析論

Consensual Assessment for Creative Products: A Review of Reliability and Validity

本研究經由創意產品共識評量之相關文獻的回顧與評析，歸納出當前創意產品共識評量所存在的信、效度問題，包括：使用Cronbach’s 係數進行評分者一致性分析，導致評分者信度會因創作者能力分布以及評分者人數多寡，而呈現不穩定的現象；僅以領域專家在「整體創造力」評量指標上的共識程度，做為效度之依據，缺乏創造力概念的聚斂效度與區辨效度之檢核；忽略主觀評量過程中評分者效應所造成的評量誤差，導致相同專業背景的評分者，在不同的研究中所呈現的評分者一致性存有相當的差異。最後，本研究以試題反應理論的Rasch 多相模式觀點，針對上述的問題，提出未來研究的可行方向，期能改善創意產品共識評量技術與提升評量的品質。

The study reviews the literature on consensual assessment of creative products and generalized problems of reliability and validity in current consensual assessment of creative products. Major issues are (a) problems of unstable rater reliability caused by creators’ ability distribution and rater’s number when Cronbach’s was used to analyze rater reliability，(b) problems of lacking examination of convergent validity and discriminant validity in creativity concepts when most researches only used domain experts’consensual level in “overall creativity” as the basis of validity， and (c) problems of ignoring measurement errors caused by raters’ effects during subjective assessing process， resulting in significant differences of rater consistency for raters of the samebackground in different research findings. Finally， to resolve these problems， the study proposes a feasible direction of applying the many-facet Rasch model for future studies in order to improve consensual assessment technique of creative products and elevating the quality of assessment.