自動化校長即席演講評分系統建置之初探

A Prelimary Study for Automatic Scoring System in Oral Presentation

為了因應大型評量的需要，自動化的口語評量系統為目前評量發展的新趨勢。近年來，自動化評分系統已逐漸應用於教育測驗、課程教學、心理計量等領域，本研究擬透過行為特徵處理的方式（behavioral signal processing），來建置一套結合音訊、視訊之自動化口語評分系統。樣本為儲訓校長，演說內容為教學、校務等相關議題。本研究提出多模態計算架構，結合培訓計畫的音訊與視訊資料，並透過詞袋模型（bag of words）與費雪矢量編碼（Fisher-vector encoding），以建置自動即席演講評分系統。研究顯示，只要蒐集到的樣本在口語表現上的變異性夠大，即能有效促進機器學習的效果。此外，由本研究的實驗可看出，機器和真人的評分，甚至會高於真人評分者彼此間的相關性。然而，未來在增進系統的辨識率方面，可透過加入文本資料、處理音視訊錄製過程中的雜訊、蒐集更多演講樣本、評分者的評分等，來提高自動化系統的準確率。

Automatic scoring system is the current trend for large scale assessment. Automatic scoring system has became a popular research tool in the field of educational assessment, curriculum teaching and psychological measurement. In this work, we employ behavioral signal processing (BSP)-based methodology to develop a computational framework that can automate the scoring process of pre-service school principals' oral presentations given at the yearly training program. Using the audio-video feature extraction approach with session-level representation techniques based on bag-ofword and Fisher-vector encoding, we can then characterize each pre-service school principal's multimodal behavior during an impromptu speech examination for automatic scoring. For the future study, we will include lexical content and annotation; collect more samples and raters which could potentiallyimprove the accuracy of this system.