How to Test Speaking Skills in Japan

39 Chapter 4: Holistic or analytic? Decide your overall approach

backs involved with holistic marking mainly impact test reliability If you want to take advantage of the benefits of this approach, it’s vital you remain cognizant of its limitations and take steps to miti gate them Anything you can do to make your holistic scheme more transparent, consistent, and fair will help What is analytic marking? In analytic marking, the general approach is to break down a per formance into smaller bit parts known as constructs 1 , such as fluency , grammar , vocabulary , pronunciation , or attitude Each construct is given its own scale and is marked separately, and the final score is the grand total of all the individual marks For example, you could mark a construct such as grammar in an impressionistic manner, or you could employ an additive or subtractive strategy, giving points or taking them away as you see fit In addition, constructs can be weighted to emphasize their importance Thus fluency could com prise 40% of the total score, with the other constructs filling up the remaining 60% Since speaking is itself so hard to pin down, there are many possible ways you could assess it When considering the vast range of options, it may help to know that linguistics researchers in sec ond language acquisition often group constructs around three basic dimensions: complexity , accuracy , and fluency (Housen & Kuiken, 2009) In other words, most of the things you could assess in a stu dent’s oral test performance can basically be boiled down to one of these three categories In effect, no matter how you slice it, you are fundamentally looking at levels of language sophistication, how ac curately it was used, or how smoothly it was spoken A helpful rule of thumb is to let your course objectives guide you 1 According to the 2nd Language Testing online glossary (www.2lti.com/glossary), a con struct refers to the knowledge, skill, or ability that is being tested but cannot necessarily be directly observed or measured, such as listening ability. Language tests attempt to measure various constructs which underlie language ability.