How to Test Speaking Skills in Japan

51 Chapter 5: Identify & define your evaluation criteria

This is one of the biggest decisions you’ll make when devising your analytic marking scheme because there are multiple ways of defin ing seemingly obvious categories Take fluency, for example We all know what that means… Right? Here is Fulcher & Davidson (2007: 7) on this point: In order to measure ‘fluency’ we have to state what we could possibly observe in speech to make a decision about whether a speaker is fluent. It turns out that many people have different definitions of fluency, ranging from simple speed of speech, to lack of hesitation (or strictly ‘pauses’, because ‘hesitation’ is a construct itself), to specific observable features of speech. Personally, my definition of fluency is how smoothly you speak. This captures the essence of this concept in a way I know my students can understand Obviously there is more to fluency than “smooth ness”, but at this point I’m more concerned about defining it prac tical terms However, that’s just me—obviously there are other ways to look at this key element of speaking ability For example, total word count could be another way to go—the more words students speak in a set period of time, the more fluent they are You could also look at the amount of silence in conversation, ie the amount of noticeable pauses Same thing goes for the other constructs you wish to track—find ways to express them simply and directly Think about what you want, then write your level descriptors in simple, clear language Similarly to holistic scales, you’ll also need to decide on how many proficiency levels you want to assess for each construct Four or five are most common, but you could experiment with more or less and see what works best Again, it all depends on what you are aiming for