How to Test Speaking Skills in Japan

21

Chapter 2: Embrace testing

Key principles for evaluating tests: Reliability, validity, and practicality

How can you tell if a test is doing its job? Well, as I’ve mentioned, the field of language testing is packed with technical terminology However, if you boil it down, there are basically just three main principles that form the core of the field: reliability , validity , and practicality Most books on testing feature these concepts promi nently They are deep and multi-faceted, like gems Fortunately, the central idea behind each principle appeals to our common sense, so they are not difficult to understand Let’s examine each one in turn and then see how we can use all of them to create a rubric of sorts for evaluating any test (speaking or otherwise) we wish to give Reliability This principle is all about the fairness and consistency with which your tests are administered Nation (2013) poses a number of help ful questions you can ask yourself when reflecting on the reliability of your exams: Do your students understand the directions? This is a key point—if some students do but others don’t, then your test is not reliable because a portion of the class did not know what to do This is especially important for speaking tests—your students will no doubt be feeling nervous, so being clear on the directions will help them calm down a lot The easiest way to think about this point is to put yourself in their shoes How would you feel if you had to take a test in Japanese but couldn’t understand the instructions? Do they know what they’ll be tested on? This is a key aspect of any CRT—in order to do well, students need to know what material will be on the test ahead of time (Brown,