How to Test Speaking Skills in Japan

41 Chapter 4: Holistic or analytic? Decide your overall approach

Disadvantages of analytic marking Of course, analytic marking also has its drawbacks One of the major ones is that tests utilizing this approach take more work and effort to administer As a result, an analytic scheme may be impractical for your needs Instead of one single score quickly given, you need to track sever al constructs simultaneously So, while conceptually easier to mark (because you’re only looking at one aspect at a time), taken together it’s quite a lot to handle, especially for teachers new to it In addition, constructs poorly chosen can have a deleterious effect on test valid ity For example, effort and creativity are conceptually quite similar, so it may be very difficult to accurately distinguish between them This feeds into a common phenomenon often found within analytic marking schemes—the halo effect This is when one preferred con struct has an inordinate amount of influence on the others (Weir, 2005) Ideally, you want to mark each construct as separately and equally as possible To avoid the halo effect, choose unique con structs and remain mindful of your biases as you go about your marking—don’t let that sweet smile and spunky personality override the poor grammar accuracy (or vice versa) Holistic and analytic marking approaches are not mutually ex clusive; it is also quite possible to blend them into one cohesive system that takes the best from both One good reason for going with a blended approach is that you can minimize their respective drawbacks This can provide you with a more balanced and flexible system To illustrate, here is a sample rating scale that combines elements of holistic and analytic scoring (Nation, 2013: 155):