Listening, the “Forgotten” English Skill: An Interview With Spiros Papageorgiou
Below is an interview with ETS’ Principal Measurement Scientist, Spiros Papageorgiou, who conducts research on the assessment of English as a second language, primarily supporting the TOEFL® family of assessments, and John Clark, Director of Strategic Initiatives. You can read more of Spiros’ research here.
In a recent book chapter, you quoted a researcher who called Listening the “forgotten skill.” Why is Listening sometimes perceived that way?
This is a quote from Gary Buck, who’s the guru of assessing listening in the field of language assessment research. To be fair, there’s been a lot of research on assessing listening skills in the last several decades. But in general, there's still a belief that listening is the skill that is the least researched.
At the same time, it’s arguably the most important skill. To put it in simple terms, we talk about speaking being the ability to communicate, right? But you can’t speak and communicate if you don't have the listening skills to process these inputs.
Of course, communication also happens in writing. But when we talk about the ability to speak in any language, the first skill that we need to develop is listening.
What are some of the main challenges in trying to determine a student's ability to listen to spoken English?
With all assessments, the core challenge is that we want to elicit enough information to provide a score, but nobody's actually interested in the score as a number. What they're interested in is what the student is able to do. So that's the biggest tension we face as developers and researchers in the language assessment field.
With listening, one of the main challenges is that we tend to evaluate the skill outside the context of actual communication. Listening is a receptive skill like reading. But unlike reading, the test taker has very little control over the input of a listening test question.
The additional challenge we have with listening is that we tend to evaluate it with the test taker being a passive recipient of information. In real life, when you listen, you have the opportunity to ask for clarification. And if you watch something, you can replay it if you did not understand something in the first place.
In a listening test, because of the typical constraints that we have with the administration of an exam, we tend to focus on an “overhearer” role that is not very natural. As a result, testing listening can be quite challenging because of these test administration tensions and the inferences that we're trying to make based on a student’s listening score.
You've written before about the distinction between monologic and dialogic inputs. Could you explain what those terms mean and how they impact the design of TOEFL’s listening section?
First, we need to consider the fact that there are many different factors that can make a listening test question difficult. From the speed of speech to the accent to background noise, many audio features can alter difficulty.
To your question: I led a study many years ago where we asked the question about monologic and dialogic input. With most listening tests, we have the test taker listen to one speaker – think of a lecture. This represents a monologic input.
Alternatively, we can feature dialogic inputs, like conversations between two speakers. (We typically have two because this makes it more practical for test administration.) So now the question is: Is there consensus in the literature on the relative difficulty of these two different inputs. Monologic versus dialogic?
Alas, the results of the study that I led were not very conclusive. Although there was a tendency for monologic input to be a little bit more difficult. The assumption behind this finding is that when you have two speakers, they collaborate to make sense of what they're trying to communicate. So that could have made those questions a bit easier.
But again, it comes down to the key principle with listening test questions. There are so many factors that affect difficulty that it's almost impossible to just pin down difficulty of listening questions to just one factor.
Is it fair to say that TOEFL’s approach to address this uncertainty is to include both types of inputs, a variety of accents, and a diversity of question types?
That's a key principle with the design of TOEFL, especially given the primary use for admissions into English-speaking academic programs.
The main idea is that we have to develop test tasks that – to the extent possible given the constraints of test administration – reflect the kind of skills and abilities that students will eventually use in real-world academic environments.
In the listening section of TOEFL, the idea has been to include both monologues and dialogues, with a variety of the most common English accents, but with very careful design principles behind the choice of accents, for example when it comes to their strength. We try to include as many of the different factors that affect difficulty in our design.
Your own experience might be an instructive example, as you did not grow up speaking English as your first language. When it came to listening, did you find it harder to understand monologues or conversations?
I’m the typical example of a monologic, monocultural kid that grew up in small town in northwestern Greece, where I only encountered speakers of my first language, Greek. I learned English as a school subject. Of course, I later studied in an English-medium department at the University of Athens, and my undergraduate degree was in linguistics and English language.
So I had a lot of exposure to English as an undergraduate student and then I went to the University of Lancaster in the UK where I did my Masters and PhD, where I also met many fellow students from different countries. I was lucky to have many educational experiences in English.
But I remember, when I was trying to learn English at a young age, listening was the most stressful part in an exam because you felt like you had so little control over what was going on.
When you take a listening test, some audio is playing, if you are lucky, it might be played twice. That’s it. In the speaking test, which can also be stressful, you can ask the examiner to repeat or clarify. In the reading test, you have control over how fast you read. In the writing test, you can start and do it all over again.
With listening, there's no control. And I think that's one of the reasons why many students, especially when they take language tests, find listening very challenging.
“Monologic, multicultural kid” is a great schoolyard insult, by the way. Do you have any tips for students who feel uncomfortable with monologic listening – both on the exam and in life itself?
As I was learning English, I have to say I find monologic discourse easier than dialogic for a number of reasons. Typically, we associate monologic discourse with more prepared language, like a presentation or a lecture. It tends to be more polished.
Think of announcements. Announcements are a great example of monological discourse, right? They tend to be structured in very specific ways. It was also always much easier for me to watch the news on a big network like CNN because the language was very structured.
What has been difficult, and continues to sometimes be difficult for me, is when native speakers of English talk to each other very quickly, with their language being more colloquial, so less structured, less rehearsed.
Do you think English learners today have unique advantages that you may not have had when you were learning the English?
Yes, learners now have amazing opportunities to develop the listening skills that when I was growing up and learning English, you know, 30 years ago, I did not have.
And primarily it's because of technology, especially with listening. So many tools are now available, from apps on the phone to educational programs on the computer. The access, the amount of information, the amount of free content is nothing like what was available to test takers before.
The key to improving your language proficiency – including listening skills – is exposure. And if you have enough exposure, of course, then it's the quality of the instruction or the input that you receive. But you need exposure to real, authentic audio input.
Yeah, it’s truly a different world. One last question for you, Spiros. You mentioned accents. On TOEFL today, we have a mix of North American, UK, and Australian accents. Tell me how we ensure these accents aren’t too thick?
With a number of characteristics of our listening tasks, there's always a tension between trying to make our tasks as authentic, as real life as possible, but also adhere to important measurement principles.
So it's very easy to say, “we need to include every possible English accent in our test because that's what happens in real life.” Even if I study in the U.S., my professor may have an accent from another country and so you perhaps you need to include all different accents. But it’s just not possible.
Given this limitation, we went with a principled approach to including accents in the listening section and have included the most common accents, like North American or British English. We also use the findings from a research study that we started about 10 years ago that measures the impact of varying accent strengths.
Such principled approach to including different accents in a listening test is important because in real life, if I don't understand someone's accent, I have time to ask clarifying questions. I don't have that opportunity in a listening test. So it's unfair for us to expect test takers to understand every single accent out there when they sit for a listening test.
Fair enough. Well, Spiros for a monologic, monoculturalist, this has been a great conversation in your non-native language.
Thank you!
