What Factors Most Impact on Chinese Students’ Performance in the IELTS Academic Reading Sub-Test? ()
1. Introduction
The International English Language Testing System (IELTS), consisting of four sub-tests: listening, reading, writing and speaking, has been recognised in many countries as a trustworthy English language test (IELTS, n.d.), so it is set as one of the entry requirements by many overseas universities, colleges, and institutions. The younger Chinese generation are often eager to get a foreign academic qualification, so achieving the required IELTS result is a necessary step before they can attempt to enrol in a foreign university.
According to the results from test taker performance in 2019, Chinese students had higher average score in the reading sub-test (6.2) and lower average score is in the writing (5.5) and speaking sub-test (5.4), respectively; in particular, the speaking score was ranked the lowest among the top 40 surveyed nationalities (IELTS, 2019). Because of this, the focus of Chinese students is more shifted towards writing and speaking when making test preparation. The academic reading sub-test, however, draws less attention because compared with other three sub-tests, most Chinese students consider the reading part as relatively easy. But in fact, even though the academic reading part is the highest scoring sub-test for Chinese students, it still lags far behind from the reading scores of other countries, such as Indonesia with the average score of 6.7 (IELTS, 2019), which means the reading score in China can be further improved.
Regarding the research field, although there are many IELTS related research papers published every year, few of them are directly connected with the academic reading sub-test. The most commonly researched area in the academic reading section is the construct validity of the test itself (IELTS, n.d.) but our search of the published literature has not found research that specifically addresses the factors which could have an impact on test takers’ performance in the reading section of the test. Therefore, in order to further help Chinese candidates improve their IELTS academic reading band scores and facilitate teachers to better understand the reading part, the purpose of our research is to address the paucity of research regarding the factors most influencing Chinese student performance in the IELTS academic reading sub-test.
2. Literature Review of the Contributing Factors
The main purpose of reading comprehension in most English tests is to obtain information for doing some tasks (Nunan, 1999). In other words, the nature of the reading comprehension task is a problem-solving activity that requires readers to look for information in the passage and understand the meaning (Bowen, Maadsen, & Hilferty, 1985). Thus, the IELTS academic reading sub-test is designed to make the test takers find the information for each test item in the passage and understand the meaning of relevant sentences so as to resolve all the test items within a given time. To be specific, candidates are required to complete three reading passages and a total of 40 question items within an hour.
But the problem for most students is how to find the relevant information efficiently in a limited time when they are facing various tasks in the test. The 40 items in the IELTS academic reading sub-test are composed of nine tasks which are True/False/Not Given, Gap Filling including sentence completion, note, table, flow-chart completion, short answer questions, and diagram label completion, Multiple Choices—Four Choices, Classification, Matching—Finding Correct Endings, List of Headings; Summary, Multiple Choices—More Than Four Choices, and Matching—Finding Paragraphs. Several researchers have already shown that the IELTS academic reading test involves the implementation of specific reading strategies to solve different reading tasks (Chalmers & Walkinshaw, 2014; Cullen, French, & Jakeman, 2014; Moore, Morton, & Price, 2012). In this case, determining the strategies or tactics that the test takers may use when handling these tasks should allow us to identify the factors that could affect their performance during the IELTS academic reading sub-test.
In 2014, the IELTS officials published the Official Cambridge Guide to IELTS edited by Cullen, French, and Jakeman. In the academic reading test guide section, it clearly suggests that the nine tasks each have distinctive features thus requiring different reading strategies. The guidebook also demonstrates that the reading strategies of locating specific information and distinguishing the main ideas from the supporting details are needed for completing various reading tasks. To be specific, some tasks (True/False/Not Given, Gap-Filling, Multiple Choice—Four Choices, Classification, and Matching—Finding Correct Endings) require students to identify the keywords in the question items, locate the keywords in the reading passages, and then understand the relevant sentences. In tackling the other reading tasks (List of Headings, Matching—Finding Paragraphs, Multiple Choice—More Than Four Choices, and Summary), students are expected to understand the main idea of each paragraph.
Consequently, it could be said that during the IELTS academic reading sub-test, when dealing with different reading tasks, candidates need to use distinctive reading strategies in accordance with the feature of each task. The characteristics of these tasks will determine what strategies are needed for facilitating test takers to find the correct answer. The next step of our study, therefore, is to identify the contributing factors associated with the specific features in the nine reading tasks in the IELTS academic reading sub-test.
The first contributing variable is the size of the English vocabulary known by the test takers. In our study, it is defined as a common variable because an individual’s vocabulary size has an impact on student performance in every reading task. Golkar and Yamini (2007) confirmed that vocabulary size is highly correlated (p = 0.000) with the English proficiency level of test takers and their reading comprehension ability. Stæhr (2008) also showed that candidate’s vocabulary level correlates highly with their reading comprehension capability. Milton, Wade and Hopkins (2010) found that vocabulary size has a significant coefficient rating with the reading component of the IELTS reading sub-test.
The second common variable is the sentence length. Weir et al. (2009) demonstrated that whatever reading tasks test takers face, they need to understand the explicit meaning of a particular sentence or a group of sentences. Klare (1974-1975) and Graves (1986) illustrated that long sentence structure would tend to increase the difficulty level of understanding a piece of reading material. Hamby and Lekes (2015) also claimed that for English Second Language learners, increasing the length of sentences would raise the difficulty level of their reading comprehension.
The third variable that impacts test taker’s performance is the ability to identify and locate the keywords in the test items and the reading passages. According to Moore, Morton and Price (2012), some tasks in the IELTS academic reading sub-test involve selectively locating relevant information, which requires test takers to be able to accurately identify and find the keywords in both the test items and the reading passage. Thus if candidates are not able to identify and locate the keywords in some tasks, they would not perform well, and the vice versa. Moore, Morton and Price (2012) also demonstrated that among all the nine tasks in the IELTS academic reading sub-test, only five tasks (True/False/Not Niven, Gap-Filling, Multiple Choice—Four Choices, Classification, and Matching—Finding Correct Endings) require candidates to quickly find detailed information in the passage. Hence, the variable of identifying and locating the keywords does not relate to all the nine reading tasks.
The last variable to be discussed is understanding the main idea of a paragraph. Just like the variable of the keywords identification, the last variable does not involve all the reading tasks. According to Moore, Morton and Price (2012), the other four tasks (List of Headings, Summary, Multiple Choice—More Than Four Choices, and Matching—Finding Paragraphs) particularly expect test takers to be capable of finding the main idea of each paragraph.
In summary, previous research, as discussed above, suggests that there are four contributing variables that should be investigated in our research. Two of them (the vocabulary size and the sentence length) are the common variables that will be applied to all nine reading tasks, and the other two variables (identifying and locating the keywords and finding the main idea of a paragraph) only relate to some specific tasks. At the conclusion of our study, we hope to measure the impact of each variable on student’s performance to facilitate Chinese students and teacher’s pedagogy in the future.
3. Methodology
3.1. Settings
Our study was conducted in a large private English language teaching institution that has more than 1000 learning centers across China. It has a very good reputation for teaching quality in China, so most Chinese students would choose this institution to prepare for all kinds of English language tests, including IELTS, TOEFL, and SAT (Scholastic Assessment Test).
3.2. Participants
The researchers invited the students who were preparing for the IELTS test and the teachers who were specialised in the IELTS academic reading sub-test as the participants. Clearly, the students were needed to help to establish their perception of the factors that would impact their performance on the nine reading tasks. To ensure the English proficiency of the student participant to be levelled, the researchers only invited those who were attending the intensive training class in that institute, because these students all passed the placement test, indicating they basically had the same English proficiency. The participation of the reading teachers was due to their rich experience and understanding towards the reading sub-test. By comparing the results from both the students and the teachers, the results of the study could be more precise so as to provide more constructive suggestions for the future IELTS teachers and test takers.
3.3. Procedures
Our study began with the completion of a pilot study with four Chinese English second language students in New Zealand to check the questionnaire’s content and the time needed to finish the questionnaire. Minor changes were then made to improve the validity of the questionnaire before the main study began.
The main study included three stages:
Main Study Stage 1: Questionnaire
67 volunteer students agreed to complete the questionnaire. The questionnaire had 27 forced-choice questions, divided into the nine reading tasks, with three questions in each task. Every question was in accordance with the variables identified in the previous literature review to determine their influence on student’s performance. For example, for the task of Classification, the three questions were:
My vocabulary size affects my performance on the task of Classification;
My ability to identify and locate the keywords affects my performance on the task of Classification;
My ability to understand relevant sentences in the reading tests affects my performance on the task of Classification.
Main Study Stage 2: Student’s Interview
Four students were randomly selected for a personal interview after completing the questionnaire. The student’s interview schedule comprised nine questions which were constructed to elicit more details about the student’s self-perceptions identified in the questionnaire.
Main Study Stage 3: Teacher’s Interview
All the IELTS reading teachers (four) in that institute participated voluntarily in a focus group interview. The teacher’s interview contained 18 questions with each reading task having two specific questions to elicit their views concerning the influencing factors on student performance, and what actions teachers could take to improve student performance in the academic reading sub-test.
3.4. Data Analysis
The analysis of the quantitative student data focused on the four variables (vocabulary size, sentence length, identifying and locating the keywords, and finding the main idea of a paragraph). All nine reading tasks were explicitly analyzed with these variables using the IBM SPSS 23 (Statistical Product and Service Solutions Version 23)’s base system. The data analysis of the qualitative approach was made through coding the field notes from the interview of the students and the reading teachers, respectively. The process was completed by sorting their answers into the different reading tasks and drawing out similarities and differences. The last step was to combine the results of the quantitative and qualitative approaches to draw the final conclusions.
4. Results
One participant skipped 6 of the 27 question items in the questionnaire, and three participants skipped one item each. According to Brown and Furruggia (2012), the respondent who missed six questions (more than 10%) should be removed from the initial dataset, so this left 66 cases in the dataset. In the following section, the results from the questionnaire will be discussed by dividing them into each reading task. The interview results of student and teacher will also be noted to make the result more comprehensive and reliable.
True/False/Not Given
The keywords for the task of True/False/Not Given are easy to identify and locate in most question items. When students find the right places for the answers of each question item in the passage, the next step is to understand the meaning of question items and sentences in the reading passage, and then judge what the relationship between them is. For instance, in Test 4 of Cambridge English IELTS 10 (2015), Question 13 is “Randy Jacobs believes that loss of life from fires will continue at the same levels, despite changes made”, and the relevant sentence in the passage is “…says Randy Jacobs, Notwithstanding all the damage that will continue to be caused by wildfires, we will no longer suffer the loss of life endured in the past because of the fire prevention and firefighting measures that have been put in place” (p. 89). It can be seen that there are some words appearing in both sentences (Randy Jacobs and loss of life), so these keywords help test takers find the location of the answer quickly.
It, therefore, could be summarised that the variable of keywords identification and location is essential for doing this task, with 94% participants regarding the variable as important. Although the other two contributors (long sentence understanding and vocabulary size) are not as important as that of the keywords, their significance for students is also confirmed (69% and 79%, respectively).
Gap-Filling
For the variable of keywords identification and location, 94% students reported they could easily find some similar sentence structures in the question items and in the passages. For example, in Test 3 of Cambridge English IELTS 9 (2013), the relevant sentence in the passage for Question 33 and 34 (The probe transmitted pictures of both 33 ______ and _______ then left the 34 ______) is “The space probe, Voyager I, launched in 1997, had sent back spectacular images of Jupiter and Saturn and then soared out of the Solar System on a one-way mission to the stars” (p. 71). The same sentence structure (... and ..., then ... the...) appears both in Question 33 and 34 and in the passage. So, students could identify what words should be picked out from the reading passage more easily.
The variable of long sentence understanding was also emphasised by 68% participants. Vocabulary size was mentioned by 66% students, suggesting knowing more words is critical for making sense of long sentences.
Multiple Choice—Four Choices
According to the students, most question items in this task are easy to locate in the passages. In contrast to the majority of student’s opinion (90%) towards the variable of keywords location and identification, all the four teachers said that the easily identified keywords could not necessarily guarantee students picking out the right choice. There are four choices in each question item but only one of them is correct, which means test takers may have difficulty in identifying the correct answer. Taking Question 36 in Test 2 of Cambridge English IELTS 8 (2011: p. 52) as an example:
36. What does the writer suggest about the study of smell in the atmosphere in paragraph E?
A. The measurement of smell is becoming more accurate.
B. Researchers believe smell is a purely physical reaction.
C. Most smell is inoffensive.
D. Smell is yet to be defined.
Finding the location of the answer is easy, as the question item itself has given obvious indication that the answer is in paragraph E:
Significant advances have been made in the understanding of the biological and chemical nature of olfaction, but many fundamental questions have yet to be answered. Researchers have still to decide whether smell is one sense or two—one responding to odours proper and the other registering odourless chemicals in the air (Cambridge English IELTS 8, 2011: p. 51).
The problem, however, is test takers have to totally understand the meaning of the four choices as well as the corresponding sentences in paragraph E, then they could select the correct answer: D.
The results of the quantitative data and teachers’ analysis in the qualitative data provided different opinions regarding the impact of the three variables. The quantitative student survey showed that keywords identification and location (90%) was more important than the other variables, while the teachers in the qualitative approach suggested that long sentence understanding was more significant. Given the small number of teacher interviews, this difference could be attributed to a non-typical cohort of interviewees. Further research beyond the scope of this study would be required to clarify the exact influence of these variables on teacher’s opinion and student questionnaire.
Classification
The question items in this task all have obvious keywords, so most students (87%) could identify and locate them in the passage. But it is quite possible that the keywords would appear in the same passage for several times, so candidates need to understand the meaning of all the related sentences for better answering the questions; that is why the size of vocabulary they know and the extent they understand long sentences were considered important by 82% and 68% students in the survey, respectively.
The similarity between the quantitative and qualitative data (interviews from teachers and students) was seen as the evidence of the role played by keywords identification and location. All teachers and students in the qualitative data also acknowledged the contribution of long sentence understanding.
Matching—Finding Correct Endings
This task usually has no more than eight question items, but the location of the answers between the first and the last question item may range from the first paragraph to the last paragraph in the passage, which means the range of locating the answers in the passage is usually large and irregular. Besides, the distribution of the answer’s location in a passage is random, which creates a huge barrier for students locating question items in the passage. What is more, even if students could find the right location, they sometime could not pick the right choice out, because this task also requires test takers to totally understand the meaning of the relevant sentences appearing in the passage and in the answer choices.
The quantitative and the qualitative data results reached consensus about the importance of the three variables (vocabulary size, the keywords identification and location, and long sentence understanding). In the quantitative results, a majority of students perceived all three variables having a great impact on their performance (81%); in the qualitative results, teachers and students also thought all three variables were important, which suggests that the three variables have a significant impact on student performance in this task.
List of Headings
This task expects candidates to find the main idea of each paragraph, which generally could be got from the first and the last sentence of each paragraph. That is why some students (85%) felt some question items were not hard to do in respect to the topic sentence identification. But sometimes the first and the last sentence of a paragraph seems irrelevant to the answer choices, so candidates need to go through the whole paragraph for answering the questions. In this case, students must read more; but in fact, the more they read, the more they have to understand, and the more likely they would misunderstand some sentence meaning.
Both the quantitative and the qualitative results acknowledged the importance of the topic sentence identification (85%) and the vocabulary size (77%), which indicates these two variables are necessary for improving student performance. Despite this, a distinction between the two approaches happened on the variable of long sentence comprehension. The qualitative data results suggested that long sentence understanding was very significant for students (both teachers and students), while the quantitative data denies that. This unexpected difference suggests further research into the variable of long sentence understanding would be helpful for future studies.
Matching—Finding Paragraphs
Generally, the distribution of the answers was large in this task; in other words, the answers may be scattered randomly in every paragraph. Candidates, therefore, need to go through the main idea of each paragraph efficiently, which suggests the ability of identifying the topic sentence is very necessary. Another aspect is about long sentence comprehension. During the process of finding the answers, if students could not thoroughly understand the sentence meaning, there is little chance to answer all question items well.
The quantitative and the qualitative data results all acknowledged the essential role played by the variable of the topic sentence identification and understanding long sentence with 82% and 75% support from participants, so the above two variables are important for students to have a better performance in this reading task.
Summary
The Summary task usually comes from one paragraph or several continuous paragraphs in a passage. So, students need to identify the main idea of each paragraph to find the relevant paragraphs, pick out the suitable words from the paragraph or paragraphs, and fill in the blanks.
Based on the above findings, the importance of identifying the topic sentence was strongly confirmed by both the quantitative (91%) and the qualitative data sets (100%). However, the variables of vocabulary size and the long sentence comprehension were proved to be more important in the qualitative results than that of in the quantitative results. Two thirds of student interviewees and all teachers in the qualitative data acknowledged the significance of these two variables, while some participants in the quantitative survey (29%) thought the two variables were less important, which suggests further study into this variance is needed.
Multiple Choice—More Than Four Choices
The main feature of this task is that the correct answers often stay together in the passage, indicating they are located not far from each other within the same section of the passage, which allows test takers to find them easily. Candidates could first determine which paragraph or paragraphs are related to the question items by understanding the topic sentence of every paragraph, and then go through the paragraph or paragraphs and pick out the correct options from the given answer choices.
The importance of the variables of vocabulary size, sentence length, and the topic sentence identification was identified and acknowledged in both the quantitative and the qualitative data. In the quantitative data, 72%, 78%, 78% of the participants recognised the importance of the vocabulary size, sentence length, and the topic sentence identification, and in the qualitative data, all interviewees recognised the importance of the three variables.
The Open-Ended Question
The open-ended question at the end of the questionnaire states ‘how do you think you could improve your performance in the authentic IELTS academic reading sub-test’. The aim of the open-ended question is to determine whether there are other factors that could also impact student performance but are not investigated in our study. 57 students (34%) wrote their comments. The most paramount finding was the significance of the vocabulary size; two thirds of the students mentioned its indispensability for improving their reading band score. Doing more reading exercises and getting more familiar with the features of each reading task and skills and strategies of dealing with different tasks were also considered to be important by a majority of the students.
5. Conclusion and Discussion
Previous research has confirmed that in the IELTS academic reading sub-test, test takers need different strategies for solving different tasks. The strategies are determined by the features of each reading task. This study has identified four variables that could have a significant impact on student performance in the test. The four variables are vocabulary size, the ability to identify and locate keywords, the ability to identify and locate the topic sentence, and the ability to comprehend long sentence structure. The significance of our study is that it is the first research that identifies and summarises the four variables together. According to the above analysis, although these four variables are shown to have different degrees of impact on candidate’s performance in the academic reading section of the IELTS test, the key finding is that the better candidates could comprehend the four variables, the better result they could achieve.
The research generates some implications for future development. The first is that our study did not measure the exact degree of influence functioned by each variable. In our study, when analysing each reading task in details, we did not identify the specific degree of their effect on student performance for each task. Finally, our research was conducted in one Chinese institution, so the findings and results may not be valid for students and teachers in other countries. Future research could identify whether the same contributing variables could also be applied to students in other countries.
The final results of our study also suggest some implications for both students and teachers in China. For students, it has clearly demonstrated three aspects where they could improve their academic reading band scores, and this information should be made available to them. First of all, students need to be made familiar with the features of every single reading task, because the reading strategies or skills used in a real test depend on candidate’s familiarity of the features of each task. For example, the task of Classification is much easier to answer by identifying the keywords rather than finding the main idea of each paragraph. If students know the features of all nine tasks well, they could use a more appropriate and effective strategy to solve the question items.
Another aspect is the student’s vocabulary size. According to our findings, the larger amount of vocabulary they know, the more they could understand each sentence, and accordingly, the better band scores they could get. Last but not the least, the ability of understanding long sentence is also very essential. If students could better understand the meaning of long sentence, they could perform well in a real test.
For teachers, the most important implication of our study is the explicit suggestions about how to improve their student performance. After identifying and confirming the impact of each contributing factor, Chinese IELTS teachers could readjust their teaching plans to specifically address the four factors more precisely in their teaching. It is hoped that these suggestions could be a valuable tool for teachers to serve Chinese students better and help them achieve their desired IELTS band score.
Our study was initially begun from one author’s own curiosity and ambition to help Chinese students achieve better IELTS results, because the test results have high stakes implications for students wishing to study overseas. Given the importance of good test results for Chinese students wishing to study abroad, we sincerely wish that our study may be a springboard to encourage more Chinese related research in the field of English language testing. We also hope that our research could assist more Chinese students get better outcomes in the IELTS reading sub-test in the future so as to realise their dream of studying abroad.