1. Full name: Nguyen Thi Quynh Yen

2. Sex: Female

3. Date of birth:  11th March 1981                                                                  

4. Place of birth: Thanh Hoa

5. Admission Decision number: 2019/QĐ-ĐHNN             Dated: 31/12/2014  

6. Changes in academic process: No changes

7. Official thesis title: An Investigation into the Cut-score Validity of the VSTEP.3-5 Listening Test

8. Major: English Language Teaching Methodology                                                        

9. Code: 9140231

10. Supervisors: 1. Prof. Nguyen Hoa            

                            2. Prof. Fred Davidson        

11. Summary of the new findings of the thesis:

This doctoral research is conducted as an ongoing attempt in building a systematic, transparent and defensible body of validity argumentation for the VSTEP.3-5 listening test. By adopting the argument-based approach recommended by Kane (2013), the study aims at investigating the cut-score validity of the VSTEP.3-5 listening test. This study only focuses on the validation of the cut-scores of a VSTEP.3-5 listening test administered in early 2017 by one institution permitted by MOET to build and administer the VSTEP.3-5 test. Here are some important findings of the study:

First, the test specifications of the VSTEP.3-5 listening test reasonably provide item-writers with the good guidance in building test forms. In general, the test tasks of the VSTEP.3-5 listening test administered in early 2017 are properly designed in accordance with the characteristics specified in the specifications and the successful completion of the test tasks is dependent on the comprehension of the text. However, the analytical data about the item difficulty and item discrimination show that there is still space for some improvement in the specifications of the VSTEP.3-5 listening test. To be more specific, more details about the difficulty level and the use of language in each item should be clearly explained so that the items can be designed with the right intended level of difficulty. Before being used in the official test forms, the test items need to be piloted if affordable in a scale of 200 persons or otherwise piloted on a small scale of 50 so as to make sure that all of the items satisfy the expected difficulty levels and can discriminate test-takers well.

Second, the reported reliability of the VSTEP.3-5 listening test in general and that administered in early 2017 is rather high with around 0.815.  This index has not yet reached the perfect number for a proficiency test with 3 cut-off points. Thus, more measures need taking to improve the test reliability.  

Third, the results of the cut-scores for level 3 and level 5 established for the VSTEP.3-5 listening test administered in early 2017 by the Bookmark standard-setting method correlate relatively well with those preset by Angoff method. It is plus and minus one for level 3 and 5 respectively, which can be seen acceptable. As for level 4, there is a 3-raw-point gap between the results of two standard setting methods, which is probably significant. The suggestion is that more research should be conducted and more evidence should be collected before any recommendations could be made for the change in the cut score for level 4.

12. Practical applicability, if any:

First, the interpretive argument developed for the cut score validity of the VSTEP.3-5 listening test  can be applied to the studies into the cut-score validity of other components of the VSTEP.3-5 test or any other high-stakes tests in Vietnam such as the VSTEP.2 test and VSTEP.3 test for school.

Second, the combination of methods ulitized in this study can be useful in improving the quality of not only English proficiency tests but also classroom tests in Vietnam.  

Third, the results of this study can be used to improve the specifications and item writer guidelines for the VSTEP.3-5 listening test in particular and the VSTEP.3-5 in general.

13. Further research directions, if any:

First, with regards to the MCQ test format like the VSTEP.3-5 listening test, a study could be conducted to make an investigation into the processes activated and strategies employed by the test-takers when taking the VSTEP.3-5 listening test and how these listening processes and strategies are related with the test-takers’ success in solving the listening comprehension items.

Second, further studies into the VSTEP.3-5 listening test may be expanded into other listening test booklets in order to build up a comprehensive validity argument for the VSTEP.3-5 listening test in particular and the VSTEP.3-5 test in general.

Third, in order to build a systematic, transparent and defensible body of validity argument for the VSTEP.3-5 test in general, further studies should be conducted into the validity of the VSTEP.3-5 Speaking test, the VSTEP.3-5 Reading test and the VSTEP.3-5 Writing test. 

Besides, in terms of the standard setting, a number of other studies could be suggested for the VSTEP.3-5 listening test as follows: (1) the reliability and validity of the placement decisions based on the cut-scores of VSTEP.3-5 listening test results can be taken into consideration by making some correlation analyses with other relevant criteria such as the grades of the test-takers on a English course or the test-takers’ performances on other tests of the same measurement, (2) the agreement of the cut-scores set for the results of the VSTEP.3-5 listening test can be investigated across replications by using other standard-setting methods, (3) a study could be about how the perspectives of different participants such as academic experts, teachers and other test users influence the results of cut-scores for the VSTEP.3-5 listening test.  

Finally, since several institutions throughout Vietnam are being permitted by the Ministry of Education and Training to develop and administer the VSTEP.3-5 test, various studies could and should be conducted into the above-mentioned issues in these  institutions and across institutions.

14. Thesis-related publications:

Nguyễn Thị Quỳnh Yến. (2015). Ensuring fairness in testing listening. Tạp chí Ngôn ngữ học và đời sống, 12(242), pp.36-39. National Political Publishing House.

Nguyễn Thị Quỳnh Yến. (2017). Một số vấn đề cần xem xét trong đánh giá tính giá trị của các điểm cắt giữa các bậc năng lực cho bài thi đánh giá năng lực tiếng Anh theo khung năng lực ngoại ngữ 6 bậc dành cho Việt Nam. Kỷ yếu hội thảo quốc gia 2017- Nghiên cứu và giảng dạy Ngoại ngữ-Ngôn ngữ-Quốc tế học tại Việt Nam, pp. 612-618. VNU Press.

Nguyễn Thị Quỳnh Yến. (2017). Building a validity argument for the Vietnamese Standardized Test of English Proficiency (VSTEP.3-5). Kỷ yếu hội thảo quốc gia dành cho Học viên cao học và nghiên cứu sinh lần thứ nhất, pp.705-712. VNU Press.