Original research: Comprehensive licensure review and adaptive quizzing assignments for enhancement of end-of-programme exit examination scores in Saudi Arabia: a quasi-experimental study (2024)

STRENGTHS AND LIMITATIONS OF THIS STUDY

• The study utilised computerised adaptive tests along with a comprehensive licensure review for senior nursing students in Saudi Arabia.

• The intervention bundle was designed based on the blueprints of the National Council Licensure Examination for Registered Nurses and Saudi Nursing Licensing Exam.

• The study lacked a control group, and all participants were recruited from one university, which could limit the generalisability of the findings.

Theoretical framework

Our study was guided by the elaboration theory, developed by Reigeluth and colleagues in 1983, which proposes that the design of educational content must follow a sequence from simple to complex.²¹ Its basic premise is a shift in teaching style from a teacher-centred to a learner-centred approach.²² The instructor organises and delivers the educational material to suit the learners’ needs and help achieve the desired learning outcomes.²¹ This theory also values a holistic sequence of instruction that can improve students’ motivation to learn. Thus, we designed the comprehensive licensure review and adaptive quizzing assignments bundle to provide educational content that can expand nursing students’ knowledge and analytical and cognitive skills. These skills are crucial to delivering safe and high-quality care.

The educational content delivered in this study incorporated the blueprint and structure of the NCLEX-RN and SNLE tests. Moreover, based on the elaboration theory, the intervention employed adaptive quizzing activities as a motivational tool for students to (1) learn from example questions, which helps them create a mental connection between new information and prior knowledge; (2) use analogies by elaborating on new information and connecting it with prior knowledge, which allows students to see patterns and relationships between concepts; and (3) aid them in comprehending new information. In general, quizzes have been used as a motivational tool to encourage students to read.^{23 24}

Furthermore, Reigeluth and Stein proposed that learners’ motivation to accomplish a predetermined objective is crucial for acquiring knowledge and skills.²¹ Thus, bundling comprehensive licensure reviews with adaptive quizzing assignments should motivate students to perform better on assignments and improve their analytical and cognitive skills. Our study sought to motivate nursing students to acquire the necessary knowledge and skills to pass the programme’s exit exam and the licensure exam, and to ultimately ensure their ability to safely handle complex healthcare needs.

Aims

The specific aims of the study were to examine (1) students’ overall mastery score and percentage of correct answers before and after a comprehensive licensure review and adaptive quizzing assignments intervention, and (2) the association of students’ demographic characteristics with their overall mastery score and percentage of correct answers, before and after a comprehensive licensure review and adaptive quizzing assignments intervention.

Study design

We utilised a quasi-experimental (pretest and post-test) single-group design. The research team followed the TREND statement checklist for non-randomised controlled trials.

Sample and setting

Nursing students were recruited from a nursing college at a public university in Saudi Arabia. The inclusion criteria were senior nursing students in the Bachelor of Science in Nursing (BSN) programme for the academic year 2021–2022. The sample size was calculated using the G-Power software V.3.1 (Heinrich-Heine-Universität, Düsseldorf, Germany). Using a set significance level of 0.05, an effect size of 0.3 and a power of 0.8, the minimum sample size was determined to be 109.

Study intervention

The study investigators designed an intervention bundle comprising a comprehensive licensure review and adaptive quizzing assignments to support senior BSN students’ transition to practice through synthesising acquired knowledge and mastering cognitive skills. Specifically, the intervention bundle aimed to help students recognise the knowledge and skills essential for registered nurses in various healthcare settings, and integrate evidence-based practice in nursing care for patients across the lifespan. The intervention bundle was also designed to foster students’ cognitive abilities in demonstrating effective clinical and decision-making skills. Students were expected to identify and manage healthcare needs across the patient lifespan and use multidisciplinary communication to enhance the healthcare provided to patients and their families. In addition, the intervention bundle was expected to promote the assimilation of professional ethical principles and values into the delivery of appropriate and culturally competent nursing care.

The intervention bundle was delivered over 15 weeks and included two main components: (1) comprehensive licensure review content presented through 3-hour synchronous lectures every week and (2) adaptive quizzing assignments (table 1). The weekly lectures were presented to reflect the content outline of the NCLEX-RN and SNLE test plans. An online adaptive testing system delivered the 23 adaptive quizzing assignments using NCLEX-RN type questions (table 1). Students were required to achieve a mastery score of 8 (highest score) in each of the 23 assignments by the end of week 15. The score fluctuated based on their sustained performance on adaptive quizzes.²⁵ Thus, the number of questions or quizzes for each assignment was not established ahead of time, as the adaptive quizzing system determined students’ performance throughout the assignments.

Intervention fidelity

The comprehensive licensure review materials were obtained from NCLEX-RN comprehensive review textbooks widely used by prelicensure graduates to prepare for the NCLEX-RN examination in USA. Furthermore, the researchers evaluated the appropriateness of using these textbooks to prepare for the SNLE by reviewing the SNLE’s blueprint against the textbook content. The adaptive quizzing assignments, pretest and post-test (ie, end-of-programme exit exam) were conducted using a valid online adaptive learning and testing source for the NCLEX-RN comprehensive licensure examination.

Study measures

Data collection

The online adaptive quizzing system was used to conduct two assessment exams at two points during the intervention. The diagnostic comprehensive exam (pretest) was conducted in the first week of the intervention to assess the students’ baseline mastery score. The end-of-programme exit exam (post-test) was conducted in the last week of the intervention to evaluate the students’ improvement (table 1). Each of the two assessment exams comprised 75 questions that had to be completed within 2 hours.

Data on participants’ mastery scores and correct answers for each exam were extracted from the online adaptive testing system. Furthermore, data on demographic characteristics were collected using an online survey sent to students at the beginning of the intervention. Data on mastery scores, correct answers and demographic characteristics were merged using the participants’ university identification number as a primary identifier across different data sheets.

Data analysis

Data were analysed using IBM SPSS Statistics for Mac V.28 software. Central tendency measures (eg, mean and median), dispersion measures (eg, SD and variance) and frequencies and percentages for categorical measures were used to present the descriptive results. A paired sample t-test was used to evaluate the mean difference in students’ performance before and after the intervention. The differences in the study variables (ie, mastery scores and percentages of correct answers) according to demographic characteristics were measured using an independent sample t-test and Pearson’s product–moment correlation. Statistical significance was set at p<0.05.

Patient and public involvement

The study did not involve patients or the public.

Table 2

Comparison of students’ mean scores before and after the intervention (n=287)

Table 3

Comparison of groups’ mean score before and after the intervention (n=287)

Supplementary data

bmjopen-2023-074469supp001.pdf

Implications and recommendations

The study findings have important implications for nursing academic administrators and educators to improve students’ performance in the nursing end-of-programme exit exam. Along with providing the knowledge and skills needed for nursing practice, academic programmes should integrate a comprehensive licensure review bundled with adaptive quizzing to foster students’ cognitive skills and improve their performance in the end-of-programme examination. Implementing this strategy can ultimately prepare students for the next step in their nursing career following graduation, such as attempting the licensure examination. Moreover, the gender performance differences in the end-of-programme exit exam warrant further evaluation from nursing educators to ensure that both male and female students excel in the exam.

Additionally, the study findings indicated that students with higher GPAs performed better in the pretest and end-of-programme exit exams compared with other students. Accordingly, faculty members should ensure that adequate academic counselling is available for students who have underperformed in the baseline test. Furthermore, programme directors could monitor students’ performance throughout the programme and consider providing additional remediation activities for students with lower GPAs prior to the exit exam. Monitoring students’ performance would help to ensure their readiness for and success in the exit exam. Moreover, this study showed that younger students performed better in the exit exam compared with older students, which warrants further evaluation to identify the influence of age on students’ performance in the exam. Future studies should also evaluate the impact of a comprehensive licensure review and adaptive quizzing assignments bundle on students’ performance in the SNLE.

Limitations

Our study had some limitations. First, it did not have a control group for comparison, which may have affected the internal validity of the study intervention. Second, the sample was restricted to one university, which could limit the generalisability of the findings due to possible differences in students’ characteristics. Future large-scale studies should include a control group and participants from different universities to reach generaliable conclusions.

Conclusion

We examined the effectiveness of comprehensive licensure reviews and adaptive quizzing assignments to improve students’ performance in the end-of-programme exit exam. The intervention improved students’ performance through their achievement of higher mastery scores and percentages of correct answers in the exit exam. The findings suggest that a comprehensive licensure review and adaptive quizzing assignments can foster students’ knowledge and cognitive skills to facilitate a successful transition to nursing practice.

Reviewer comments:

Author'smanuscript:

Twitter: @malmotairy1, @DrAdnanAlmanie, @Naji_PhD, @AhmNahari, @Dr_ReemSaeed

Contributors: MMA, AI, AN and NA conceived the investigation; MMA and RA designed and delivered the investigation, MMA, DA, NA, AN, AI and HM managed the collection of data through pre-exams and post-exams; and MA and AI conducted the analysis. All authors drafted the manuscript and contributed to the interpretation and intellectual revision of the manuscript and agreed on the final version. MMA is the author acting as a guarantor in this study.

Funding: This research received funding from the Deputyship for Research and Innovation, Ministry of Education in Saudi Arabia (IFKSUOR3–091–3).

Competing interests: None declared.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Patient consent for publication

Not applicable.

Ethics approval

This study was approved by the Standing Committee for Scientific Research Ethics at King Saud University (Ref No. KSU-HE-22-001).

2. National Council of State Boards of Nursing . What you need to know about nursing Licensure and boards of nursing [online]. n.d.Available: https://www.ncsbn.org/Nursing_Licensure.pdf

5. Al-Alawi R, Alexander GL. Systematic review of program evaluation in baccalaureate nursing programs. J Prof Nurs2020;36:236–44. 10.1016/j.profnurs.2019.12.003 [PubMed] [CrossRef] [Google Scholar]

6. Giddens JF. Changing paradigms and challenging assumptions: redefining quality and NCLEX-RN pass rates. J Nurs Educ2009;48:123–4. 10.3928/01484834-20090301-04 [PubMed] [CrossRef] [Google Scholar]

7. Jeffreys MR. Jeffreys's nursing universal retention and success model: overview and action ideas for optimizing outcomes A-Z. Nurse Educ Today2015;35:425–31. 10.1016/j.nedt.2014.11.004 [PubMed] [CrossRef] [Google Scholar]

8. Roa M, Shipman D, Hooten J, et al.. The costs of NCLEX-RN failure. Nurse Educ Today2011;31:373–7. 10.1016/j.nedt.2010.07.009 [PubMed] [CrossRef] [Google Scholar]

9. Thompson CW. Retaining graduate nurses who experience NCLEX failure: recommendations for supporting retest success. Nurse Leader2023;21:299–302. 10.1016/j.mnl.2022.11.013 [CrossRef] [Google Scholar]

11. Kasprovich T, VandeVusse L. Registered nurses’ experiences of passing the NCLEX-RN after more than one attempt. J Nurs Educ2018;57:590–7. 10.3928/01484834-20180921-04 [PubMed] [CrossRef] [Google Scholar]

12. Lutter SL, Thompson CW, Condon MC. Tutoring for success: empowering graduate nurses after failure on the NCLEX-RN. J Nurs Educ2017;56:758–61. 10.3928/01484834-20171120-11 [PubMed] [CrossRef] [Google Scholar]

13. McGillis Hall L, Lalonde M, Kashin J. People are failing! something needs to be done: Canadian students' experience with the NCLEX-RN. Nurse Educ Today2016;46:43–9. 10.1016/j.nedt.2016.08.022 [PubMed] [CrossRef] [Google Scholar]

14. Meehan CD, Barker N. Remediation for NCLEX-RN success in high-risk nursing students. Teaching Learning Nurs2021;16:254–7. 10.1016/j.teln.2021.02.003 [CrossRef] [Google Scholar]

15. Quinn BL, Smolinski M, Peters AB. Strategies to improve NCLEX-RN success: a review. Teaching Learning Nurs2018;13:18–26. 10.1016/j.teln.2017.09.002 [CrossRef] [Google Scholar]

16. Czekanski K, ho*rst BJ, Kurz J. Instituting evidence-based changes to improve first-time NCLEX-RN® pass rates. J Nurs Regulat2018;9:11–8. 10.1016/S2155-8256(18)30049-8 [CrossRef] [Google Scholar]

17. Czekanski K, Mingo S, Piper L. Coaching to NCLEX-RN success: a Postgraduation intervention to improve first-time pass rates. J Nurs Educ2018;57:561–5. 10.3928/01484834-20180815-10 [PubMed] [CrossRef] [Google Scholar]

18. Hyland JR. Building on the evidence: interventions promoting NCLEX success. OJN2012;02:231–8. 10.4236/ojn.2012.23036 [CrossRef] [Google Scholar]

19. Conklin PS, Cutright LH. A model for sustaining NCLEX-RN success. Nurs Educ Perspect2019;40:176–8. 10.1097/01.NEP.0000000000000326 [PubMed] [CrossRef] [Google Scholar]

20. Almarwani AM. The effect of integrating a nursing licensure examination preparation course into a nursing program curriculum: a quasi-experimental study. Saudi J Health Sci2022;11. 10.4103/sjhs.sjhs_87_22 [CrossRef] [Google Scholar]

21. Reigeluth CM, Stein FS. The elaboration theory of instruction. In: Reigeluth CM, ed. Instructional-design theories and models: an overview of their current status. New York: Routledge, 1983: 335–81. [Google Scholar]

22. Wilson B, Cole P. A critical review of elaboration theory. ETR&D1992;40:63–79. 10.1007/BF02296843 [CrossRef] [Google Scholar]

23. Lei SA, Bartlett KA, Gorney SE, et al.. Resistance to reading compliance among college students: instructors’ perspectives. Coll Stud J2010;44:219–29. [Google Scholar]

24. Stratton G. Does increasing textbook Portability increase reading rates or academic performance?Inquiry2011;16:5–16. [Google Scholar]

26. Simon-Campbell EL, Phelan JC. Effectiveness of an adaptive quizzing system as an institutional-wide strategy to improve student learning and retention. Nurse Educ2016;41:246–51. 10.1097/NNE.0000000000000258 [PubMed] [CrossRef] [Google Scholar]

27. Parcell H, Morton K, Froble D, et al.. Evaluating the effect of pre-exam adaptive quizzing on nursing student exam scores. Nurs Educ Perspect2022;43:E100–2. 10.1097/01.NEP.0000000000000988 [PubMed] [CrossRef] [Google Scholar]

28. Presti CR, Sanko JS. Adaptive quizzing improves end-of-program exit examination scores. Nurse Educ2019;44:151–3. 10.1097/NNE.0000000000000566 [PubMed] [CrossRef] [Google Scholar]

29. Cox-Davenport RA, Phelan JC. Laying the groundwork for NCLEX success: an exploration of adaptive quizzing as an examination preparation method. Comput Inform Nurs2015;33:208–15. 10.1097/CIN.0000000000000140 [PubMed] [CrossRef] [Google Scholar]

30. Oliver BJ, Pomerleau M, Potter M, et al.. Optimizing NCLEX-RN pass rate performance using an educational Microsystems improvement approach. J Nurs Educ2018;57:265–74. 10.3928/01484834-20180420-03 [PubMed] [CrossRef] [Google Scholar]

31. Butcon VE, Pasay-An E, Indonto MCL, et al.. Assessment of determinants predicting success on the Saudi nursing Licensure examination by employing artificial neural network. J Educ Health Promot2021;10:396. 10.4103/jehp.jehp_652_20 [PMC free article] [PubMed] [CrossRef] [Google Scholar]

33. Havrilla E, Zbegner D, Victor J. Exploring predictors of NCLEX-RN success: one school’s search for excellence. J Nurs Educ2018;57:554–6. 10.3928/01484834-20180815-08 [PubMed] [CrossRef] [Google Scholar]