Validity and reliability of the augmented-reality simulator as an assessment tool in dental education

• Zalim Z. Balkizov
• Hossam Hamdy
• David Christopher Morgan Taylor

Abstract

Background. The assessment of psychomotor skills in dental education is traditionally based on subjective evaluations by instructors, which can lead to inconsistency and bias. The integration of augmented reality (AR) simulators offers a potential solution by providing objective, real-time feedback. However, the validity and reliability of AR-based assessment tools remain underexplored. This study investigates the effectiveness of an AR dental simulator in evaluating students’ cavity preparation skills compared to expert assessments.

Aims and objectives. This study aims to evaluate the validity and reliability of an AR simulator as an assessment tool in dental education. Specifically, it seeks to:

Compare AR simulator-generated scores with expert evaluations.

Assess the consistency and objectivity of simulator-based feedback.

Determine the simulator’s potential to enhance skill acquisition through real-time assessment.

Material and methods. Twenty first-year dental residents participated in this study, performing cavity preparations using the BeDentPro AR simulator. Their work was assessed by five experienced faculty members using a standardized grading rubric. The simulator provided automated measurements of cavity dimensions, preparation accuracy, and procedural efficiency. Statistical analysis, including Kendall’s Coefficient of Concordance and Spearman’s correlation, was conducted to compare expert and simulator evaluations.

Results. High correlation (r>0.7) was observed between the simulator’s evaluations and expert assessments. The simulator provided precise, quantifiable data on parameters such as cavity depth, extension, and preparation time. Additionally, it identified errors such as over-preparation, excessive tissue removal, and deviations from ideal angulation.

Discussion and conclusion. The findings demonstrate that AR simulation provides valid and reliable assessments comparable to expert evaluations while offering additional quantitative insights. By eliminating subjectivity, AR-based assessment tools can enhance competency-based training in dentistry. Future research should explore their integration into summative evaluations and broader dental curricula.

Keywords: validity; assessment; augmented reality; dental simulation

Future dentist training is a significant autonomous area in Health Professions Education (HPE), represented by a multidimensional complex process (Iacopino, 2007). Curricula in the current practice field vary between different countries (Kumar, 2017). Meanwhile, the commonalities include a similar structure that can be described as transitioning from receiving basic knowledge or learning fundamental sciences to preclinical training using simulation and escalating to supervised performance on real patients (Buchanan, 2001).

Dentist training in Russia includes five years of primary education according to the traditional curriculum, 2-3 years of clinical residency, and continuing professional development (CPD) (Veselkova 2018). Since 2015 all health professionals willing to practice on the territory of the Russian Federation are subject to a licensing examination that includes multiple choice question (MCQ) testing, an objective structured clinical examination (OSCE), and an interview (Kagramanyan et al., 2021). Simulation-based education is one of the biggest trends announced in HPE in Russia (Veselkova 2018). Currently, every dental program delivered within a school reporting to the Mini­stry of Health of Russia includes training in simulation centers for the mastery of clini­cal skills. The importance of introducing digital technologies to future dentist training in the country appears to be undisputable (Kagramanyan et al., 2021).

Several publications are dedicated to the needs of the present HPE in Russia. Specific issues related to such a substantial element of the educational process as assessing the competence of health professionals in the country have been outlined. For example, researchers claim that the information produced by existing examinations does not allow one to judge junior specialists' readiness for independent practice objectively (Chelyshkova et al., 2018). A possible reason behind this limitation is the insufficient use of qualitative data in the assessment of HPE in Russia (Dorozhkin et al., 2016).

This study is about the training of dentists in dental schools and how it was changed over time to the current training, which incorporates simulation in training and assessment.

Problem statement and aim of the study

Assessment of procedures in dental education is commonly based on the perception of the senior dentists or faculty - trainer while observing a procedure performed by a student or a resident, on a real patient or in simulation environment. The existing visual evaluation of the quality of the pre­paration of the cavity of a student by teachers is entirely dependent on observation by the naked eye and judgment based on experience.

This observation is transformed into a score, which is usually categorized as "meet expectations" of the assessors or "exceed expectations," or "below the expectations," i.e. not competent in this procedure. The common procedure all dentists should be competent in doing it is the preparation of the cavity.

Meanwhile, the degree of precision in operative dentistry may play a substantial role when the 0.5‑mm or 1 angle error may appear critical and irreversible to the patient. Cavity preparation may cause a range of complications, such as excessive removal of dental tissues and damage to neighboring teeth and soft tissues (gum, tongue, etc.). In prosthodontics, the ability to prepare teeth to narrow tolerances can affect the longevity of restorations.

Another issue related to assessment in dental education is associated with the growing number of students admitted to dental schools and the low teacher-student ratio. Sometimes it poses difficulties in providing detailed feedback to all learners, especially when all grading for the whole group should be completed in a single session.

This study hypothesized that precise evaluation of psychomotor skills and detailed in-time feedback is essential in dental education and can be achieved by high-fide­lity simulators.

Methods

In this study, a prospective quantitative correlation method was used in the evaluation of the validity and reliability of the dental Simulator (BeDentPro of the BeMedSkilled company) in measuring cavity preparation by dental residents. Ethical permission was obtained from the GMU IRB.

Twenty first-year residents were selected for this study. This constitutes the total number of first-year dental residents in a two-year residency program at the Russian National Medical Research Center for Dentistry. Participants had previous cavi­ty preparation training during undergraduate training programs in dental schools.

Five assessors participated in the study; all are practicing dentists, and two are faculty members of the Russian National Medical Research Center of Dentistry. For calibration of assessors, instruction and hands-on session was performed.

All twenty participants attended an information session at which the purpose of the study was explained, and a lecture on the preparation of a cavity was given, and signed the Consent form. Residents were given an introduction via slides and 3D animations, which showed the whole procedure from different angles and demonstrated the indented result of the procedure. They also received instructions from trained faculty members prior to the procedure.

The sessions were conducted in 15-mi­nute sessions on two simulators simultaneously. In each session, participants were gi­ven the same set of instructions with a maxi­mum of 15 minutes for preparation.

After that, participants were asked to perform the preparation of tooth number 16 according to the FDI classification (Fig. 1), using class II Black's Classification of Caries Lesions (Fig. 2).

Procedure

Dental Simulator used for the research is BeDentPro (2^nd version of Leonardo-dental), a multi-task AR simulator teaching and assessing clinical competencies integrated in a realistic environment with accurate 3D visualization (Fig. 3). The Simulator has a magnetic-field motion-tracking system with an accuracy of about 0.3 mm in three dimensions and 0.1 degrees of inclination in three axes at 800 Hz. Using the accurate 6DOF motion tracking system, the dental tool is tracked and measured with sub-millimeter accuracy. This is a critical element of the objective assessment outcome, as a precise skill assessment can be made using the data provided by the tracking system.

Tiny microsensors are seamlessly integrated into actual dental tools used in the clinical setting. Due to the nature of the AC Electromagnetic Tracking, system components such as the sensors and the source are fully embeddable.

Calculations of the progress of the procedure are based on the interaction of the bur tip and teeth model in the virtual 3D space. All 3D models are made up of voxels¹ which can be described as three-dimensional matrix of numbers, where each value represents a pixel. 3D models exactly replicates physical models of the teeth. Whenever the physical bur perforates physical model of the tooth, software deletes values of the voxels corresponding to the eliminated particles from the tooth model.

¹ A voxel, short for "volume pixel," is the three-dimensional (3D) equivalent of a pixel in two-dimensional (2D) space. Voxels are extensively used in the fields of medical imaging (like MRI and CT scans), computer graphics, and volumetric rendering. In medical imaging the value of a voxel might correspond to a range of densities allowing tissues to be differentiated from each other. In computer graphics, they are used for creating 3D models and scenes, especially in video games and simulations.

All calculation immediately reflects on the 3D model in the main window and on the tooth model in the area of the dynamics of preparation (Fig. 4, 5). User can easily rotate, zoom, and move both images. Evaluation window, where user can compare current state of the tooth to the optimal model layer by layer in three dimensions is available during and after the procedure (Fig. 6).

After the session simulator gives accurate measurements of the preparation performed on the teeth models, using actual dental tools provides students with the same look and feel in natural clinical settings (Fig. 6).

In addition to tracking the teeth preparation, the Simulator enables other procedures, including the selection of anesthesia type specific to the patient, based on the questionary, performance of anesthesia, and virtual X-ray (Fig. 7), which reflects the current state of the tooth. In addition, a video record of the sessions from the lamp position and 3D record are stored for debriefing to discuss the entire process of the preparation, rather than just the outcome, and posture of the student during the procedure.

After the preparation, the tooth model was extracted and sealed in a separate plastic bag labeled with the participant id and provided to assessors who were asked to evaluate the preparation using criteria based on measurement of specific para­meters in the checklist.

Measurements used as criteria are as shown in Fig. 8:

· outline consistency;

· mesiodistal extension;

· buccal-lingual extension;

· width of isthmus;

· depth;

· internal shape consistency;

· depth of axial wall;

· width of the isthmus of the axial wall.

Assessment

A checklist (Table 1) for the assessment was developed using standard evaluation metrics. Each preparation was assessed independently by the Simulator and five examiners using an identical scoring system. Examiners were blinded to the residents' identity, data from the Simulator, and the exa­miner.

Examiners assessed the residents using traditional visual observation and the checklist, which contains seven individual criteria that have an associated specific point value and are added together to generate the total score.

The rubrics on the grading sheet are outline, Mesiodistal extension, Buccal-Lingual extension, width, depth, Internal shape, depth, and width of the axial wall. The total score is calculated as a sum of points from all rows.

Results

Reliability. The Kendall's Coefficient of Concordance (W) is used to determine the consistency among experts. A higher W value indicates greater agreement. The results show that all W scores were above 0.5 and were statistically significant (p<0.001). Kendall's W ranges from 0 (no agreement) to 1 (complete agreement). Therefore, for W>0.5, justifies agreement of expert opinions (Table 2).

The Spearman correlation (Rs) provides the strength of the relationship between the rankings of the assessment methods. Rs values ranging between 0.366 and 0.645 indicate different levels of correlation among the assessment methods. The intermediate level of correlations (Rs>0.5) was found for Outline, M-D Extension, Depth, and Internal shape; moderate was Rs (0.366-0.455) for B-L Extension, Width, Depth of axial wall, Width of the axial wall.

Kendall's Coefficient of Concordance test was performed. All W values (separately for each of the eight parameters) were higher than 0.4 at p<0.001. The obtained data allow us to conclude about the significant consistency of all used methods of assessment (experts and Simulator) of residents' knowledge and skills. Thus, the Simulator is at least as good as the expert assessment (Table 3).

Validity. This comparative analysis shows that the Simulator's evaluation is comparable (valid) with the experts' assessments. There's no statistically significant difference, suggesting that both the experts and the Simulator are equally valid in their assessments (Table 4).

A comparative analysis of the experts’ and Simulator’s assessments (U-test Mann-Whitney) also showed their comparability (Table 5). On average (median), residents received a "satisfactory" result on all eight elements of the assessment components. No statistically significant differences existed between the expert assessments and the Simulator data (p>0.05).

The results show scores range from 9 to 22 across the different examiners and the Simulator. The Simulator's average score is between the examiners' scores, suggesting that it is neither too lenient nor too strict (Fig. 9, Tables 6, 7). The potential objectivity of the Simulator is highlighted since it lacks human bias.

Correlation analysis performed by Spearman and Tau-b Kendell methods established the presence of significant correlations of all assessment methods. The strongest correlations (r>0.7) were recorded between the total score of the Simulator and Exami­ners 1, 2, and 3, that is, with the majority of experts (Table 8).

The final step was a correlation analysis of the rank of the sum of the scores of all experts and the Simulator. A correlation analysis of the total score/rank of the expert's assessment and the Simulator's assessment revealed a robust direct correlation between the Simulator score and the Average Expert Total Score (r=0.805; p<0.001) (Fig. 10).

With the comparability of the estimated results of all eight components of the final assessment of residents, the Simulator had the advantage of being able to determine the exact size of the estimated index (Table 9). The latter may testify in favor of the greater measuring accuracy of the Simulator, which performs the evaluation based on precisely defined measurements, excluding subjective evaluation and the human factor.

Validity. The Simulator provides an objective measurement for 6 out of 8 evaluation criteria. This means that the Simulator can give exact measurements for most of the criteria, highlighting its potential validity as an assessment tool.

The results, as depicted in Table 11, represent the precision with which the Simulator evaluates the performance of residents based on specific criteria like "M-D Extension", "B-L Extension", and others.

The statement that the Simulator "excludes subjective evaluation and the human factor" indicates that its measurements are unbiased and purely based on the data, which can increase the validity of its evalua­tions.

Reliability. The Simulator seems to consistently provide measurements within a certain range, denoted as the Interquartile Range (IQR). IQR represents the middle 50% of data, which means the Simulator consistently measures within this range, suggesting a high degree of reliability.

The data further breaks down into categories of "excellent", "satisfactory" and "unsatisfactory" based on predefined ranges for each criterion. For instance, in the case of "M-D Extension", students who achieved measurements within 1.2-1.3 mm were considered excellent, while those in the range of 1.4-1.6 mm were deemed satisfactory. This categorical breakdown provides a consistent scale of measurement, further establishing reliability.

For "M-D Extension": most residents (50%) scored satisfactory with measurements between 1.4-1.6 mm, while 40% received unsatisfactory scores.

For "B-L Extension": the majority of residents either scored excellent (35%) or satisfactory (40%). Only a small fraction (25%) received unsatisfactory scores.

Similarly, for the other criteria ("B Width", "Depth", "Depth-II of the axial wall", and "Width-II of the axial wall"), the results varied, but the Simulator consistently measured and categorized the scores.

In summary, we can state the potential benefits of using a Simulator for evaluating dental procedures. The Simulator's capability to provide exact measurements and categorize performance objectively suggests its high validity and reliability as an assessment tool.

DISCUSSION

This study investigated the validity and reliability of an augmented reality (AR) simulator in assessing cavity preparation skills among dental residents. The findings suggest that the simulator provides consistent and objective evaluations comparable to those of experienced faculty, thus supporting its potential as a robust tool for formative and possibly summative assessment in dental education. However, a more critical examination reveals both opportunities and limitations.

The reliability of expert evaluations, as evidenced by Kendall’s W values exceeding 0.5 across all domains, indicates substantial inter-rater agreement. Nevertheless, it is noteworthy that variations among assessors still emerged, with some assessors demonstrating a pattern of leniency or stringency. These discrepancies underscore a persistent challenge in clinical education - the subjective influence of individual assessor judgment - even when standardized rubrics are employed. The simulator’s capacity to neutralize such bias, by relying on precisely defined quantitative metrics, represents a substantial advancement.

The strong correlation (r>0.7) between simulator-generated scores and those of three out of five experts supports the tool's criterion-related validity. However, moderate correlations in certain domains (e.g., width of axial wall, B-L extension) suggest that not all dimensions of clinical performance are captured with equal precision. It is possible that certain nuances, such as tactile feedback, angulation quality, or subtle anatomical considerations, remain better perceived by human evaluators.

An important strength of the simulator is its ability to generate fine-grained, objective measurements (e.g., cavity depth, isthmus width) in real time. This granularity provides learners with immediate, actionable feedback - a pedagogically valuable feature that is often difficult to replicate in traditional settings due to faculty workload and limited instructional time. Nevertheless, the overreliance on measurable parameters risks neglecting contextual and integrative aspects of clinical judgment. For instance, the simulator may accurately detect over-preparation but not necessarily discern why the error occurred (e.g., poor ergonomic posture, inappropriate instrument selection).

Despite the favorable comparison between simulator and expert assessments, the current study has several limitations. The sample size, while representative of a single cohort, is small and institution-specific, limiting generalizability. Furthermore, the assessors' calibration process was not described in sufficient detail to evaluate consistency across examiners beyond their scoring patterns. Additionally, the study does not report intra-rater reliability, nor does it explore the impact of repeated simulator use over time - both of which are critical for establishing tool stability and learning curve effects.

Another limitation is the lack of qualitative feedback from students or assessors regarding their perceptions of the simulator’s usability, realism, or pedagogical value. While objective data are central to validating assessment tools, user experience and perceived fidelity can influence engagement and skill transferability in simulation-based education.

Finally, while the simulator performed comparably to expert assessments, its use should not be viewed as a replacement but rather a complement to human judgment. A hybrid model - leveraging the precision of technology alongside the interpretive capacity of expert educators - may offer the most effective framework for clinical skills assessment.

Recommendations

The assessment obtained on the Simulator agrees and directly correlates with the experts' assessments. The Simulator had the advantage of being able to quantify the exact size of the preparations as well as give additional quantitative parameters of effective preparation time, percentage of caries and healthy tissue removal, and pulp damage event, determined with high accuracy (to 0.5 mm), which no expert can do. In addition, the Simulator completely levels out the human factor and can perform both evaluation and training around the clock.

CONCLUSION

This study demonstrates that the augmented reality simulator provides a reliable and valid method for assessing cavity preparation skills in dental residents. The high degree of agreement between simulator outputs and expert evaluations supports its use as a credible assessment tool within dental education. The simulator’s ability to deliver precise, real-time, and objective feedback addresses well-documented challenges in manual skill assessment, such as rater variability and limited faculty availability.

However, while the simulator offers significant benefits in terms of standardization and efficiency, it does not fully replicate the nuanced judgment and holistic insight of experienced clinicians. Certain aspects of psychomotor performance - such as ergonomics, hand positioning, and clinical reasoning - may still require human interpretation and mentorship. Thus, its greatest utility may lie in supporting, rather than replacing, expert evaluation.

The integration of simulation-based assessment should be approached thoughtfully. Institutions are encouraged to adopt a blended approach that combines the objectivity and scalability of AR technology with the experiential and reflective dimensions of faculty-guided evaluation. Future research should focus on longitudinal outcomes, learner satisfaction, and the simulator’s performance in high-stakes or summative contexts.

In summary, while the simulator represents a promising advancement in dental education assessment, its effectiveness will depend on careful implementation, ongoing validation, and integration within a broader educational strategy that values both technology and human expertise.

RECOMMENDATIONS

Based on the findings of this study and a critical evaluation of its implications, the following recommendations are proposed for educators, curriculum designers, and institutional decision-makers:

Integrate Simulation into Multimodal Assessment Frameworks. The AR simulator should be incorporated as part of a comprehensive assessment system that combines objective digital data with structured human evaluation. This hybrid model will ensure that both quantifiable technical accuracy and qualitative clinical reasoning are assessed.

Use Simulator Data to Enhance Formative Feedback. Institutions should priori­tize the simulator's strengths in formative settings. Real-time, granular feedback can support deliberate practice, self-assessment, and remediation - key components of skill acquisition in competency-based education.

Develop Faculty Calibration and Oversight Protocols. While simulator outputs are objective, human assessors remain vital. Regular calibration and training should be implemented to ensure that faculty interpretations align with both rubric criteria and simulator benchmarks.

Invest in Simulator Literacy and Trai­ning. Both students and instructors require orientation not only in technical use but also in interpreting simulator-generated metrics meaningfully. Structured workshops and user guides should accompany implementation.

Pilot Simulator Use in Low-Stakes Summative Assessments. Before full-scale adoption in high-stakes exams, institutions should pilot the use of simulator evaluations in lower-stakes summative contexts. This allows for adjustment, user feedback collection, and evidence accumulation regarding predictive validity.

Conduct Further Research on Educational Impact and Cost-Benefit. Future studies should investigate the long-term educational impact of simulator use on clinical performance, decision-making, and patient safety. Economic analyses are also recommended to evaluate cost-effectiveness relative to traditional methods.

In conclusion, this research illuminates the credibility and robustness of Simulator in assessing residents. The significant correlations and consistency among experts and between experts and the Simulator suggest that the Simulator is an equally valid and potentially more objective tool for assessment. As we move towards an increasingly digitized era, tools like the Simulator can serve as valuable assets in educational and evaluative domains, provided they maintain the standards of reliability and validity demonstrated in this research.

In essence, the research underscores the significant potential of the Simulator as a reliable and valid assessment tool, matching, and in some cases potentially surpassing the evaluations given by human experts. Proper integration and continuous updates will ensure that it remains a vital tool for assessments in the future.

REFERENCES

1.Al-Saud L.M., et al. Feedback and motor skill acquisition using a haptic dental simulator. Eur J Dent Educ. 2017; 21 (4): 240–7. DOI: https://doi.org/10.1111/eje.12214

2.Bolotov V., Lenskaya E. The reform of education in new Russia: a background report for the OECD review of Russian education. ISRE Newsletter on East European, Eurasian, and Russian Education. 1997; 6 (1): 4–21.

3.Boud D., Molloy E. Feedback in higher and professional education: Understanding it and doing it well. 2012. DOI: https://doi.org/10.4324/9780203074336

4.Buchanan J.A. Use of simulation technology in dental education. J Dent Educ. 2001; 65 (11): 1225–31. DOI: https://doi.org/10.1002/j.0022-0337.2001.65.11.tb03481.x

5.Burrer P., et al. Real-time feedback of the applied light-curing technique and its impact on degree of conversion of composite restorations – a study with undergraduate dental students. J Pers Med. 2021; 11 (10): 1012. DOI: https://doi.org/10.3390/jpm11101012

6.Chandratilake M., Davis M., Ponnamperuma G. Evaluating and designing assessments for medical education: the utility formula. Internet J Med Educ. 2009; 1 (1): 1–17.

7.Chelyshkova M.B., et al. Сross-analysis of big data in accreditation of health specialists. Electron J Gen Med. 2018; 15 (5): em72. DOI: https://doi.org/10.29333/ejgm/93469

8.Choi Y.F., Wong T.W. High-fidelity simulation training programme for final-year medical students: implications from the perceived learning outcomes. Hong Kong Med J. 2019; 25 (5): 392–8. DOI: https://doi.org/10.12809/hkmj197898

9.Dorozhkin E.M., et al. Innovative approaches to increasing the student assessment procedures effectiveness. Int J Environ Sci Educ. 2016; 11 (14): 7129–44.

10. Downing S.M. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003; 37 (9): 830–7. DOI: https://doi.org/10.1046/j.1365-2923.2003.01594.x

11. El-Kishawi M., Khalaf K., Winning T. Understanding motor skill learning as related to dentistry. Dent J. 2021; 9 (6): 68. DOI: https://doi.org/10.3390/dj9060068

12. Froumin I., Remorenko I. From the «best-in-the world» soviet school to a modern globally competitive school system. In: Audacious Education Purposes. Springer, 2020: 233–50. DOI: https://doi.org/10.1007/978-3-030-41882-3_9

13. Gottlieb R., et al. Predicting performance in technical preclinical dental courses using advanced simulation. J Dent Educ. 2017; 81 (1): 101–9. DOI: https://doi.org/10.1002/j.0022-0337.2017.81.1.tb06252.x

14. Halsband U., Lange R.K. Motor learning in man: a review of functional and clinical studies. J Physiol Paris. 2006; 99 (4–6): 414–24. DOI: https://doi.org/10.1016/j.jphysparis.2006.03.007

15. Hattori A., et al. Effect of the haptic 3D virtual reality dental training simulator on assessment of tooth preparation. J Dent Sci. 2022; 17 (1): 514–20. DOI: https://doi.org/10.1016/j.jds.2021.06.022

16. Hewson M.G., Little M.L. Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med. 1998; 13 (2): 111–6. DOI: https://doi.org/10.1046/j.1525-1497.1998.00027.x

17. Higgins D., et al. A scoping review of simulation-based dental education. MedEdPublish. 2020; 9: 36. DOI: https://doi.org/10.15694/mep.2020.000036.1

18. Iacopino A.M. The influence of «new science» on dental education: current concepts, trends, and models for the future. J Dent Educ. 2007; 71 (4): 450–62. DOI: https://doi.org/10.1002/j.0022-0337.2007.71.4.tb04296.x

19. Kagramanyan I.N., et al. Historical aspects of the medical education system transformation. National Health Care (Russia). 2021; 2 (1): 32–40. DOI: https://doi.org/10.47093/2713-069x.2021.2.1.32-40

20. Kumar S. Training pathways in oral and maxillofacial surgery across the globe – a mini review. J Maxillofac Oral Surg. 2017; 16 (3): 269–76. DOI: https://doi.org/10.1007/s12663-017-1020-0

21. Lee C., et al. The role of digital 3D scanned models in dental students' self-assessments in preclinical operative dentistry. J Dent Educ. 2018; 82 (4): 399–405. DOI: https://doi.org/10.21815/jde.018.046

22. Massironi D., Pascetta R., Romeo G.M. Precision in Dental Esthetics. Quintessence, 2013.

23. McGleenon E.L., Morison S. Preparing dental students for independent practice: a scoping review of methods and trends in undergraduate clinical skills teaching in the UK and Ireland. Br Dent J. 2021; 230 (1): 39–45. DOI: https://doi.org/10.1038/s41415-020-2505-7

24. Miyazono S., et al. Use of digital technology to improve objective and reliable assessment in dental student simulation laboratories. J Dent Educ. 2019; 83 (10): 1224–32. DOI: https://doi.org/10.21815/jde.019.114

25. Nardi A., Ranieri M. Comparing paper-based and electronic multiple-choice examinations with personal devices: impact on students’ performance, self-efficacy and satisfaction. Br J Educ Technol. 2019; 50 (3): 1495–506. DOI: https://doi.org/10.1111/bjet.12644

26. Nassar H.M., Tekian A. Computer simulation and virtual reality in undergraduate operative and restorative dental education: a critical review. J Dent Educ. 2020; 84 (7): 812–29. DOI: https://doi.org/10.1002/jdd.12138

27. Panczyk M., Gotlib J. Assessment of reliability, sensitivity, objectivity and validity of MCQ pharmacology exams as a potential output variable for predictive analysis. Indian J Pharm Educ Res. 2015; 49 (1): 1–9. DOI: https://doi.org/10.5530/ijper.49.1.1

28. Ringsted C., Hodges B., Scherpbier A. «The research compass»: an introduction to research in medical education: AMEE Guide No. 56. Med Teach. 2011; 33 (9): 695–709. DOI: https://doi.org/10.3109/0142159X.2011.595436

29. Sadid-Zadeh R., et al. Agreement between faculty and %comparison for preparation depth assessment in a dental technique skills course. J Dent Educ. 2019; 83 (5): 553–9. DOI: https://doi.org/10.21815/jde.019.067

30. Schlenz M.A., et al. Undergraduate dental students’ perspective on the implementation of digital dentistry in the preclinical curriculum: a questionnaire survey. BMC Oral Health. 2020; 20 (1): 78. DOI: https://doi.org/10.1186/s12903-020-01071-0

31. Stangl D.K. Encyclopedia of statistics in behavioral science. J Am Stat Assoc. 2008; 103 (482): 881–2. DOI: https://doi.org/10.1198/jasa.2008.s230

32. Streiner D.L., Norman G.R. Health Measurement Scales: a practical guide to their development and use. Oxford (Engl.); New York: Oxford University Press, 2008. DOI: https://doi.org/10.1093/acprof:oso/9780199231881.001.0001

33. Tiu J., et al. Convergence angles and margin widths of tooth preparations by New Zealand dental students. J Prosthet Dent. 2016; 116 (1): 74–9. DOI: https://doi.org/10.1016/j.prosdent.2016.01.003

34. Urbankova A. Impact of computerized dental simulation training on preclinical operative dentistry examination scores. J Dent Educ. 2010; 74 (4): 402–9. DOI: https://doi.org/10.1002/j.0022-0337.2010.74.4.tb04885.x

35. Veselkova E.G. Medical education in Russia: history, current state, principles. Pedagogika professionalnogo meditsinskogo obrazovaniya [Pedagogy of Professional Medical Education]. 2018; 3: 6–14. (in Russian) [Веселкова Е.Г. Медицинское образование в России: история, современное состояние, принципы // Педагогика профессионального медицинского образования. 2018. № 3. С. 6–14.]

36. Wierinck E., et al. Effect of augmented visual feedback from a virtual reality simulation system on manual dexterity training. Eur J Dent Educ. 2005; 9 (1): 10–6. DOI: https://doi.org/10.1111/j.1600-0579.2004.00351.x

All articles in our journal are distributed under the CC BY-NC-ND 4.0 (Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International)