Received date: June 20, 2018; Accepted date: July 27, 2018; Published date: August 03, 2018
Citation: Cardona-Arias JA, Córdoba JP, Ibarra AAV (2018) Efficacy of Obstetric Simulation in the Learning of Skills Related to Birthing Care in Medical Students, Medellin-Colombia. Transl Biomed Vol.9 No.3.152. doi:10.21767/2172-0479.100152
Copyright: © 2018 Cardona-Arias JA, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Introduction: In Colombia, studies regarding the effectiveness of simulation-based medical training programmes are scarce.
Objective: To evaluate the effectiveness of a simulationbased obstetric educational intervention in the learning of skills related to birthing care in medical students in Medellin (Colombia).
Methods: A randomized experimental study of 28 students who received the intervention and 21 students who did not receive the intervention and served as the control group. A scale was constructed and psychometrically evaluated to measure competencies in three domains, Being, Knowing, and Doing, in birthing care; this scale was applied before and after the intervention, generating a rating between 0 (poorest) and 100 (best). The effectiveness of the educational intervention was evaluated using differences-indifferences and linear regression analysis.
Results: The scale showed excellent psychometric properties in the domains of knowing and doing, with Cronbach’s α-value greater than 0.80; furthermore, 100% success in internal consistency, discriminant power, and content validity added to the high explained variance. The intervention did not generate significant changes in the competency in the being domain and was effective in improving competencies in the Knowing and Doing domains, with increases of 23.9 and 27.2 points, respectively.
Conclusion: The simulation-based birth-related skills learning programme was effective in increasing the competencies of knowing and doing in medical students.
Efficacy; Simulation; Medical education; Obstetric birth; Colombia
Education by competences seeks to allow future graduates to appropriately and effectively adjust themselves to changing circumstances and problems; to predict effects, consequences, and possible errors in their professional practice, and to give to the student the role of protagonist in the learning [1-3].
Consistent with the development of medical training via competencies, the implementation of simulation has become global, with widespread evidence of its favourable effect on the knowledge and praxis of healthcare professionals . In a simulated environment, the student is given the opportunity to know, to know how, and to be; the errors can be corrected prior to exposure to real patients; reduces the time necessary to learn abilities and improves learning curves versus those based on classical training [5,6].
Medical simulation has shown benefits in various areas of healthcare, including the following: a reduction in surgical times and errors in surgical techniques; in cardiology, improved diagnosis and treatment of the major cardiopathologies; in anaesthesiology, efficacy in the development of various skills, as well as in gastrointestinal endoscopy .
Particularly in obstetrics, the use of simulation has been encouraged since the 1990s, especially in training for medical specialties . In this area, it is reported that students in learning groups based on simulations, in comparison to those in a control group who did not receive this training, proved to be more certain in upper uterine measurements, Leopold’s manoeuvres, care for artificial membrane rupture, and the placement of intrauterine pressure catheters . There has also been documentation of the efficacy of simulation in skill development prior to contact with real patients in the use of forceps . One study in a hospital in Tanzania involved 89 employees in simulation-based training and showed improvements in knowledge, skill, and confidence in patient care . Finally, research involving 113 students reported that the simulation-based group showed improved confidence in the ability to aid in a vaginal birth, in addition to higher marks compared to the control group who did not attend a simulation .
Despite all the aforementioned findings, nor can one find an evaluation scale for competencies with sufficient psychometric validity to use in measuring the effects of interventions in simulation. The objective of the present research project was to evaluate the effectiveness of an educational intervention based on an obstetric simulation on the learning of competencies related to birthing care in medical students in Medellin (Colombia).
Type of study
A randomized experiment was conducted to evaluate the effectiveness of a simulation-based educational intervention on medical competencies related to birthing care. The assignment of each student to one of the two study groups was done in excel using a sequence of random numbers 1 (treatment) and 2 (control).
The estimated sample size necessary to detect differences equal to or greater than 10 points between the intervention and control groups, based on a competency rating scale that assigned scores between 0 and 100, taking a deviation of 10 points in each group, an α error of 0.05, and a β error of 0.10. With these parameters, the size of each group should be 22 students.
Medical students currently enrolled during professional training phase (years 4-5), attending an obstetrics course. With the intention of homogenizing the sample in terms of academic achievement, students with average marks in the extremes (i.e., a grade point average in the prior semester less than 3.5 or greater than 4.0), as well as students repeating the course, were excluded.
After applying the criteria, a group of 49 students were identified; the students were then randomly assigned to the treatment (n=28) and control (n=21) groups.
Three phases lasting one month. In the first phase, a theoretical class related to birth physiology and the mechanism and work of birth was taught; it began with a survey on the competencies of Being, Knowing, and Doing. Each subject was given access to the same bibliography to study the course material and, in the second phase (two weeks later), was given a one-hour simulation guided by advisors. In the third phase (two weeks later), each student attended a simulated spontaneous vertex birth. Finally, each student was given the same survey on competencies. The labor was simulated by a mannequin programmed from the monitor of the coordinator of the simulation room, particularly phase two (birth) with an attempt for each student.
This group also had three phases, the first and third being the same as those of the intervention group; in phase two, the students in the control group did not attend a simulation but instead spent one hour (the same time as the advisor-assisted simulation) studying in detail the same topics in a master class. In this group, a theoretical workshop on the stages of pregnancy with emphasis on the behaviours to follow during the expulsive phase or birth was conducted, similar to the skills applied in the simulation.
Ratings of competencies in Being, Knowing, and Doing related to birthing care, developed with the assistance of a doctor and a professional in pre-hospital care. To establish ratings in these three competency domains, a multidimensional scale was constructed based on a reflective psychometric model in which observable items or variables are consequences or reflections of the construct, as opposed to a formative model in which items are causal of the construct or dimension of competency . The scale was completed by the teacher of the course at the end of the practice in simulation in the group intervened and by the same teacher at the end of the theoretical task in the control group.
Construction of the competency rating scale in Being, Knowing, and Doing related to birth
The following three domains of competency were defined: Being, Knowing, and Doing; for all three, a literature review was performed to identify indicators or items to measure each domain. Meanwhile, consultations with an obstetrics professor, a professional leading a simulation lab, and a doctor with a postgraduate degree in education with an emphasis on medical training and evaluation were made to identify items. After this phase, redundant items were eliminated, and the competency scale was designed for Being, Knowing, and Doing; it was applied to the 49 students to evaluate its psychometric properties in a manner that was reproducible and valid . The final score of each dimension was calculated with the following statistic: [(Sum of items of each dimension-minimum score)/rank]*100.
Psychometric evaluation of the reproducibility and validity of the scale to measure competencies in Being, Knowing, and Doing .
For reproducibility, reliability was evaluated using Cronbach’s α (satisfactory for values ≥ 0.70), internal consistency with the Spearman correlations (satisfactory for values ≥ 0.40), and the ability to discriminate using the Spearman correlations (satisfactory for item-domain correlations to which it better fits than the item-domain correlation to which it does not belong).
For validity, criteria of logical validity were applied, evaluated with students for acceptability and professionals for applicability. The predictive validity with the proportion of explained variance, and content validity with factorial analysis (satisfactory for λ coefficients ≥ 0.40). The goodness of fit in the factorial analysis was estimated with the Kaiser-Meyer- Olkin statistic (satisfactory for values ≥ 0.7) and Bartlett’s Test of Sphericity (satisfactory for values p<0.05) . It is worth noting that the construct did not break down, given that the factorial loads were estimated individually for each domain and given that the sample size did not meet the minimum size required for multivariate analysis (minimum 8 times the number of independent variables). The convergence of the three competency domains was evaluated using Pearson correlations due to compliance with the assumption of bivariate normality evaluated using the Shapiro-Wilk test.
Evaluation of the effectiveness of the intervention regarding the competencies of being, knowing, and doing
The homogeneity of the variables of sex, age, previous birthing care, and previous attendance at an obstetric simulation were determined using the Pearson’s Chi-squared, Mann-Whitney U, and Fisher’s exact tests. The ratings were compared in each competency domain, from the start and end of the study, using Student’s t-Test for paired samples.
The effectiveness of the intervention in each of the three competency dimensions was established with a paired differences-in-differences model [(the average of the difference between the final rating and the initial rating for each individual in the intervention group)-(the average of the difference between the final rating and the initial rating for each individual in the control group)]. Additionally, the effect was established with two more models: i) a simple linear regression model, taking the difference that each student had in each competency domain rating from start to finish as the dependent variable, with the group (Treatment/Control) being the independent variable; and ii) a multivariate linear regression model, using the same variables as the simple model and adding sex, age, previous birthing care, and previous simulation training as independent variables to obtain a less skewed measurement of the global effect of the intervention.
According to Resolution 8430 of the Ministry of Health of Colombia, the present study did not carry risk. It respects the consent, confidentiality, privacy, and dignity of the subjects studied. Given the efficacy found, after the study, the control group subjects also received the intervention. The study was approved by an ethics committee of the Cooperative University.
In evaluating the reproducibility of the scale, excellent results were found for the domains of Knowing and Doing, with Cronbach’s α values greater than 0.80 and 100% success in internal consistency and discriminant power. Regarding content validity, it was found to have a success rate of 87.5% in the Being domain and 100% in the Knowing and Doing domains; simultaneously, the explained variance was high, and convergence was only found in the competency domains of Knowing and Doing, with a moderate correlation (Table 1).
|Item-domain correlation range||0.21-0.67||0.40-0.77||0.61-0.75|
|Percentage of success||87.5 (7/8)||100 (8/8)||100 (8/8)|
|Item-other domain correlation range||0.05-0.51||0.05-0.52||0.05-0.54|
|Percentage of success||87.5 (14/16)||100 (16/16)||100 (16/16)|
|Factorial load range||0.17-0.64||0.47-0.81||0.60-0.79|
|Percentage of success||87.5 (7/8)||100 (8/8)||100 (8/8)|
|Bartlett’s Sphericity (Vp)||0.007||0||0|
|Floor and ceiling effects|
|Percentage of explained variance||25||42.1||49.6|
|Convergence of domains||Pearson Correlations|
|Competency rating Being||1||0.093||0.17|
|Competency rating Knowing||-||1||0.53|
Table 1: Evaluation of the reproducibility and validity of the competency scale of being, knowing, and doing in birthing management.
It is worth specifying that the item related to “I feel capable of attending a birth” presented a Rho=0.209 with the Being domain, a Rho=0.377** with the Knowing domain, and a Rho=0.513** with the Doing domain and a factorial load (λ coefficient)=0.178 with its domain, which demonstrates its low contribution to the construct of competencies in Being.
After the psychometric evaluation of the scale, the homogeneity of the treatment and control groups was determined for the variables of sex, age, previous birthing care, and previous training with an obstetric simulation (Table 2).
|Has attended a birth|
|Has been training in obstetric simulation|
|Age||Median||Inter-quartile range||Median||Inter-quartile range|
a: Chi2 Pearson. b: Fisher. c: Mann-Whitney U
Table 2: Description of the study groups.
In the being competency domain, no significant difference between the ratings before and after the intervention was observed in either study group. By contrast, in the Knowing and Doing domains, despite the presence of differences in the pre-intervention ratings, a greater increase in the postintervention ratings after the application of the educational strategy of the simulation group was observed. A difference was found in the differences of the statistically significant paired measures, with an increase of 12 points in the Knowing domain and of 13.6 points in the Doing domain, which are attributable to the educational intervention (Table 3).
|Variables||Simulation X ± DE||Control X ± DE||Differences in paired differences (CI 95%)|
|Pre-intervention||41.3 ± 12.8||38.9 ± 11.3||-0.9 (-4.3; 2.5)|
|Post-intervention||39.7 ± 10.8||40.9 ± 10.3|
|Pre-intervention||51.2 ± 17.0||38.2 ± 16.4||12.0 (5.8; 18.2)|
|Post-intervention||72.2 ± 19.6||46.5 ± 15.4|
|Pre-intervention||46.5 ± 19.1||37.5± 17.6||13.6 (7.2; 20.0)|
|Post-intervention||70.3 ± 18.7||44.5 ± 14.6|
Table 3: Comparison of ratings in the being, knowing, and doing competency domains in the treatment and control groups before and after the intervention.
Taking only the treatment group, the difference pre-and post-intervention was-1.6 (CI 95=-7.7; 4.5) in the Being domain, 21.0 (CI 95=11.2-30.8) in the Knowing domain, and 23.8 (CI 95=14.0-33.6) in the Doing domain.
In both the initial and final measurements, no statistical association between the ratings of the Being, Knowing, and Doing competencies and age (Vp Rho<0.05) or sex (Vp Student’s t<0.05) was found. Similarly, no significant difference in the ratings of the three competencies (at the baseline measurement or after the intervention) according to the variables of previous birthing care or attendance at an obstetric simulation was found (Vp Student’s<0.05).
Finally, the educational intervention based on obstetric simulation was effective in improving the competencies related to Knowing and Doing, with a difference of 23.9 points between the treatment and control groups in the Knowing domain, adjusted for the variables of age, sex, previous birthing care, and previous simulation training, whereas the final effect in the Doing domain was 23.8 points higher in the group that participated in the intervention compared to the control group (Table 4).
|Linear regression||Y= Δ Rating post-intervention-Rating pre-intervention|
|Simple model||Being β (SE)||Knowing β (SE)||Doing β (SE)|
|Group (Treatment/Control)||-1.6 (3.4)||21.0 (5.5)**||23.8 (5.5)**|
|Adjusted model (multivariate)|
|Group (Treatment/Control)||-2.8 (3.4)||23.9 (5.5)**||27.2 (5.5**)|
|Age||0.4 (0.4)||-1.6 (0.7)*||-1.5 (0.7*)|
|Sex (Female/Male)||2.3 (3.4)||4.1 (5.5)||1.7 (5.5)|
|Previous birthing care (Yes/No)||-5.3 (4.2)||0.7 (6.7)||4.3 (6.7)|
|Simulation training (Yes/No)||8.0 (7.0)||-3.7 (11.3)||9.3 (11.3)|
Table 4: Crude and adjusted effect of the simulation on the ratings of competencies related to being, knowing, and doing.
In this study, a scale to measure competencies in birthing care was designed and psychometrically validated with the dimensions of Knowing, Doing, and Being; simultaneously, the effectiveness of simulation in improving the given competency domains in medical students was evaluated. This component is novel in the measurement, given that medical education does not avail itself of many instruments with a strong design and psychometric evaluation, ensuring the validity of the reported results to evaluate training programmes.
In this sense, it is important to note that the design of scales for the measurement of subjective events is increasingly frequent in medicine, given the multitude of events that cannot be directly observed or can perhaps be observed through one unique variable. This trend clearly illustrates the need to construct instruments that enable a valid and reliable estimation of multiple constructs, in this case, those of competency. In this study, a multidimensional scale with excellent properties of reliability, internal consistency, discriminant power, and content validity in the competencies of Knowing and Doing was designed; the implication is that the items measure the same element in each dimension, presenting a better correlation with their dimension versus the others and adequately representing the construct that they intend to measure. Therefore, it has clear relevance for measuring the outcomes of the intervention .
The simulation-based intervention generated significant changes in the competencies of Knowing and Doing, with increases of 23.9 and 27.2 points, respectively. In searching for previous works that group the dimensions of Being, Knowing, and Doing using a clinical simulation, no similar studies in the scientific literature were found. Only related studies with knowledge or abilities developed in training programmes and that arrive at the same types of conclusions related to the effectiveness of simulation in widely varied outcomes were identified.
In Medellin (Colombia), after simulated labour, a measurement of the acquired knowledge of healthcare personnel regarding postpartum haemorrhage was taken; there was a knowledge (comparable to competencies of Knowing) and procedures (similar to competencies of Doing) retention rate of 30% as measured by a written test . To this can be added studies that show the effect of simulation of certain obstetric practices, such as the measurement of fundal height, Leopold’s manoeuvres, care for artificial membrane rupture, and the placement of intrauterine pressure catheters .
Other studies have reported that, despite a lack of conclusive effects on the utility of simulation in clinical decision-making, in comparing groups with and without this type of intervention, better results in the former group in variables such as the recollection of clinical information and the degree of confidence in medical decisions are found .
In obstetric emergencies (specifically post-partum haemorrhage), comparing the effect of teaching based on lectures, simulation, and a combination of the two against the degree of knowledge and skill of the study’s subjects, it was found that the simulation group was the only group with a significant improvement in its ability to manage a case, increased confidence in care, and better communication skills . In general, simulation has a positive effect on obstetric ability, confidence in care, and clinical participation on the part of the student . In other groups, such as midwives, a positive effect on outcome such as confidence, satisfaction with learning, and the knowledge level has been noted .
In the Rodriguez group study, a significant improvement in student confidence in the execution of procedures has been reported . In others, using a survey administered before the simulation and immediately at the end of the session, it has been demonstrated that simulation improves confidence in care during a normal birth . Furthermore, simulation improves confidence during actual patient care and optimizes response and quality of care in maternal and neonatal emergencies [21-23].
In general, the majority of studies are similar in terms of the types of outcomes measured and how they are evaluated, i.e., basically, self-reported questionnaires, interviews, or checklists for measuring the degree of confidence or the acquisition of a skill, among other measurements that differently result in the construction and validation of a scale for measuring competency in Being, Knowing, and Doing. This situation has resulted in difficulty in precisely comparing the effectiveness reported in this research versus previous studies, although the findings in the Knowing and Doing competencies correspond with the positive effects that have been reported in other studies in the realms of knowledge and skills.
However, in the competency dimension of Being, neither good psychometric properties nor a positive intervention effect were found, which can be attributed to the fact that the simulation generally focused on knowledge and skills or could instead be attributed to the fidelity of the simulation used. Thus, there are three levels: i) low fidelity for spaces that do not simulate the complete clinical environment, where the emotional component is found only in the learning of a procedure and is subsequently forgotten due to the lack of a clinical context and attention focused on the demonstration of a skill; ii) moderate fidelity, which is characterized by the demonstration of the technical skill as well as patient interaction, allowing the possibility of evaluating competencies related to communication, attitude, the doctorpatient relationship, and bedside manner, among others; and iii) high fidelity to demonstrating knowing, technical knowhow, and attitudinal components related to decision-making, leadership, teamwork, and responsibility, among others, in an environment that is very similar to clinical reality .
In this vein, in the competency domain of Being, learning that significantly occurs through simulation requires an enhancement of the fidelity of the simulator and of training evaluation: deconstruction-type, reflection, or debriefing, which enables a reconstruction of actions, the sharing of feelings, appreciations of clinical situations, analysis of strengths and errors, and designing a plan for encountering new situations . This enhancement is growing in importance, as has been corroborated over the last several years by many countries that have incorporated clinical simulation into medical curricula, in which debriefing has been shown to be a key element in giving learning from simulated experiences its most significant impact, given that it enables analysis, meaning finding, and learning from lived experience .
On the other hand, it should be remembered that simulation has its maximal educational benefit in contexts with high numbers of students and low numbers of professors and practice sites, which allow the professor to focus on timely skills and multiple objectives . This is of utmost importance in obstetrics training due to the risks inherent in the situations to be managed in the specialty and the fact that inadequate management results in a significant increase in neonatal and maternal mortality and morbidity . As a result, simulation in this specialty is a proactive perspective on training and mitigating these risk and errors, improving communication in teams, and developing and transferring skills in students .
Among the limitations of this study is the lack of a complete psychometric evaluation, which, to perform retroactively, would require the inclusion of additional parameters such as construct validity in terms of structural validity, transcultural validity, and hypothesis testing in the different groups. Similarly, future studies should evaluate additional outcomes such as the degree of confidence developed by the student, anxiety, communication skills, and satisfaction with learning, among others. These limitations are similar to those expressed in the studies previously cited, in which were generally low sample sizes, short study times, effect measurements based on abilities evaluated by clinics (which are difficult to reproduce), and dissimilar degrees of simulation fidelity.
In conclusion, the simulation-based programme for learning birth-related competencies was effective in improving the competencies of Knowing and Doing in medical students, with the competency rating scale presenting excellent psychometric outcomes in the dimensions of Knowing and Doing.23437