Game-Based Training to Teach HIPAA & Social Media

Purpose

According to the Health Insurance Portability and Accountability Act (HIPAA), any communications regarding Personally identifiable information (PII) and Protected Health information (PHI) must be secure and transmitted only to permissible parties. Social media communications open up abundant possibilities for PII and PHI to be improperly shared (Cain, 2011). HIPAA violations via social media pose great risks to health care organizations who can be fined for employee violations. Violations can also result in the termination of employment and other long-term negative effects on a health professional’s career. Much of the HIPAA violations that occur on social media happen because professionals don’t understand that they are violating HIPAA privacy rules by casual use of this technology. This study examined whether an online game-based training for behavioral health care professionals and administrators can result in statistically significant increases in their ability to identify situations that are and are not HIPAA violations on social media.

Description of Online Scenario-Based Training

The course titled “HIPAA Do’s and Don’ts: Electronic Communication and Social Media” is offered through Relias Learning’s library. The training focuses on teaching learners to identify actions on social media that could result in a HIPAA breach. In the course, learners are presented multiple scenarios social media scenarios (20+) and through a game, are challenged to avoid “committing a HIPAA violation” within the scenarios. Immediate feedback is provided to learner’s responses and at the end of the course, they are told how many (if any) HIPAA breaches were committed and then given a hypothetical consequence (fines, terminated from job or jail time).  (See examples in Figure #1 and 2)

Fig. #1 and 2: Screen shots of the course

Participants and Methods

The study was conducted with 35 employees at a school and child welfare/behavioral health organization in a suburb outside of Boston, Massachusetts. We received 95 consent forms to participate in the study and 49 completed the intervention with the assessments. A preliminary screening of the 49 original participants identified 14 individuals who already had superior levels of knowledge (scores >90%) and were tested out of the study.

Participant’s demographics:

• 66% of the participants are female
• 23% are Child Care Workers, 11% were Clinicians, 17% were administers, 17% were Managers, 32% reported other.
• 75% of the participants had been working within the field for 3+ years

Participants first completed a pre-assessment and survey (baseline) and then completed the 1-hour training after which they immediately took a post-assessment and post survey. Participants then completed a 3-month follow-up assessment and survey. The assessments tested general knowledge (see #2 below) and scenario-based knowledge (#3 below) and were completed on a personal computer on their own time. The surveys measured participant’s perceptions of behavior and knowledge regarding HIPAA and social media as well as their learning experience.

Fig.2: Sample Assessment Question

Results and Discussion

An overall knowledge score (general knowledge + scenario-based knowledge) was computed based on the percentage of correct responses to the 34 questions.  The data suggest a significant increase (p = .003) in scores between the Baseline assessment (76.5%) and the Post-training assessment (84.0%) indicating an immediate improvement in overall knowledge of HIPAA violations.  However, this level of knowledge was not maintained in the 3-month Follow-up assessment as there was a significant decrease (p<.003) between the Post-training assessment and the Follow-up (77.3%).

A confidence score was also assessed at the Baseline, Post-test and Follow-up and transformed into a 100-point scale. These data suggest there was smaller but still significant increase (p = .01) in confidence between the baseline (58.1%) and the post-training assessment (67.5%).  As opposed to the trend in overall knowledge, the confidence data indicate there was not a parallel significant decline in confidence at the follow-up assessment (69.7%).

Chart 1 and 2: Pre-Assessments, Post-Assessments and Follow-up Assessment scores for Overall Knowledge and Overall Confidence

General Knowledge vs. Scenario-Based Knowledge

Within the knowledge assessment, 8 of the 34 questions focused on general knowledge of HIPAA violations (see below). The respondents were presented with a mix of HIPAA violations (bold) and non-violations (not bold) and asked to select all of the responses that are HIPAA violations and/or could lead to a breach.

The responses indicate that the knowledge of general HIPAA violations was already at a high level and did not change significantly across all three assessments (Baseline: 80.4%; Post-training: 81.4%; Follow-up: 83.5%). This could be due to the fact that 75% of the participants had worked in the field of health-care for 3+ years and had gained a general understanding of HIPAA violations. Scenario-based knowledge of HIPAA violations did not follow this same pattern. Participants showed an immediate significant increase (75.3% to 84.8%, p < .001) in scenario-based knowledge (identifying violations within scenarios) which then decreased significantly at the 90-day follow-up (75.4%, p < .002)

Chart 3 and 4: Pre-Assessments, Post-Assessments and Follow-up Assessment scores for General HIPAA Knowledge and Scenario-based Knowledge.

Conclusion

The online scenario-based training had a significant and positive impact on immediate scenario-based knowledge (Chart 4: 75.3% vs. 84.8%, p < .003). This level of knowledge was not sustained at the 90-day follow-up (75.4%) while confidence did sustain over time (Chart 2: pre-58.1%, post-67.6%, and follow-up- 69.7%). These gaps between knowledge and confidence suggests that over-confidence in abilities may lead to potential HIPAA violations as the individual’s confidence inhibits decision-making.

Further investigation of the components of overall knowledge revealed the participants in this study demonstrated a consistently high level of general HIPAA knowledge across all data points (Chart 3: pre-80.4%, post-81.4% and follow-up-83.5%) indicating that the participants had previous knowledge of what and what is not a HIPAA violation. Yet these data also suggest that this knowledge did not generalize to specific social media scenarios (Chart 4: pre-75.3%, post- 84.8% and follow-up- 75.4%).

Two potential training implications for these findings are to provide more scenario-based practice and to emphasize the underlying principles of HIPAA. The inconsistent pattern of general knowledge to reality-based scenarios indicates more training opportunities have to be given for practicing the varying ways in which HIPAA violations can occur through social media. Yet, future training also needs to help participants understand underlying HIPAA principles as they apply to complicated and specific scenarios.

Since no training can address all potential HIPAA violation scenarios they might face in their day-to-day job, understanding the principles of HIPAA may allow them to identify both the behaviors that violate HIPAA and the reasons that HIPAA may be violated.

Future studies can evaluate online scenario-based training with retention activities via technology or in-person as well as providing specific feedback to learners on why certain scenarios would be identified as a HIPAA violation.

References

Cain, Jeff (2011). Social media in health care; The case for organizational policy and employee education. American Journal of Health Syst. Pharm. 68:1036-40