In our webinar, Why Should Your Staff Stay? Invest in Their Success to Ensure Your Own, Relias product managers Melissa Lewis-Stoner and Justin Hess shared how leaders at behavioral health organizations can use personnel assessments and learning in combination to help their employees feel competent and effective in their jobs.
During the webinar, we received lots of thoughtful question—so many that we couldn’t answer them all during the webinar. You’ll find some of them below along with responses from Melissa and Justin.
Q: These assessments seem to be measuring pretty broad categories. Is there a reason for that?
A: Our assessments must apply to anyone in a specific position nationwide. Thus, we must ensure that the knowledge, skills and abilities of any given role will be generalizable to anyone in that specific position anywhere in the country. When we develop an assessment, we first look to governmental agencies and professional organizations for guidance into what overarching competencies they believe create success in a position. Then, we have a panel of subject matter experts that helps us further refine the assessment and ensure that anyone who takes it will be able to understand it and that the results will be applicable, no matter where they are in the country.
Q: All employees need the soft skills measured by the situational assessment. Can we use the assessment for all employees?
A: When we created the situational assessment for direct care workers in behavioral health, we had a whole panel of direct care workers—almost 30 people—help us create this assessment. Because the situational assessment presents a care worker with highly specific situations, as the name of the assessment implies, we wanted there to be no doubt that those situations would be applicable and really make sense to a direct care worker.
Because the line can be a little blurry as to how direct care workers is defined, we would encourage you to give that assessment to a few people you classify as a direct care worker and make sure that the scenarios make sense to them. If they take the assessment and say, “Yes, the terminology makes sense and these scenarios are true to life,” then by all means, use the assessment for others in that position.
Q: Where can I access these assessments?
A: The assessments are available in the Relias platform and are an add-on product. If you are a current customer and want to see a demonstration of the assessments, reach out to your account manager or someone in client support.
Q: Why aren’t all of the assessments pre-hire or validated?
A: If an assessment is intended for pre-hire use (i.e., selection decisions, such as hiring an individual), it must meet the very strict standards that the Equal Employment Opportunity Commission (EEOC) sets forth for use in these scenarios. Essentially, the EEOC requires statistical proof that the assessment is measuring what it claims to measure, its items (i.e., questions) are sound and functioning correctly, and that steps have been taken to minimize adverse impact (bias) as much as possible. Meeting these standards takes a great deal of time, rigorous study, and resources; it takes months to initially develop, analyze, and refine the assessment. Post-hire assessments, by design, are not intended for the same high-stakes selection decisions. They’re intended to be used for developmental discovery and not, say, punitive action. These types of assessments are not held to the same scientific standards as pre-hire assessments. However, that is not to say they aren’t held to a level of rigor and scrutiny—they absolutely should be! However, these assessments should be examined from a different frame of reference.
Q: What are “soft” factors and can they really be measured?
A: Soft factors are characteristics and traits like empathy or compassion. They are often difficult to intuit during short observations of behavior, such as during an interview or even the first month or so on the job. However, they can be measured! That’s one of the greatest value of assessments—the ability to gain insight into these often complex and difficult-to-observe characteristics and traits. Having this information allows you to probe areas of concern or strength with a candidate or current employee so that you can gain even greater insight into how they approach situations, manifest behavior, and view the world. While these assessments are powerful, be careful—do not to reject a candidate based on the results of a single assessment in isolation. Always use it with other information, such as interviews, resume reviews and reference checks.
Q: Would it be good to give the clinical assessment to a direct care worker a few months into their employment to see how they are retaining the job knowledge?
A: Absolutely! Clinical assessments are great to use as check-ins for your employees. They can help you evaluate the effectiveness of your onboarding and training. You can also use clinical assessments to help you identify professional development opportunities among your current workforce and to focus growth. In doing regular assessment of your employees, you’ll also identify high-potentials—those employees who are exemplary in one or many aspects of their job. You’ll want to identify their strengths early and leverage them often to really elevate your workforce development efforts.
Q: What if I only have a few candidates, or even just one, for a role? Is there any point to using assessments pre-hire or are they only worth it when you have a lot of candidates?
A: Think of it like this: If you have to take someone for a job out of necessity to fill the position, would you want to go in blind? What if you could have at least some information related to their strengths, knowledge gaps, and possible red flags into some of the softer factors critical to success in their role? I may be biased, but I always opt for more information, not less, when hiring a candidate. Knowing more information allows me to better tailor interview questions to probe deeper into my areas of concern, allows me to prepare an onboarding plan to address areas where I know they’re deficient, and helps me understand where I can best leverage their strengths the moment they’re in my door. In short, assessments aren’t only useful to compare candidates, they also serve as a valuable source of information that will equip you with the tools you need to make the best of the candidate you’re bringing in, even if they’re not the ideal person for the job.
Q: Can assessments be “gamed” by the person taking it to produce results they think you want?
A: There are a few psychological phenomena that come into play when someone attempts to “game” an assessment: cheating on job knowledge assessments, participation bias, and socially desirable responding. Let’s unpack each of these and how assessments can be constructed to make gaming the assessment as difficult as possible.
First is cheating. For job knowledge assessments, the items should be sufficiently difficult, be timed, not allow the questions to be copied and pasted or captured, and restrict access to the assessment (so that someone can’t get a question, log out, Google it, log back in, answer, and repeat). The items should also be sufficiently complex and go beyond simple textbook knowledge that can be easily and quickly researched; rather, items should measure a healthy mix of judgment, critical thinking, application, and recall of knowledge.
Next is participation bias, which, put simply, is where a person attempts to tailor their answers to what they think you want to see or would find most valuable. For knowledge assessments, the same concepts can be applied from cheating. For situational assessments, judgment questions that put takers in potentially negative or ambiguous situations with no clear correct answer and require them to choose the most effective AND least effective response will help to overcome this issue.
The last, socially desirable responding, is the phenomenon whereby people overinflate their perceived attributes. This mainly applies to self-report assessments and is easily filtered out with knowledge and situational assessments using the techniques just mentioned, as a person who does not possess the requisite knowledge, skills, or abilities will be identified by poor scores.
Q: If I use assessments in the pre-hire process, do I have to use them for every candidate?
A: Absolutely. When you make assessment part of your hiring process, you must standardize your process and apply your decision-making criteria similarly and fairly across all applicants of the same type. If not, you could run into legal issues if a candidate who was turned down for a job believes they were discriminated against or were the victim of disparate impact.
Q: If you are assigning a mentor to another staff member, do you share the results of the staff member’s assessment with the mentor? Can you?
A: Ideally, information should only be shared at a high level, especially if those mentors or coaches are not trained on how to appropriately interpret assessment results (chances are, they’re not). This means that the information shared should be enough to help them guide their mentee and allow the two individuals to organically discover problem areas and solutions, but not too much that the information could compromise the integrity of the relationship.
Let’s look at the example discussed in the webinar. Joe scored low on empathy, and Tammy fits the ideal profile and scored the highest on empathy. You could provide some high-level guidance to Tammy, explaining that Joe is new to his role and has a limited frame of reference to understand clients’ experiences. Working with her as a mentor would provide him with valuable understanding and awareness of client experiences both past and present.
Q: Can you really change a person’s qualities like empathy or compassion?
A: Not if you tackle these qualities in the same way you would, say, technical knowledge around one’s job. For example, if someone scores low on empathy, I can’t simply sit them down in front of a computer, assign them a few courses, and they’ll suddenly be empathetic. Change in these characteristics starts with awareness that a problem exists. Once it’s acknowledged by the individual and they have buy-in and the desire to improve, the best way to see positive change in these characteristics is to pair them with a mentor or coach that is high in one of these areas. The mentor who is high in empathy/compassion should then model appropriate behaviors related to these characteristics during everyday situations while the mentee is observing. Ideally, the mentor should debrief the mentee on why they handled situations that required empathy or compassion the way they did, what cues the mentor picked up on that indicated the need to use those characteristics, and then have the mentee practice the techniques under guidance and with critical, meaningful feedback.
Q: Do other organizations use assessments in this way?
A: Yes, hospitals and acute care settings often use assessments to inform hiring and placement decisions for nursing and allied health staff and have found them to be quite beneficial in lowering their turnover rates and decreasing the time they spend onboarding. While their hiring and employee development challenges aren’t directly comparable, they do share much in common.
Q: We onboard a large number of employees per month. How can you suggest personalizing onboarding with large numbers?
A: Making onboarding more personalized and efficient with assessments will depend heavily on a standardized process and crosswalks to specific training that tie heavily into the assessment knowledge domains. First, when the assessments are given should be standardized—ideally, everyone should be assessed up to a week prior to their first day of onboarding to give your organization time to pick out and assign the correct courses. A crosswalk that looks at each knowledge domain and pairs a number of appropriate courses will be critical. It’s even more helpful if your organization sets thresholds for which courses get assigned. For example, if a person scores below 50% on documentation, perhaps they are assigned an overview course, an intermediate course, and then an advanced course. However, perhaps those who score 80% or above will receive only the advanced course, and exactly which course each person receives can be tailored in a more granular fashion depending on which assessments they got incorrect or can be more blanketed based on overall domain.