For those senior-level executives who run human resources and operations, the greatest challenge for the next decade isn’t just “finding” workers; it’s identifying, validating, and deploying the right skill set at just the right time. The capability gap between what is on a person’s résumé and what they can actually do, however, has shrunk considerably, as the rapid advancement of technical skills over the last five years has rendered many skills already obsolete. Thus, the skills-based organization that had previously been a theoretical concept has now become an absolute requirement for successful businesses to remain competitive.
One of the main barriers to evolving into a skills-based organisation is that many companies only use generic, standardised, off-the-shelf assessments that do not accurately reflect the level of skill required for a specific job. In the past, creating unique, validated assessments has taken many months of work by specialized subject matter experts and psychometricians, making mass implementation across a global organisation virtually impossible. Today, however, the convergence of generative technology with standardized skills frameworks is allowing for this transformation to occur.
As noted in McKinsey’s 2023 report on the state of AI, generative tools have reached a level of maturity where they can handle complex content creation tasks that were previously the sole domain of specialists. For L&D and talent leaders, this means the ability to auto-generate job-specific assessments that are both accurate and aligned with the company’s strategic goals.
The Role of Skill Taxonomies as a Business Foundation
To understand how AI makes these judgments, you have to first understand the data structures that lead to the reason why AI uses a taxonomy (structure) when assessing skills for job performance and training. Without a taxonomy, an AI has no structure to follow, much like a map without a key. A job skill taxonomy breaks down the skills of an individual job into categories according to the industry, role, and proficiency level of the skills required for that job. This ensures that everyone within a company uses the same definitions to define the competencies for their employees. The U.S. has many frameworks (authoritative structures) for creating the skill taxonomies the AI will use to build its skill assessments. One of the largest databases for defining jobs is O*Net Online. This site is sponsored by the U.S. Department of Labor and allows users to define over a hundred occupations by what tasks, tools, and technical skills are associated with that occupation. By using O*Net as a guide, an organization is able to use decades of labor market research to adequately define the roles of their employees. For those in the digital and IT sectors, the SFIA Framework provides a global framework for defining skills and levels of responsibility. When leadership has established a skill taxonomy based on these frameworks, they ensure that the assessment generated by their AI is not merely what the AI thinks a given role is or what it requires to do its job, but instead is built from established industry definitions of competency. All organizations must follow the same definition of competency in order to maintain the integrity of their pipeline.The Mechanics of Automated Item Generation
AIG stands for Automated Item Generation, and it’s not just about asking a chatbot to ”make me a quiz.” This is a complex process by which test items are generated using a variety of logical models. Research conducted by Gierl et al., published in PMC, indicates that due to this technology, it is possible for organizations to create large quantities of highly valid and reliable test items much faster than can be produced using traditional methods for the purpose of developing psychometrically-sound assessments. In the business world, AIG operates by means of “item models” that define the variables associated with creating a question. Based upon the skill taxonomy, the AI identifies the major concept being assessed (i.e., the node of knowledge) and generates a series of alternate versions of a question using the same base concept. Therefore, the assessments remain up-to-date and personal to each person taking the test. In turn, because no two candidates will have access to identical sets of questions and, consequently, there would be less opportunity for candidates to cheat on their exam. Furthermore, these tools are often paired with Computerized Adaptive Testing or CAT. In an adaptive environment, the assessment adjusts its difficulty in real time based on the user's answers. If a candidate answers a question correctly, the next item is more challenging. This creates a much more efficient testing experience. Instead of a 50-question "one size fits all" exam, a candidate might only need to answer 15 targeted questions for the system to determine their proficiency level with a high degree of confidence.Business Benefits and the ROI of Automation
Switching to AI-based assessments gives lots of great perks to your company's profit. The biggest benefit is cutting down the time it takes a furlough or new employee to become competent to work. You can identify either what areas your new hires have skill gaps in or what you've already trained them to do. Time is money and by doing this, you save thousands of hours of lost productivity! The second benefit is, many times, the level of objectivity that an assessment provides is lost when using traditional hiring methods. By using demonstration of verified skills rather than background, history or job title, organizations can access a much larger pool of qualified candidates. This is especially important in a tightly packed labor market in which it may be difficult to find "perfect" candidates. The third benefit of using these types of assessments is their scale. Continuous assessment allows for quickly completing short assessments applicable to specific job functions without waiting until the end of the year (annual performance evaluation period). As a result, the information that gets generated from an assessment builds an up-to-date skills inventory, which gives leaders the ability to see the skills and capabilities of their employee base on an ongoing basis. As Assess.com says, "The ongoing creation of new item-sets is what allows for this type of ongoing measurement." Key Leadership Metric: Percentage of roles with mapped skill profiles. Organizations that map at least 80% of their critical roles to a skill taxonomy see significantly higher internal mobility rates and lower external recruiting costs.Implementing AI Assessments for Technical Support
You’ve got an enterprise that’s looking to hire a 100 Technical Support Specialists. In the past, the HR team would likely simply use a generic “Customer Support Test” that isn’t relevant to the specific software you’re using in your enterprise. Using AI gives you an entirely different way to approach this process. Since the scenario-based assessments were created directly from a validated model, these assessments provide accurate predictions of a candidate’s future job performance. Additionally, the data reflects more than just whether or not the candidate passed; it also shows that the candidate is a “Level 4” in Troubleshooting and a “Level 2” in Active Listening. The company can then develop a customized onboarding plan for that individual candidate based on their strengths and weaknesses. The ROI is immediate because the company hires better candidates, and these candidates become productive sooner than those hires made through traditional methods.Implementation Playbook for Senior Leaders
For leaders ready to implement these systems, the following steps are recommended:- Audit Your Current Measurement Tools: Most organizations are surprised to find how much they spend on outdated, generic assessments that do not correlate with job performance.
- Adopt a Universal Taxonomy: Whether it is O*NET, SFIA, or a custom internal framework, you must have a "single source of truth" for what skills mean within your company.
- Pilot Automated Item Generation: Start with a single department or a high-volume role. Use the AI to generate a bank of items and have your best performers take the test to validate its difficulty and relevance.
- Integrate with Your Learning Ecosystem: Assessment data is only useful if it leads to action. Ensure your assessment tool communicates directly with your Learning Management System.
- Establish Governance: Set clear rules for how assessment data will be used. Transparency is essential for maintaining employee trust and ensuring that the technology is used to support growth, not just to filter people out.
