We've all heard someone say, "I know a guy who got his [xxx] and couldn't do anything! That certification isn't worth the paper it's printed on."
Don't mistake anecdotes for reality. There's ample evidence of the value -- and more precisely, the ROI -- of certifications. A recent Lumina Foundation survey shows business leaders lack confidence in higher education institutions' ability to produce students with the skills and competencies needed in business. The same study shows support for some kind of post-secondary training while pointing out the unaffordability of higher education. Two thirds of business leaders see a candidate's domain-specific knowledge and skills as more important than their major or the university they attended.
Not exactly breaking news. But the point is that certifications are less expensive than post-secondary education, and because they're job-specific, business leaders perceive them as desirable. This is supported in the just-released InformationWeek US IT Salary Survey, where IT staff listed certification as nearly six times more valuable in developing their careers than college courses, and nearly eight times more valuable than an MBA; IT management listed certification as nearly 2.5 times more valuable than an MBA.
[The best candidates offer more than just IT skills. Read 3 Non-Technical Skills IT Managers Should Hire For.]
Even among 1,014 IT executives surveyed, most see certs as more valuable than an MBA in developing their careers.
Clearly, there is perceived value within the job market for certifications, and that translates into real money. The US Census Bureau reports that having a professional certification can increase your monthly earnings by as much as 40%. Certifications are not only less expensive than traditional university degree programs, but they also increase your earnings and make you more attractive to employers. Done. End of argument on value, OK?
The validity of certifications
But if value isn't what people complain about when they besmirch IT certifications, what are they questioning?
Usually it's the fit, function, or what a certification proves. Validity is, in a nutshell, the degree to which an exam measures and differentiates a pre-described level of skill or knowledge.
Validity has many enemies, including the exam design process, exam delivery and security, and the perceived inability to efficiently measure what CIOs care about: the competence of an individual instead of procedural knowledge. Let's look at all three. I'm not going to bash, or praise, any specific certification. The goal is to provide a matrix to help technical and nontechnical hiring managers evaluate any given program by asking the right questions.
1. Exam development and design
Most major IT certification programs employ a rigorous exam development process, including formal job-task analysis and beta tests that define what the exam should measure and ensure that it does so. This is standard practice, and any program that falls short should be dismissed out of hand.
Assuming that these development processes are in place, the challenge to validity is the perspective used to create these exams and the actual jobs analyzed. Here are three questions to ask:
Of course, if a certification states that it conveys a particular skill but doesn't, the problem could be exam fraud.
2. Exam delivery and security
Security of exam materials is essential to maintaining the validity of a certification. The fact is, exam fraud is rampant and probably has been since the first test was developed. People steal exams and sell them online. Vendors create test-prep materials that teach the exam instead of
the skills and knowledge needed to do the work. Proxy test-taking is also a real problem. All these factors contribute to the number of individuals who have certifications but not skills.
Of course, boosting security increases the complexity and cost of getting candidates through the process, so vendors may respond by increasing participation to offset the cost of protecting program integrity. Here are three questions to ask:
3. Determining competence
Unfortunately, even the most stringent security efforts cannot address the final complaint: the inability of exams to measure competence.
Fundamentally, the problem is the perception that multiple-choice, standardized exams are incapable of measuring competence. In other words, they're designed to measure knowledge, while competence is the efficient and successful application of knowledge. A common perception is that only practical, lab-based exams are capable of determining a candidate's competence.
In reality, multiple-choice exams can measure a lot more than many people understand, and practical exams are much more limited than many realize. The degree to which multiple-choice exams can determine competence has to do with the way they're developed and the cognitive complexity of questions. Here are two questions to ask:
A well-designed exam can differentiate competent candidates from weak ones. Certainly anyone can guess a few correct answers, but the exam should take that into account and ensure that candidates cannot pass strictly on the basis of lucky guesses.
Many employers gravitate toward certifications that require practical, lab-based examinations that require candidates to demonstrate their capability, arguing that these determine competence more effectively than multiple-choice exams. While practical exams can be great for showing competence, their scope can be limited. They also have other limitations: They're more expensive to create and administer, measuring success can be subjective, and they often sacrifice breadth of knowledge for depth within a limited domain. In short, a certification that's based on only a practical exam remains subject to the other two validity issues and does not by itself guarantee that any given candidate will perform competently in the varied contexts of the real world.
Ideally, certification programs should use multiple-choice exams of escalating complexity to identify which candidates are most likely to have a good breadth of knowledge, and then subject them to a practical exam that lets them demonstrate their ability to apply that knowledge.
In an ideal world, certifications have value and provide the real-world, job-specific skills employers are looking for. When evaluating the validity of any certification, ask these questions:
To get the full value from certifications, CIOs and hiring managers must start asking the right questions.
Can the trendy tech strategy of DevOps really bring peace between developers and IT operations -- and deliver faster, more reliable app creation and delivery? Also in the DevOps Challenge issue of InformationWeek: Execs charting digital business strategies can't afford to take Internet connectivity for granted.Ken Salchow has worked in the technology industry for over 25 years in a variety of positions from programming to independent consulting. He has been employed with F5 Networks for the last 14 years and is currently the Program Manager for Professional Certification. He holds ... View Full Bio