09:06 AM

IT Certifications: 3 Ways To Judge Value

We've debated the worth of technical certifications for decades. Here's how to determine if a program is worth your time and money.

7 Super Certifications For IT Pros
7 Super Certifications For IT Pros
(Click image for larger view and slideshow.)

We've all heard someone say, "I know a guy who got his [xxx] and couldn't do anything! That certification isn't worth the paper it's printed on."

Don't mistake anecdotes for reality. There's ample evidence of the value -- and more precisely, the ROI -- of certifications. A recent Lumina Foundation survey shows business leaders lack confidence in higher education institutions' ability to produce students with the skills and competencies needed in business. The same study shows support for some kind of post-secondary training while pointing out the unaffordability of higher education. Two thirds of business leaders see a candidate's domain-specific knowledge and skills as more important than their major or the university they attended.

Not exactly breaking news. But the point is that certifications are less expensive than post-secondary education, and because they're job-specific, business leaders perceive them as desirable. This is supported in the just-released InformationWeek US IT Salary Survey, where IT staff listed certification as nearly six times more valuable in developing their careers than college courses, and nearly eight times more valuable than an MBA; IT management listed certification as nearly 2.5 times more valuable than an MBA.

[The best candidates offer more than just IT skills. Read 3 Non-Technical Skills IT Managers Should Hire For.]

Even among 1,014 IT executives surveyed, most see certs as more valuable than an MBA in developing their careers.

Certs and tech-specific training are far and away the most valued from a career-advancement perspective, according to the InformationWeek 2014 US IT Salary Survey.
Certs and tech-specific training are far and away the most valued from a career-advancement perspective, according to the InformationWeek 2014 US IT Salary Survey.

Clearly, there is perceived value within the job market for certifications, and that translates into real money. The US Census Bureau reports that having a professional certification can increase your monthly earnings by as much as 40%. Certifications are not only less expensive than traditional university degree programs, but they also increase your earnings and make you more attractive to employers. Done. End of argument on value, OK?

The validity of certifications
But if value isn't what people complain about when they besmirch IT certifications, what are they questioning?

Usually it's the fit, function, or what a certification proves. Validity is, in a nutshell, the degree to which an exam measures and differentiates a pre-described level of skill or knowledge.

Validity has many enemies, including the exam design process, exam delivery and security, and the perceived inability to efficiently measure what CIOs care about: the competence of an individual instead of procedural knowledge. Let's look at all three. I'm not going to bash, or praise, any specific certification. The goal is to provide a matrix to help technical and nontechnical hiring managers evaluate any given program by asking the right questions.

1. Exam development and design
Most major IT certification programs employ a rigorous exam development process, including formal job-task analysis and beta tests that define what the exam should measure and ensure that it does so. This is standard practice, and any program that falls short should be dismissed out of hand.

Assuming that these development processes are in place, the challenge to validity is the perspective used to create these exams and the actual jobs analyzed. Here are three questions to ask:

  • Is the vendor's certification designed to validate real-world use of its products, or was it designed to validate skills needed by partners to sell more gear?
  • Does the exam validate a candidate's knowledge of only the vendor's product, or does it encompass all the skills needed to be successful in the real world for a given technology area?
  • If an individual who holds a particular certification is unable to understand or perform a given task, does that really mean the certification is worthless, or are you misinterpreting how the it meshes with your expectations?

Of course, if a certification states that it conveys a particular skill but doesn't, the problem could be exam fraud.

2. Exam delivery and security
Security of exam materials is essential to maintaining the validity of a certification. The fact is, exam fraud is rampant and probably has been since the first test was developed. People steal exams and sell them online. Vendors create test-prep materials that teach the exam instead of

the skills and knowledge needed to do the work. Proxy test-taking is also a real problem. All these factors contribute to the number of individuals who have certifications but not skills.

Of course, boosting security increases the complexity and cost of getting candidates through the process, so vendors may respond by increasing participation to offset the cost of protecting program integrity. Here are three questions to ask:

  • Does the vendor use strict security procedures, such as limiting who can take exams and how often they can retest?
  • Does the vendor use tools such as biometrics to validate who is taking the exams?
  • Does the vendor view certification as an independent process, or is it simply a marketing tool that puts sales and market goals ahead of exam integrity?

3. Determining competence
Unfortunately, even the most stringent security efforts cannot address the final complaint: the inability of exams to measure competence.

Fundamentally, the problem is the perception that multiple-choice, standardized exams are incapable of measuring competence. In other words, they're designed to measure knowledge, while competence is the efficient and successful application of knowledge. A common perception is that only practical, lab-based exams are capable of determining a candidate's competence.

In reality, multiple-choice exams can measure a lot more than many people understand, and practical exams are much more limited than many realize. The degree to which multiple-choice exams can determine competence has to do with the way they're developed and the cognitive complexity of questions. Here are two questions to ask:

  • Is the exam written so a candidate can regurgitate a memorized response, or is it written at a level that requires her to analyze and evaluate information to arrive at the correct answer?
  • Do the exams cover real-world deficiencies or risks that require experienced application of the technology, or do they assume an "ideal" world that exists only in marketing brochures?

A well-designed exam can differentiate competent candidates from weak ones. Certainly anyone can guess a few correct answers, but the exam should take that into account and ensure that candidates cannot pass strictly on the basis of lucky guesses.

Many employers gravitate toward certifications that require practical, lab-based examinations that require candidates to demonstrate their capability, arguing that these determine competence more effectively than multiple-choice exams. While practical exams can be great for showing competence, their scope can be limited. They also have other limitations: They're more expensive to create and administer, measuring success can be subjective, and they often sacrifice breadth of knowledge for depth within a limited domain. In short, a certification that's based on only a practical exam remains subject to the other two validity issues and does not by itself guarantee that any given candidate will perform competently in the varied contexts of the real world.

Ideally, certification programs should use multiple-choice exams of escalating complexity to identify which candidates are most likely to have a good breadth of knowledge, and then subject them to a practical exam that lets them demonstrate their ability to apply that knowledge.

The takeaway
In an ideal world, certifications have value and provide the real-world, job-specific skills employers are looking for. When evaluating the validity of any certification, ask these questions:

  • Does it measure the skills we need in our organization?
  • Does the vendor have sufficient integrity that we can be sure candidates who have the certification actually earned it?
  • Finally, is the certification designed to promote the competent application of skills and knowledge in the real world?

To get the full value from certifications, CIOs and hiring managers must start asking the right questions.

Can the trendy tech strategy of DevOps really bring peace between developers and IT operations -- and deliver faster, more reliable app creation and delivery? Also in the DevOps Challenge issue of InformationWeek: Execs charting digital business strategies can't afford to take Internet connectivity for granted.

Ken Salchow has worked in the technology industry for over 25 years in a variety of positions from programming to independent consulting. He has been employed with F5 Networks for the last 14 years and is currently the Program Manager for Professional Certification. He holds ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Email This  | 
Print  | 
More Insights
Copyright © 2021 UBM Electronics, A UBM company, All rights reserved. Privacy Policy | Terms of Service