How to Identify a Good Certificate?

Intro­duction

A new trend has been apparent in IT for about fifteen years now: Not only may we learn throughout life, but we can acquire certifi­cates for having expanded our knowledge. Two words in this last sentence should make the inter­ested reader sit up and take notice: “acquire” and “knowledge”.

A certificate must be paid for! Therefore, the first question we want to ask ourselves is whether you can buy a certificate without having substan­tially increased your knowledge. How are certification proce­dures organized to prevent such malpractice?

Secondly, we turn to the question of what certifi­cates can or do verify: theoretical knowledge – i.e., every­thing that can be learned from books, or real practical experience that grows and changes over the years. Should certifi­cates perhaps even have an expiration date? Are there certifi­cates that check whether I maintain or expand my knowledge and experience once certified? Which promises are used to advertise certifi­cates and what can you make of these promises?

Certification procedure

There is a wide range of certifi­cates on offer, yet most certifi­cates and certification proce­dures are based on a similar process with some compa­rable variants. Figure 1 shows the basic pattern for certification procedures.

If a training provider wants to offer a training course for a certificate, they first have to consider whether they are able to teach the topics contained in the syllabus (step 1 in figure 1). If this is the case, the training provider must be licensed by the board respon­sible for this certificate (step 2). With the corre­sponding license agreement, the board ensures that the training provider imple­ments the board’s syllabus and, if necessary, has the quality of its training materials assured by the board.  If you are a prospective examinee looking for a training provider for a certificate, you should always check if the training provider actually holds the required license.

Once the examinee has found a training provider that suits them, they register for the respective training course and pay the training course fee (step 3+4). If the examinee wants to take the exam right after the training course, the training provider registers the examinee for the exam with a certification body shortly before or during the training course (step 5). Certification bodies are autho­rized for the exami­nation by the board respon­sible for the certificate. The question pool from which the certification body compiles the exami­nation question­naires is developed by the same independent board that defined the syllabus for the training course.

Most training courses are organized in such a way that the exami­nation can be taken directly after a training course that lasts several days (step 6). For this purpose, the certification body appoints an independent non-specialist examiner to conduct the exam on site. The exam is admin­is­tered by a non-specialist examiner in order to prevent them from helping the examinees with the exam in any case.

The certification body receives an exami­nation fee from the examinee for this service (step 7). The examiner has the examinee complete a multiple-choice test (step 9) – either digitally or on paper. They received the tests in paper form from the respon­sible certification body (step 8). Following the exami­nation, the digital tests are evaluated directly by the certification body (step 11) and the result is announced (step 12). If exam sheets in paper form are used, the examiner sends the completed exam sheets back to the certification body (step 10). There, the answers are evaluated, and the number of correct answers is deter­mined (step 11). The examinee is then informed about their result by email. If the examinee has given enough correct answers, they receive their certificate (step 13).

Click on image to enlarge.

Figure 1: Certification procedure from the perspective of the examinee [DST]

This process, which at first glance seems relatively compli­cated for the examinee, was created to counteract the danger presented in the intro­duction that certifi­cates can simply be bought.

A good certificate is charac­terized by the fact that the defin­ition of the contents, the training course, and the exami­nation are the respon­si­bility of different insti­tu­tions that are independent of each other (see figure 2).

Click on image to enlarge.

Figure 2: Division of tasks [DST]

There are different variants to this compre­hensive certification procedure for individual sub-processes:

1. Prepa­ration without training course (see figure 3)
2. Remote exami­nation (see figure 4)
3. Public exami­nation
4. Exami­nation at a test center

If an examinee wants to take the exam for a certificate without prepa­ration by a training provider, the exami­nation fee is somewhat higher for most certifi­cates (step 5 in figure 3). Books are offered for most certifi­cates to facil­itate self-study (step 6 in figure 3).

Click on image to enlarge.

Figure 3: Prepa­ration without training course [DST]

For the exam, the examinee has the three alter­na­tives listed above.

Since the coron­avirus pandemic, many training courses are offered remotely, and therefore location-indepen­dently, so a remote exami­nation is the logical step. For this reason, a remote exam is now offered with many certifi­cates. The exam is taken remotely by the examinee and is monitored by an examiner who connects to the exami­nee’s computer and watches the examinee with a camera. Thus, the need to travel is elimi­nated for all parties involved. Proce­dures that allow for online exams to be taken without super­vision, on the other hand, invite malpractice.

Click on image to enlarge.

Figure 4: Remote exami­nation [DST]

In addition, some certification proce­dures offer examinees the possi­bility of attending a public exami­nation or a test center where they take their exam under personal supervision.

So, to summarize the answer to our first question: In proce­dures that follow the process presented here with a separation of respon­si­bil­ities and where the exam is taken under super­vision, it is ensured that you cannot buy the certificate.

Knowledge or experience?

But what about the second issue? What do certifi­cates verify? Theoretical knowledge or practical experience? Well, this question actually depends on the type of certificate!

Any certificate that only consists of a multiple-choice test merely requests theoretical knowledge. The boards, of course, try to create exam questions that can be answered with practical experience only, but that is very difficult with the multiple-choice pattern.

Certifi­cates that fall into this category usually carry the label “Foundation Level”. The Foundation Level is explicitly adver­tised by the providers as a basic certificate [FGG10]. The examinee masters a field’s basic concepts after­wards. These basic terms can be learned, their meaning can be explained to the examinee. After the exam or training course, the examinee speaks the language of this domain.

Certifi­cates that build on the “Foundation Level” usually go beyond a pure multiple-choice test. These certifi­cates often carry the addition “Advanced Level”, and sometimes “Profes­sional” or “Master”. For these advanced certifi­cates you have to demon­strate practical experience in some way.

For some certifi­cates you must provide testi­mo­nials from your employers for projects that fit the topic of the certificate: e.g., 18 months of testing tasks in projects, or 18 months of project management or subproject management.

Some other advanced certifi­cates include an oral exami­nation in addition to the multiple-choice test. In some cases, there is no training course in the tradi­tional sense, but an attempt is made to simulate a kind of project situation in which the partic­i­pants work together in the respective field.

Then there are some certifi­cates that come with the unpleasant feature of having to be renewed regularly every three or five years. Either the exam must be taken again, or the examinees have to collect credit points that prove certain activ­ities in the certified domain: Conference atten­dance, presen­ta­tions, lectures, article publi­ca­tions. This ensures that the examinees’ experience does not become obsolete.

As far as the question of knowledge and experience is concerned, we note that the basic certificate, the Foundation Level, resembles a theoretical driving test. The theory, i.e., the concep­tu­al­ization and the rules, are mastered, but there is no practical experience. In this respect, the basic certifi­cates should always be taken for what they are: Theoretical knowledge that must be acquired in order to complete advanced certificates.

Conclusion

If you are looking for further training with a certificate, plan for a basic certificate and corre­sponding advanced certifi­cates, depending on your current level of knowledge. Only advanced certifi­cates can really testify your practical experience.

Furthermore, you should insist on a proctored exami­nation and only choose certifi­cates with a clear separation of respon­si­bility for content, training course, and examination.

While researching the right training provider, don’t let yourself be fooled by pretty brochures and appear­ances. Try to get an idea of whether the training managers you are being offered spend most of their time on projects in the field – which means they only earn money with training courses occasionally. If you have found such a training provider, it is much more likely that you will come out of the training course not only with a certificate, but with actual practical advice.

We hope that equipped with this knowledge, you will be able to assess the quality of certifi­cates offered on the market and to identify the most suitable further training for yourself.

[FGG10] Fahl, W.; Ghadir, P.; Gharbi, M.: Vom Sinn und Unsinn einer Zerti­fizierung für Softwarearchitekten – CPSA‑F: Ein gemein­samer Nenner für Softwarearchitekten (EN: On the Sense and Nonsense of a Certification for Software Archi­tects – CPSA‑F: Common Ground for Software Archi­tects); Offprint OBJEK­T­spektrum 11/2010

[DST] The process models are domain stories: www.domainstorytelling.org