Intsoft Intsoft Visual AI SOP System

Criteria Alignment Training

Published: Updated: 19 min read

Anyone can write down what "acceptable" means. Getting a group of people to apply it the same way, on the same part, on different days, is a different problem. The usual approach — read the document aloud, sign the sheet — produces almost nothing, because the difficulty has never been the wording. It is the part that sits on the line, where a person has to decide, with no obvious answer. Effective training puts people in front of exactly those parts and makes them commit to a judgement before anyone discusses it.

Diagram of a set of borderline samples used for alignment training

Key takeaways

  • Train on borderline parts, not on obvious ones — obvious parts teach nothing.
  • Everyone judges independently first; discussion before judging contaminates the result.
  • The disagreements are the material; pursue the reasoning, not a verdict.
  • One session decays. Repeat it, and make it a gate for new starters.
Isometric diagram of an industrial camera and an edge computing box connected by a single cable
The station-level hardware: one industrial camera plus one edge computing box

Misalignment shows up as the same part judged differently by two people, or by the same person on two days. Its cost is commonly underestimated: what a customer experiences is the resulting variation in outgoing quality. The aim of training is not memorising the criteria but reaching a stable, defensible judgement on the parts that sit near the line — and being able to explain to somebody else why that judgement was reached.

Borderline parts, not typical ones

The choice of material decides whether the session does anything. A tray of clearly good and clearly bad parts produces unanimous answers and no discrimination, and everyone returns to their own habits afterwards. What needs practising is the boundary: a scratch that just touches the limit, a colour difference inside the grey zone, a dimension near either end of the tolerance. These are the parts that generate disagreement, rework and customer complaints.

Collecting them is continuous work and cannot be improvised. Whenever a part on the line causes hesitation and someone has to fetch a supervisor or an engineer, that part is the best material available — keep it, note the conclusion reached and the reasoning behind it. Over time this set becomes the most useful asset the team has for training, and it is the same management problem described in managing the sample library, seen from the training side.

How to run the session

  1. Everyone judges independently, in writing

    Number the parts, present them in an arbitrary order, and have each person record a conclusion and the reason for it, with no discussion in between. Independence is the point: as soon as one person states an answer, the others converge on it and the data is worthless.

  2. Lay the results side by side

    Tabulate the judgements per part. Parts everyone agreed on can be set aside; the ones with a wide spread of answers are the content of the session. The disagreements normally concentrate on a small number of parts, which is what makes the next step feasible in the time available.

  3. Ask for the reasoning, not the answer

    For each contested part, have the people who disagreed explain which clause of the criteria they applied and what they saw. Pursuing this often reveals that the disagreement is not about eyesight at all — the criteria simply do not cover the situation.

  4. Amend the criteria then and there

    If the discussion exposes a gap, close it: what counts as a reject, what is acceptable, and who arbitrates. The amendment belongs in the controlled document, not in the minutes. This is the only step in the whole session that improves the system rather than the people.

Why it has to be repeated

Alignment decays, for several independent reasons. People change: new starters arrive, others rotate, someone returns from long leave with a slightly different eye. Parts change: material batches, surface treatments and supplier switches all introduce appearances the old examples do not cover. And the criteria themselves change: revisions made in response to a customer complaint or a process improvement are usually communicated on paper and never re-practised. Any one of these reduces consistency.

A workable rhythm follows: every new starter passes a full session before working unsupervised; every criteria revision is followed by a repeat session within a fortnight; the whole group repeats at least twice a year. Number the parts afresh and shuffle the order each time, so that nobody can answer from memory of the sequence. If a repeat produces almost identical results to the previous one, that is a good sign. If consistency has suddenly dropped, something changed that nobody noticed — and that signal is worth having on its own.

Common questions about criteria training

If the system judges automatically, is training people still needed?

Yes, and more than before. The system's verdicts have to be reviewed and acted on by people, particularly for rejects and uncertain results, and the sample sets that validate the system are confirmed by people. If human judgement is scattered, a consistent system has nothing consistent to land on — review is where it gets broken up again.

How is the training measured?

By agreement rates: on a set of borderline parts, the proportion where each person matches the reference conclusion, and the pairwise agreement between people. Recorded at each session, these give a trend. More useful still is where the disagreement concentrates — if it always falls on one type of feature, the criteria need supplementing there.

Do borderline samples change over time?

Yes, particularly rust on metal, wear on coatings and ageing of plastics. Store them sealed, handle them sparingly, and compare them against their photographs periodically to confirm their appearance has not shifted. If a sample has changed, withdraw it from the training material — practising against a standard that has itself moved spreads the wrong standard.

Related

Do your people judge the same way?

Tell us the marginal cases you argue about most. We will help design a training and review routine built on borderline parts.

Contact us

Related articles

View all

Need specific advice for your production line?

Our engineers can share practical approaches to process error prevention and SOP compliance inspection for your scenario.

Request a free trial plan