Model Drift and Recalibration: When a Retrain Is Actually Needed

A vision system does not perform on a flat line after go-live. It performs on a slowly declining curve. Lamps age, lenses collect dust, fixtures wear, incoming parts get revised and processes get tweaked. None of it triggers an alarm, and all of it erodes accuracy continuously. By the time the line notices more false alarms, the drift is usually weeks old and those weeks are already embedded in the traceability record. The answer is not frequent retraining, which is expensive and introduces fresh uncertainty. It is to separate the two kinds of drift first: hardware drift is addressed by cleaning and recalibration, while data drift is what calls for retraining. Without that distinction, it is very easy to answer a dirty lens with a retrain.

Two kinds of drift, separated first

The separation method is simple: run a fixed set of standard samples and look first at whether the images themselves changed. If they have — darker, softer, shifted — it is hardware drift and no amount of model work will help. Only if the images look the same but the verdicts changed does the data layer come into question. That standard set should be fixed at go-live: a few standard parts, or a sealed batch of images. What matters is that the same set is used every time so runs are comparable. The whole exercise takes minutes and catches problems before they spread, which makes it the best return on effort of any maintenance action available.

When a retrain is genuinely justified

Retraining has real costs: collecting new data, labelling it, training, validating, redeploying, and carrying the risk that the new model performs worse on the old conditions. It should therefore be triggered by a clear signal rather than executed on a fixed schedule. Three signals recur. First, a new condition appears that the training set never covered — a new part, a new incoming material, a new fixture — and threshold adjustment cannot resolve it. Second, batch-grouped statistics show a systematic decline in a few specific batches, which means the distribution really did move. Third, the criterion itself changed, as when process tightens a tolerance. Conversely, if the decline is mild and the standard samples show the images themselves changed, clean and recalibrate first; that is not a retraining problem.

• New data must cover the new conditions: feeding in only the most recent data dilutes the old conditions and the model learns the new while forgetting the old.
• Keep training and validation separate: the validation set must contain old-condition samples, or degradation is invisible.
• Regression test before deploying: a new model runs the standard samples first and must not be worse than the current one before it is switched in.
• Keep a rollback path: the ability to revert quickly should exist before the first retrain, not after the first bad one.

Version records: without them you will argue

Every cleaning, recalibration, threshold change and retrain alters the behaviour of the system. Without version records, historical verdicts lose their explanation: the same image passed last month and fails this month, and nobody can say what happened in between. In a traceability conversation that is indefensible. Model version and threshold version therefore belong in every verdict record, which is the same requirement stated in write-back field design. Version records have a second benefit: when the line reports that things got worse recently, the timestamps of version changes can be compared directly, and the answer usually appears within minutes rather than after a full investigation.

Suggested maintenance intervals

As a reference rhythm: standard sample reviews weekly where shifts are sensitive and monthly elsewhere; lens and window cleaning every one to three months depending on dust load; camera angle and focus re-verification quarterly, and immediately after any overhaul or relocation; a model-level reassessment every six months, even when nothing looks wrong, so that the latest data has been looked at before the next change is needed. These are not fixed rules, but writing them into the maintenance agreement keeps a system usable two or three years later instead of turning it into equipment that is installed but not trusted. Whether this capability sits in-house or with the supplier is discussed in support and response terms.