How to Convert Detection Accuracy into a Camera Specification

The usual mistake is to start from the camera: "twenty megapixels ought to be enough." The chain runs the other way. First fix the dimension being measured and its tolerance, then work out the pixel size that tolerance demands, then derive the field of view and resolution, and only then pick a camera model. Every link in that chain is one-directional. This article gives the full conversion, a lookup table for common combinations, a worked example, and the three places where paper accuracy quietly fails to survive contact with the shop floor.

Diagram relating field of view, pixel size and measurement accuracy

Key takeaways

  • Accuracy is field of view divided by effective pixels. Fix the field of view first, not the camera.
  • Pixel size is a geometric limit; real repeatability is typically two to five times worse.
  • Shrinking the field of view is the cheapest accuracy improvement available — often cheaper than a better camera.
  • Subpixel edge detection improves repeatability, not absolute measurement. Use it carefully.
Isometric diagram of an industrial camera and an edge computing box connected by a single cable
The station-level hardware: one industrial camera plus one edge computing box

Accuracy gets used loosely in vision projects. Sometimes it means how many pixels the camera has, sometimes the smallest defect the algorithm can resolve, sometimes how much repeated measurements of the same part scatter. Those three quantities differ by orders of magnitude, and mixing them produces arguments like "why can a twenty megapixel camera not measure to 0.02 mm?" The only way to settle it is to separate the links: tolerance, field of view, pixel size, real repeatability. The chain runs one way. Starting from the camera and working backwards answers a question nobody asked.

Start from the requirement, not the sensor

A very common opening is a discussion of sensor specifications: twenty megapixels, 3.45 micron pixels, rolling shutter. Those numbers matter, but they answer "what can this camera do", not "what does this inspection need". The correct starting point is a plainer question: what is the tolerance on the dimension being checked. If a gap is specified at 0.2 ± 0.1 mm, the verdict has to be reliable at the 0.1 mm scale, and the camera should be selected around that number rather than bought first and adapted to afterwards.

Going from tolerance to pixel size, the working rule is to keep the pixel size between one third and one tenth of the tolerance. One third is the floor, adequate when the task only has to separate clearly good from clearly bad. One tenth is for cases that must report a measured value or feed a trend chart. The ratio is not dogma; what it really encodes is that the measurement system error has to be substantially smaller than the tolerance, otherwise there is no way to tell how much of the scatter belongs to the product and how much to the gauge. This is the same principle that underlies detection and precision metrics, viewed from the other side.

A lookup table worth keeping

Combine field of view with resolution and you get pixel size. The table below works through the common combinations. The tolerance column assumes three times the pixel size, which is the permissive floor for a pass or fail decision.

Field of view, resolution and the resulting pixel size (calculated across the width)
Field of viewResolutionPixel sizeWorkable tolerance
30 mm2448 px (5 MP)about 0.012 mm0.04 mm and above
100 mm2448 px (5 MP)about 0.041 mm0.12 mm and above
100 mm5472 px (20 MP)about 0.018 mm0.06 mm and above
300 mm5472 px (20 MP)about 0.055 mm0.17 mm and above
300 mm12288 px (100 MP)about 0.024 mm0.08 mm and above
800 mm12288 px (100 MP)about 0.065 mm0.20 mm and above

What the table says is blunt: field of view is the main price of accuracy. The same 20 MP sensor gives a 0.018 mm pixel at a 100 mm field and degrades to 0.055 mm at 300 mm — a factor of three. So the first thing to fight for is not a more expensive camera but a smaller field of view: move the camera closer, fit a longer focal length, or switch to a local measurement with mechanical locating. Where the cell allows halving the field of view, that doubles the accuracy for free. The effect of mounting position and bracket form on this is covered in how to mount a camera.

Pixel size is not accuracy

Pixel size is a geometric limit: the physical distance one pixel covers. What decides whether a system is usable is repeatability — leave the same part in place, measure ten times, look at the spread. That number is typically two to five times the pixel size, and the reasons are worth listing one by one, because each corresponds to money that should be spent.

  • Lens distortion. A standard lens can distort by several percent at the edge of the field, which translates into a systematic offset of tens of pixels on a dimension. Calibration removes most of it, but where there is no calibration it is always present.
  • Mechanical repeatability. Every part arrives in a slightly different position and orientation, and the transport stops with some tolerance. This term is often far larger than the camera limit itself.
  • Calibration error. The pixel size is itself a calibrated quantity: the accuracy of the calibration target, the focus state during calibration and the flatness of the target all feed into it.
  • Edge stability. Edge finding works on grey-level gradients, so changing illumination, surface glare and part burrs all shift the detected edge by a pixel or two. This is precisely why lighting design deserves its own budget.

A full worked example

Take a concrete case and walk the whole chain. The task is the centre distance between two holes in a metal part, drawing dimension 50 ± 0.08 mm, so the tolerance band is 0.16 mm, and a measured value is required rather than a bare verdict. Step one, take one tenth of the tolerance as the pixel size ceiling: 0.16 ÷ 10 = 0.016 mm. Step two, fix the field of view: the largest part dimension is about 120 mm, so allow locating margin and take 150 mm across. Step three, derive the resolution: 150 ÷ 0.016 ≈ 9375 pixels. Step four, choose the camera: a 12288 pixel sensor gives 0.0122 mm across a 150 mm field, which clears the requirement; a 5472 pixel sensor gives 0.027 mm in the same field, only one sixth of the tolerance — workable but with little margin. At that point the hardware direction is settled.

The next step matters just as much. If the high resolution option does not fit the budget or the cycle time, the right move is to change the inspection approach rather than force the camera. Three common retreats: use a local field of view with mechanical locating; split the centre distance into two local dimensions measured separately and combined; or downgrade the requirement from "report a measured value" to "decide pass or fail". All three are more sensible than fitting an oversized sensor. For that kind of trade-off, the item by item review in the visual SOP selection checklist applies directly.

Common questions about accuracy conversion

If the pixel size works out to 0.02 mm, can the system measure to 0.02 mm?

No. Pixel size is the geometric limit; usable capability is roughly three to five times that, so about 0.06 to 0.1 mm in this example. Reaching the 0.02 mm scale requires field of view, optics, mechanical repeatability and calibration to move together. A higher resolution sensor is one link of that chain, not the answer to it.

Does more resolution always mean better accuracy?

Not necessarily. More pixels usually means smaller photodiodes and lower signal to noise per pixel, which raises the lighting requirement, while data volume and cycle time pressure rise with it. Accuracy is governed by pixel size relative to field of view, and by how stable the mechanical and optical setup is. Resolution is one input among several.

The camera is already installed. Can accuracy still be improved?

Usually along two paths. The first is shrinking the field of view by moving the camera closer or fitting a longer focal length, which is the most direct and effective step. The second is improving stability: stiffening the bracket, improving the lighting, recalibrating. Neither requires a new camera, and in practice both usually beat swapping the sensor.


What tolerance do you need to hold?

Send us the drawing requirement and the part dimensions, and we will work out the field of view and resolution, and say which links will limit the achievable accuracy.

Contact us

Related articles

View all

Need specific advice for your production line?

Our engineers can share practical approaches to process error prevention and SOP compliance inspection for your scenario.

Request a free trial plan