Selecting Measurement Accuracy, Resolution, and Repeatability

A measurement becomes trustworthy only after the physical input, conversion chain, scaling, communications path, and displayed value have been checked separately. Follow the value from the measurand to the final tag; a stable number can still be inaccurate, and extra displayed digits can still exceed the system's usable resolution.

Where does the measurement value travel?

The data path starts at the physical quantity, not at the controller or display. The sensor responds to the measurand, signal conditioning converts that response, an analog-to-digital converter quantizes it, software scales the raw value, and a display, PLC tag, historian, or control loop consumes the result.

Path element	Failure or limitation	Check
Measurand and mechanical installation	Poor contact, loading, mounting stress, or unsuitable location	Confirm that the sensor experiences the quantity being evaluated.
Sensor and wiring	Offset, interference, lead resistance, grounding, or damaged conductors	Inspect the physical connection and compare the signal at the source and receiver.
Input conversion	Range mismatch, quantization, noise, or filtering	Verify the configured input type, range, filter, and raw count behavior.
Scaling	Wrong engineering-unit endpoints or arithmetic	Compare raw counts and scaled values at multiple known inputs.
Network address or port	The client reads the wrong device, channel, or tag	Trace the displayed value back to the configured source identifier.
Update timing	Aliasing, stale data, or filtered transients	Compare source and destination timestamps while changing the input.

A network can transport an incorrect measurement without altering it. Layer one first: prove the physical input and conversion before troubleshooting protocol routing. Check: apply a controlled input and confirm that the corresponding raw channel changes in the expected direction.

What does measurement accuracy describe?

Accuracy describes closeness to an accepted reference value. For reading x_i and reference x_ref, the signed error is e_i = x_i - x_ref. Repeated error with the same sign indicates bias in the complete measurement chain. Accuracy is not the spread of repeated readings.

Use a reference with suitable calibration status and uncertainty. The comparison includes the installed sensor, wiring, conversion, scaling, and indication unless individual components are being tested separately. A component accuracy specification cannot be assigned automatically to the assembled channel; each contributing error and the reference uncertainty must be considered.

Observation	Likely meaning	Next check
Readings cluster away from the reference	Repeatable measurement with bias	Check zero, span, scaling, installation, and calibration.
Error changes across the range	Span, linearity, or range configuration problem	Test multiple points rather than correcting only zero.
Reference and channel both fluctuate	The test cannot isolate channel error	Stabilize the input and compare synchronized readings.

Check: record the reference, indicated value, signed error, test condition, and time at each test point.

What does resolution limit?

Resolution is the smallest input change that produces a distinguishable recorded change. On a graduated instrument it can be associated with the smallest readable division. In an electronic chain, the usable resolution also depends on sensor response, converter step size, input range, noise, filtering, scaling, communications encoding, and display rounding.

Displayed decimal places are formatting, not proof of resolution. If a value is transmitted or displayed with finer increments than the source can discriminate, the extra digits contain no additional measurement information. Likewise, converter counts do not establish end-to-end resolution when input noise makes several counts unstable.

Setting	Effect on the data path	Pitfall
Input range	Maps the physical signal into the available conversion range	An unnecessarily wide range can reduce discrimination over the operating region.
Filter	Reduces fast variation	More filtering can make the value look stable while increasing response delay.
Display precision	Changes shown digits	It does not improve sensor or converter resolution.
Update interval	Controls when new values become visible	A slow update can hide real input changes.

Check: change the applied input in progressively smaller controlled increments and identify the smallest change that appears reliably at the final destination.

How is repeatability tested?

Repeatability describes the spread produced by repeated measurements of the same item under the same defined conditions. Hold the operator or automated method, instrument, setup, location, environment, direction of approach, and test interval constant. Changing these conditions evaluates broader reproducibility rather than repeatability alone.

A direct summary is the observed range, R = x_max - x_min. A sample standard deviation can describe the distribution when enough repeated observations are collected. Always retain the individual readings; the range depends strongly on sample count, while a single statistic can conceal drift or steps.

1. Stabilize the physical input and reference.
2. Approach the test point in the same direction for every trial unless hysteresis is the property under test.
3. Record repeated readings without recalibrating between trials.
4. Review the sequence for drift, periodic interference, discrete count changes, and outliers.
5. Calculate the range and any required statistical measure.

Sampling and filter timing belong in the test record. Readings taken before settling measure dynamic response as well as repeatability. Check: repeat the sequence and confirm that the calculated spread remains acceptable under the same conditions.

How do accuracy, precision, and repeatability differ?

Precision describes how closely repeated results agree with one another. Repeatability is precision evaluated under tightly controlled, unchanged conditions. Accuracy instead compares results with a reference. A channel may therefore be precise but inaccurate, accurate on average but poorly repeatable, or limited by resolution before either property can be evaluated properly.

Pattern	Accuracy	Repeatability	Diagnostic direction
Tight cluster near the reference	Good	Good	Verify performance across the required range.
Tight cluster away from the reference	Poor	Good	Find bias, scaling error, or calibration offset.
Wide spread centered near the reference	The average may be close, but individual readings are unreliable	Poor	Check noise, installation, process variation, timing, and resolution.
Readings move only in coarse steps	Not decided by this observation	May appear artificially stable	Test end-to-end resolution before judging spread.

Check: plot repeated readings against the reference so that bias, spread, drift, and quantized steps remain separate.

How is the complete measurement path commissioned?

1. Define the measurand, engineering units, operating range, and acceptance criteria for accuracy, resolution, repeatability, and response time.
2. Inspect the sensor installation, conductors, shielding, grounding, and input termination before changing software.
3. Verify the input type, conversion range, scaling endpoints, filter, update timing, and displayed precision.
4. Confirm that each network address, channel, and tag maps to the intended physical input.
5. Apply controlled reference inputs at multiple points through the operating range and record signed error.
6. At selected points, collect repeated readings under unchanged conditions and calculate their spread.
7. Introduce the smallest required input change and confirm that it survives conversion, communications, and display formatting.
8. Perform a final dynamic test from the physical input to every consuming tag, display, alarm, historian value, or control calculation.

The final record must connect each reference input to the raw value, scaled value, destination identifier, timestamp, error, and repeated-reading spread. Check: repeat the end-to-end test after saving the configuration and restarting any component whose retained settings affect the path.

FAQ