5.6 - Calibration Constants [UE9 Datasheet] | LabJack
« Close

Datasheets and User Guides

App Notes

Software & Driver


5.6 - Calibration Constants [UE9 Datasheet]

This information is only needed when using low-level functions and other ways of getting binary readings. Readings in volts already have the calibration constants applied. The UD driver, for example, normally returns voltage readings unless binary readings are specifically requested.

Calibration Constant

The majority of the UE9's analog interface functions return or require binary values. Converting between binary and voltages requires the use of calibration constants and formulas.

When using Modbus the UE9 will apply calibration automatically, so voltages are sent to and read from the UE9 are formatted as a float.


Which Constants Should I Use?

The calibration constants stored on the UE9 can be categorized as follows:

  • Analog Input
  • Analog Output
  • Internal Temperature

Analog Input: Since the UE9 uses multiplexers, all channels (except 129-135 and 137-143) have the same calibration for a given input range.

Analog Output: Only two calibrations are provided, one for DAC0 and one for DAC1.

Internal Temperature: This calibration is applied to the binary reading from channel 133 or channel 141 (internal temp).


UE9 Input Ranges

The UE9 input ranges can be found in section 2.7.2. For your convenience, that table has been provided again below.

Table 2.7.2-1. Nominal Analog Input Voltage Ranges

  Gain Max V Min V
Unipolar 1 5.07 -0.01
Unipolar 2 2.53 -0.01
Unipolar 4 1.26 -0.01
Unipolar 8 0.62 -0.01
Bipolar 1 5.07 -5.18

UE9 Calibration Formulas (Analog In)

The readings returned by the analog inputs are raw binary values (low level functions). An approximate voltage conversion can be performed as:

Volts(uncalibrated) = (Bits/65536)*Span (Single-Ended)

Volts(uncalibrated) = (Bits/65536)*Span – Span/2 (Differential)

Where span is the maximum voltage minus the minimum voltage from the table above. For a proper voltage conversion, though, use the calibration values (Slope and Offset) stored in the internal flash on the Control processor.

Volts = (Slope * Bits) + Offset


UE9 Calibration Formulas (Analog Out)

Writing to the UE9's DAC require that the desired voltage be converted into a binary value. To convert the desired voltage to binary select the Slope and Offset calibration constants for the DAC being used and plug into the following formula.

Bits = (DesiredVolts * Slope) + Offset


UE9 Calibration Formulas (Internal Temp)

Internal Temperature can be obtained by reading channel 133/141 and applying the following formula.

Temp (K) = Bits *  TemperatureSlope


UE9 Calibration Constants

The table below shows where the various calibration values are stored in the Mem area. Generally when communication is initiated with the UE9, three calls will be made to the ReadMem function to retrieve the first 3 blocks of memory. This information can then be used to convert all analog input readings to voltages. The high level Windows DLL does this automatically.

Table 5.6-1. Calibration Constant Memory Locations

Block # Byte Normal ADC Nominal Value  
0 0 Slope, Unipolar G=1 7.7503E-5 volts/bit
0 8 Offset, Unipolar G=1 -1.2000E-2 volts
0 16 Slope, Unipolar G=2 3.8736E-5 volts/bit
0 24 Offset, Unipolar G=2 -1.2000E-2 volts
0 32 Slope, Unipolar G=4 1.9353E-5 volts/bit
0 40 Offset, Unipolar G=4 -1.2000E-2 volts
0 48 Slope, Unipolar G=8 9.6764E-6 volts/bit
0 56 Offset, Unipolar G=8 -1.2000E-2 volts
1 0 Slope, Bipolar G=1 1.5629E-04 volts/bit
1 8 Offset, Bipolar G=1 -5.176 volts
Block # Byte Miscellaneous Nominal Value  
2 0 Slope, DAC0 8.4259E+02 volts/bit
2 8 Offset, DAC0 0.0000E+00 volts
2 16 Slope, DAC1 8.4259E+02 volts/bit
2 24 Offset, DAC1 0.0000E+00 volts
2 32 Slope, Temp (133/141) 1.2968E-02 degK/bit
2 48 Slope, Temp (133/141, Low) 1.2968E-02 degK/bit
2 64 Cal Temp 2.9815E+02 degK
2 72 Vref 2.4300E+00 volts
2 80 Reserved    
2 88 Vref/2 (129/137) 1.2150E+00 volts
2 96 Slope, Vs (132/140) 9.2720E-05 volts/bit
Block # Byte Hi-Res ADC (UE9-Pro) Nominal Value  
3 0 Slope, Unipolar G=1 7.7503E-05 volts/bit
3 8 Offset, Unipolar G=1 -1.2000E-02 volts
4 0 Slope, Bipolar G=1 1.5629E-04 volts/bit
4 8 Offset, Bipolar G=1 -5.1760E+00 volts

Format of the Calibration Constants

Each value is stored in 64-bit fixed point format (signed 32.32 little endian, 2's complement). Following are some examples of fixed point arrays and the associated floating point double values.

Table 5.6-2. Fixed Point Conversion Examples

Fixed Point Byte Array  
(LSB, ..., MSB) Floating Point Double
{0,0,0,0,0,0,0,0} 0.0000000000
{0,0,0,0,1,0,0,0} 1.0000000000
{0,0,0,0,255,255,255,255} -1.0000000000
{51,51,51,51,0,0,0,0} 0.2000000000
{205,204,204,204,255,255,255,255} -0.2000000000
{73,20,5,0,0,0,0,0} 0.0000775030
{225,122,20,110,2,0,0,0} 2.4300000000
{102,102,102,38,42,1,0,0} 298.1500000000


Note typo in section titled: "Which constants should I use?"  Should that refer to UE9 vs. U6 in the first line?

Thanks.  Fixed it.