5.4 - Calibration Constants [U6 Datasheet]

This information is only needed when using low-level functions and other ways of getting binary readings. Readings in volts already have the calibration constants applied. The UD driver, for example, normally returns voltage readings unless binary readings are specifically requested.

Calibration Constant

The majority of the U6's analog interface functions return or require binary values. Converting between binary and voltages requires the use of calibration constants and formulas.

When using Modbus the U6 will apply calibration automatically, so voltages are sent to and read from the U6, formatted as a float.

Which Constants Should I Use?

The calibration constants stored on the U6 can be categorized as follows:

• Analog Input
• Analog Output
• Current Source
• Internal Temperature

Analog Input: The U6 has 4 gains and the pro has a 24-bit sigma-delta converter, so total of eight calibrations are provided: one for each gain for each converter. The U6 uses multiplexed channels connected to a differential input amp so, single ended and differential readings use the same calibration.

Analog Output: Only two calibrations are provided, one for DAC0 and one for DAC1.

Current Source: Two calibrations are provided, one for Iout0 and one for Iout1. The calibrations are the number of amps measured during calibration. These are just a number; there is no related formula.

Internal Temperature: This calibration is applied to a reading from channel 14 (internal temp) after the binary has been converted to Volts.

U6 Input Ranges

The U6 has a total of 8 input ranges. Four single ended and four differential. The eight ranges are:

Table 5.4-1. Input Ranges

Note that the minimum differential input voltage of -10.58 volts means that the positive channel can be as much as 10.58 volts less than the negative channel, not that the positive channel can be +10 and the negative -10 as this results in a +20V signal which is outside the range that the U6 can measure. The voltage of any analog input pin, compared to ground, must be in the range -10.58 to +10.10 volts.

U6 Calibration Formulas (Analog In)

Depending on how an analog reading is obtained either 16 or 24 bits are returned. All readings and the calibration constants are 16-bit aligned. This means that 24-bit values must be justified to 16-bit values before applying a calibration. To justify a 24-bit value to 16-bits divided it by 256 and store it as floating point, so that the information in the lower 8-bits is retained. The approximate nominal conversion from binary to voltage is:

Volts(uncalibrated) = (Bits/65536)*Span (Single-Ended)

Volts(uncalibrated) = (Bits/65536)*Span – Span/2 (Differential)

Binary readings are always unsigned integers.

Where span is the maximum voltage minus the minimum voltage from the tables above. The actual nominal constants are provided in the tables below, and should be used if the actual calibration constants are not read for some reason. Most applications will use the actual calibrations constants (PositiveSlope, Offset, Center, NegativeSlope) stored in the internal flash.

if(Bits < Center)
        Volts = (Center - Bits) * NegativeSlope
else
        Volts = (Bits - Center) * PositiveSlope

The offset calibration has been provided so that the same simple formula used on the U3 and UE9 can be used on the U6. When using the simple formula negative values will be off by a few bits (up to 5 bits in testing, but this value has not been characterized). The simple formula is:

Volts = (Slope * Bits) + Offset

U6 Calibration Formulas (Analog Out)

Writing to the U6's DAC require that the desired voltage be converted into a binary value. To convert the desired voltage to binary select the Slope and Offset calibration constants for the DAC being used and plug into the following formula.

Bits = (DesiredVolts * Slope) + Offset

U6 Calibration Formulas (Internal Temp)

Internal Temperature can be obtained by reading channel 14, applying the proper voltage conversion then using the following formula.

Temp (K) = (Volts *  TemperatureSlope) + TemperatureOffset

U6 Calibration Constants

Below are the various calibration values are stored in the Mem area. Generally when communication is initiated with the U6, ten calls will be made to the ReadMem function to retrieve the first 10 blocks of memory. This information can then be used to convert all analog input readings to voltages. The high level Windows DLL (LabJackUD) does this automatically.

Table 5.4-2. Calibration Constants

Format of the Calibration Constants

Each value is stored in 64-bit fixed point format (signed 32.32 little endian, 2's complement). Following are some examples of fixed point arrays and the associated floating point double values.

Table 5.4-3. Calibration Constants Format