« Close

Datasheets and User Guides

App Notes

Software & Driver

 

5.4 - Calibration Constants

This information is only needed when using low-level functions and other ways of getting binary readings. Readings in volts already have the calibration constants applied. The UD driver, for example, normally returns voltage readings unless binary readings are specifically requested.

 

Calibration Constant

The majority of the U3's analog interface functions return or require binary values. Converting between binary and voltages requires the use of calibration constants and formulas.

When using Modbus the U3 will apply calibration automatically, so voltages are sent to and read from the U3, formatted as a float.

 

Which Constants Should I Use?

The calibration constants stored on the U3 can be categorized as follows:

  • Analog Input
  • Analog Output
  • Internal Temperature

 

Analog Input: Since the U3 uses multiplexed channels connected to a single analog-to-digital converter (ADC), all low-voltage channels have the same calibration for a given configuration. High-voltage channels have individual scaling circuitry out front, and thus the calibration is unique for each channel. The table below shows where the various calibration values are stored in the Mem area. Generally when communication is initiated with the U3, four calls will be made to the ReadMem function to retrieve the first 4 blocks of memory. This information can then be used to convert all analog input readings to voltages. Again, the high level Windows DLL (LabJackUD) does this automatically.

Analog Output: Only two calibrations are provided, one for DAC0 and one for DAC1.

Internal Temperature: This calibration is applied to the bits of a reading from channel 30 (internal temp).

 

U3 Input Ranges

The U3 input ranges can be found in section 2.6.2 of the User's Guide. For your convenience, that table has been provided again below.

Table 2.6.2-1. Nominal Analog Input Voltage Ranges for Low-Voltage Channels

  Max V Min V
Single-Ended 2.44 0
Differential 2.44 -2.44
Special 0-3.6 3.6 0

Table 2.6.2-2. Nominal Analog Input Voltage Ranges for High-Voltage Channels

  Max V Min V
Single-Ended 10.3 -10.3
Differential N/A N/A
Special -10/+20 20.1 -10.3

 

U3 Calibration Formulas (Analog In)

The readings returned by the analog inputs are raw binary values (low level functions). An approximate voltage conversion can be performed as:

Volts(uncalibrated) = (Bits/65536)*Span (Single-Ended)

Volts(uncalibrated) = (Bits/65536)*Span – Span/2 (Differential)

Where span is the maximum voltage minus the minimum voltage from the table above. For a proper voltage conversion, though, use the calibration values (Slope and Offset) stored in the internal flash on the Control processor.

Volts = (Slope * Bits) + Offset

 

U3 Calibration Formulas (Analog Out)

Writing to the U3's DAC require that the desired voltage be converted into a binary value. To convert the desired voltage to binary select the Slope and Offset calibration constants for the DAC being used and plug into the following formula.

Bits = (DesiredVolts * Slope) + Offset

 

U3 Calibration Formulas (Internal Temp)

Internal Temperature can be obtained by reading channel 30 and using the following formula.

Temp (K) = Bits *  TemperatureSlope

 

U3 Calibration Constants

Below are the various calibration values are stored in the Mem area. Generally when communication is initiated with the U3, eight calls will be made to the ReadMem function to retrieve the first 8 blocks of memory. This information can then be used to convert all analog input readings to voltages. Again, the high level Windows DLL (LabJackUD) does this automatically.

Table 5.4-1. Normal Calibration Constant Memory Locations

  Starting      
Block # Byte   Nominal Value  
0 0 LV AIN SE Slope 3.7231E-05 volts/bit
0 8 LV AIN SE Offset 0.0000E+00 volts
0 16 LV AIN Diff Slope 7.4463E-05 volts/bit
0 24 LV AIN Diff Offset -2.4400E+00 volts
1 0 DAC0 Slope 5.1717E_01 bits/volt
1 8 DAC0 Offset 0.0000E+00 bits
1 16 DAC1 Slope 5.1717E+1 bits/volt
1 24 DAC1 Offset 0.0000E+00 bits
2 0 Temp Slope 1.3021E-02 degK/bit
2 8 Vref @Cal 2.4400E+00 volts
2 16 Reserved - -
2 24 Reserved - -

Table 5.4-2. Additional High-Voltage Calibration Constant Memory Locations

Block # Byte   Nominal Value  
3 0 HV AIN0 Slope 3.1400E-4 volts/bit
3 8 HV AIN1 Slope 3.1400E-4 volts/bit
3 16 HV AIN2 Slope 3.1400E-4 volts/bit
3 24 HV AIN3 Slope 3.1400E-4 volts/bit
4 0 HV AIN0 Offset -10.3 volts
4 8 HV AIN1 Offset -10.3 volts
4 16 HV AIN2 Offset -10.3 volts
4 24 HV AIN3 Offset -10.3 volts

 

Format of the Calibration Constants

Each value is stored in 64-bit fixed point format (signed 32.32 little endian, 2's complement). Following are some examples of fixed point arrays and the associated floating point double values.

Table 5.4-3. Fixed Point Conversion Examples

Fixed Point Byte Array  
{LSB, ..., MSB} Floating Point Double
{0,0,0,0,0,0,0,0} 0.0000000000
{0,0,0,0,1,0,0,0} 1.0000000000
{0,0,0,0,255,255,255,255} -1.0000000000
{51,51,51,51,0,0,0,0} 0.2000000000
{205,204,204,204,255,255,255,255} -0.2000000000
{73,20,5,0,0,0,0,0} 0.0000775030
{255,122,20,110,2,0,0,0} 2.4300000000
{102,102,102,38,42,1,0,0} 298.1500000000

5 comments

Hello,

I contact you because I have a problem. I would first of all thank you for the quality of the posted comments.

I want to use my Labjack U12 to convert a voltages to bits (ADC). Please do you have a program witch does it, or any indication to test the ADC. Thanks

Best regards

Romu

This is the U3 User's Guide.

Are you using the U12 on Windows?  If so, start here:

http://labjack.com/support/u12

Greetings; 

I am able to properly read the calibration constants from my U3 using Delphi, with the function:

FErrorCode := eGet(FHnd, LJ_ioGET_CAL_CONSTANTS, 0, val, @loc);

where 'loc' is an array of double and 'val' is dummy double. 

However, when I try to write calibration data to the U3 using the function:

FErrorCode := ePut(FHnd, LJ_ioPUT_CAL_CONSTANTS, $4C, $6C, @loc);

The function executes with no errors (FErrorCode is 0), but the data is not correct when I re-read the calibration area. Is there a different method I need use to write calibration constatns to the U3? 

Thanks-   Jason

 

The LJ_ioGET_CAL_CONSTANTS and LJ_ioPUT_CAL_CONSTANTS IO types are deprecated and their functionality are no longer updated. Please use these equivalent calls instead:

FErrorCode := eGet(FHnd, LJ_ioGET_CONFIG, LJ_chCAL_CONSTANTS, val, @loc);

FErrorCode := ePut(FHnd, LJ_ioPUT_CONFIG, LJ_chCAL_CONSTANTS,  $4C6C, @loc);

As for writing the calibration constants, there was a bug in the UD driver causing the calibration constants to be written incorrectly for the U3 and U6. We fixed that in UD v1.46 which we just released. Please download and run the Windows installer (LabJack-2015-05-07) to install the updated UD driver with the fix:

http://labjack.com/support/u3

Terrific- the driver update did the trick. I appreciate your quick response and work to fix the driver bug. This great service is an example of why I continue to use LabJacks wherever I need to implement PC control of external equipment.

Thanks!  Jason