« Close

Datasheets and User Guides

App Notes

Software & Driver

 

20.1 Calibration Constants

The T7 automatically returns calibrated readings, so most people need not concern themselves with this section.

The factory applied calibration constants are stored in Internal Flash and can be accessed at any time through the use of the Modbus registers discussed in the parent to this section (Internal Flash section).

The calibration constants begin at memory address 0x3C4000, or in decimal format d3948544. The structure (location) of each calibration value can be seen in the code snippet below. 

typedef struct{
	float PSlope;
	float NSlope;
	float Center;
	float Offset;
}Cal_Set;

typedef struct{
	Cal_Set HS[4];
	Cal_Set HR[4];

	struct{
		float Slope;
		float Offset;
	}DAC[2];

	float Temp_Slope;
	float Temp_Offset;

	float ISource_10u;
	float ISource_200u;

	float I_Bias;
}Device_Calibration;

 

The full size of the calibration section is 164 bytes, or 41 floats.  

The reason that there are 'Cal_Set's for each High Speed 'HS' and High Resolution 'HR', is that there are 2 analog converters on a T7-Pro.  A standard T7 uses only the High Speed analog converter, so only the HS[4] calibration values will be populated with valid information. A T7-Pro will have calibration information for both high speed, and high resolution converters.

Additionally, there are distinct sets of positive slope (PSlope), negative slope (NSlope), Center, and Offset values for each of the 4 gain settings on the device.

High speed AIN calibration values HS[4]:
HS[0] = calibration for gain x1
HS[1] = calibration for gain x10
HS[2] = calibration for gain x100
HS[3] = calibration for gain x1000

High resolution (-Pro only) AIN calibration values HR[4]:
HR[0] = calibration for gain x1
HR[1] = calibration for gain x10
HR[2] = calibration for gain x100
HR[3] = calibration for gain x1000

 

 

Nominal Calibration Values:

±10V Range:

  • Positive Slope: 0.000315805780
  • Negative Slope: -0.000315805800
  • Binary Center: 33523
  • Voltage Offset: -10.586956522

±1V Range:

  • Positive Slope: 0.000031580578
  • Negative Slope: -0.000031580600
  • Binary Center: 33523
  • Voltage Offset: -1.0586956522

±0.1V Range:

  • Positive Slope: 0.000003158058
  • Negative Slope: -0.000003158100
  • Binary Center: 33523
  • Voltage Offset: -0.1058695652

±0.01V Range:

  • Positive Slope: 0.000000315806
  • Negative Slope: -0.000000315800
  • Binary Center: 33523
  • Voltage Offset: -0.010586956

DACs:

  • Slope: 13200
  • Offset: 0

Temperature:

  • Slope: -92.6
  • Offset: 467.6

Current Sources:

  • 10 μA: 0.000010
  • 200 μA: 0.000200

AIN Bias Current: 0.000000015

 

1 comments

isp's picture

I can't find anything like this page

https://labjack.com/support/datasheets/u3/hardware-description/ain/binar...

applicable to T7/T7-PRO we have here.

Could you point me to correct calculation?

The code I found it one of examples C_T7_TCP_Modbus_Stream/src/calibration.c does the following:

    if(*volts < devCal->HS[gainIndex].Center)
        *volts = (devCal->HS[gainIndex].Center - rawAIN) * devCal->HS[gainIndex].NSlope;
    else
        *volts = (rawAIN - devCal->HS[gainIndex].Center) * devCal->HS[gainIndex].PSlope;
    return 0;
So it doesn't use the offset at all to calculate value in volts.

That would be okay, but changing the range results in different voltages read from the same output.

Specifically here is what I have:

Range: 0.01: Slope: 3.1554174e-7 Center: 33342.086 Offset: -0.010516284 Raw-value: 33529 -> Volts: 5.897919e-5
Range: 0.1: Slope: 3.1550958e-6 Center: 33493.71 Offset: -0.10566833 Raw-value: 35948 -> Volts: 0.007743517
Range: 1.0: Slope: 3.1550582e-5 Center: 33508.027 Offset: -1.057076 Raw-value: 35369 -> Volts: 0.05871477
Range: 10.0: Slope: 3.1554952e-4 Center: 33510.492 Offset: -10.572616 Raw-value: 33716 -> Volts: 0.064847894
So for the range of 0.1V 0.0077V error seems to be way to hight for a 14bit ADC - more like 8bit one.

I totally might be doing something wrong elsewhere, but this is the first thing I found that looks wrong.

Thank you in advance!