Appendix B - Resolution Tables | LabJack

# Appendix B - Resolution Tables

The following tables use typical noise measurements with the LabJack U3 and UE9 to determine the noise-free and effective resolutions that can be expected with the LJTick-InAmp (LJTIA). The LJTIA was connected to an analog input on the LabJack and had IN+ shorted to IN- shorted to GND.

The counts of peak-to-peak noise were determined by collecting 128 points from the analog input and subtracting the minimum binary value from the maximum binary value. For the U3 these are based on 12-bit values, while for the UE9 these are based on 24-bit values.

The noise-free resolution is based on the peak-to-peak noise counts, and corresponds to the resolution where no variation would be seen.

The RMS noise counts is the standard deviation of the 128 collected binary values, and the effective resolution values are based on this RMS value. The effective resolution can be thought of as a specification met by most points, while the noise-free specifications are met by all points.

The “@LJ Inputs” values are in terms of the LabJack U3/UE9 analog input, which is the LJTIA output. Those values are divided by the LJTIA gain to determine the “@LJTIA Inputs” values, which are the resolutions that apply to the signal input to the LJTIA. For instance, a single-ended channel on the LabJack U3 with an LJTIA gain of 201 has a noise-free resolution of about 9 μV and an effective resolution of about 1.8 μV.

### LabJack U3:

Table B.1. LJTIA Gain = 1-201 with a U3.

 LJTIA Gain = 1 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts SE 2 11 0.001191 0.00119141 0.5 13 0.000298 0.00029785 Diff 2 11 0.002383 0.00238281 0.5 13 0.000596 0.0005958 0-3.6 2 11 0.002383 0.00238281 0.5 13 0.000596 0.0005957 LJTIA Gain = 11 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts SE 2 11 0.001191 0.00010831 0.5 13 0.000298 0.00002708 Diff 2 11 0.002383 0.00021662 0.5 13 0.000596 0.00005415 0-3.6 2 11 0.002383 0.00021662 0.5 13 0.000596 0.00005415 LJTIA Gain = 51 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts SE 2 11 0.001191 0.00002336 0.5 13 0.000298 0.00000584 Diff 2 11 0.002383 0.00004672 0.5 13 0.000596 0.00001168 0-3.6 2 11 0.002383 0.00004672 0.5 13 0.000596 0.00001168 LJTIA Gain = 201 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts SE 2 11 0.001806 0.00000898 0.5 12.7 0.000367 0.00000182 Diff 2 11 0.003612 0.00001797 0.5 12.7 0.000733 0.00000365 0-3.6 2 11 0.003612 0.00001797 0.5 12.7 0.000733 0.00000365

#### LabJack UE9 & UE9-Pro (LJTIA Gain = 1 & 11):

All "counts" data in the following UE9 tables are from 24-bit values. To equate to counts at a particular resolution (Res) use the formula counts/(2^(24-Res)). For instance, with the UE9 set to 12-bit resolution and the 0-5 volt range, there are 8192 counts of noise when looking at 24-bit values. To equate this to 12-bit data, we take 8192/(2^12), which equals 2 counts of noise when looking at 12-bit values.

Table B.2. LJTIA Gain = 1, Resolution 0-18 with a UE9

 Resolution=0-12, LJTIA Gain=1 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 12288 10.4 0.0037 0.00370048 2350 12.8 0.000701 0.00070111 0-2.5 20480 9.7 0.003006 0.00300572 4100 12 0.00061 0.00061035 Resolution = 17,LJTIA Gain = 1 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 408 15.3 0.000124 0.00012394 78 17.7 0.000023 0.00002348 0-2.5 620 14.7 0.000094 0.00009393 120 17.1 0.00018 0.0000178 Resolution = 18+ (UE9-PRO, LJTIA Gain = 1) Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 96 17.4 0.000029 0.00002891 20 19.7 0.000006 0.00000587

#### Table B.3. LJTIA Gain = 11, Resolution 0-18 with a UE9

 Resolution = 0-12, LJTIA Gain = 11 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 12288 10.4 0.0037 0.00033641 2350 12.8 0.000701 0.00006374 0-2.5 20480 9.7 0.003006 0.00027325 4100 12 0.00061 0.00005549 Resolution = 17,LJTIA Gain = 11 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 408 15.3 0.000124 0.00001127 78 17.7 0.000023 0.00000213 0-2.5 620 14.7 0.000094 0.00000854 120 17.1 0.00018 0.00000162 Resolution = 18+ (UE9-PRO, LJTIA Gain = 11) Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 110 17.2 0.000033 0.00000302 20 19.7 0.000006 0.00000053

#### LabJack UE9 & UE9-Pro (LJTIA Gain = 51 & 201):

All "counts" data in the following UE9 tables are from 24-bit values. To equate to counts at a particular resolution (Res) use the formula counts/(2^(24-Res)). For instance, with the UE9 set to 12-bit resolution and the 0-5 volt range, there are 8192 counts of noise when looking at 24-bit values. To equate this to 12-bit data, we take 8192/(2^12), which equals 2 counts of noise when looking at 12-bit values.

Table B.4. LJTIA Gain = 51, Resolution 0-18 with a UE9

 Resolution = 0-12, LJTIA Gain = 51 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 12288 10.4 0.0037 0.00007256 2350 12.8 0.000701 0.00001375 0-2.5 20480 9.7 0.003006 0.0005894 4100 12 0.00061 0.00001197 Resolution = 17, LJTIA Gain = 51 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 472 15.1 0.000142 0.00000279 87 17.6 0.000025 0.00000049 0-2.5 776 14.4 0.000116 0.00000227 147 16.8 0.000022 0.00000043 Resolution = 18+ (UE9-PRO), LJTIA Gain = 51 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 172 16.6 0.00005 0.00000099 29 19.2 0.000008 0.00000016

Table B.5. LJTIA Gain = 201, Resolution 0-18 with a UE9

 Resolution = 0-12, LJTIA Gain = 201 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 16384 10 0.004883 0.00002429 3500 12.2 0.001063 0.00000529 0-2.5 32768 9 0.004883 0.00002429 6480 11.3 0.00061 0.00000493 Resolution = 17, LJTIA Gain = 201 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 1232 13.7 0.000376 0.00000187 185 16.5 0.000054 0.00000027 0-2.5 2104 13 0.000305 0.00000152 376 15.4 0.000058 0.00000029 Resolution = 18+ (UE9-PRO), LJTIA Gain = 201 Range Peak-to-peak Noise Noise-Free Resolution Noise-Free Res. @LJ Inputs Noise-Free Res. @LJTIA Inputs RMS Noise Effective Resolution Effective Res. @LJ Inputs Effective Res. @LJTIA Inputs volts counts bits volts volts counts bits volts volts 0-5 484 15.1 0.000142 0.00000071 106 17.3 0.000031 0.00000015