« Close

Datasheets and User Guides

App Notes

Software & Driver

 

4.3.7 - Stream Mode

The highest input data rates are obtained in stream mode. The following IOTypes are used to control streaming:


LJ_ioCLEAR_STREAM_CHANNELS
LJ_ioADD_STREAM_CHANNEL
LJ_ioADD_STREAM_CHANNEL_DIFF  //Put negative channel in x1.
LJ_ioSTART_STREAM             //Value returns actual scan rate.
LJ_ioSTOP_STREAM
LJ_ioGET_STREAM_DATA

The following constant is passed in the Channel parameter with the get stream data IOType to specify a read returning all scanned channels, rather than retrieving each scanned channel separately:


LJ_chALL_CHANNELS  //Used with LJ_ioGET_STREAM_DATA.

The following are special channels, used with the get/put config IOTypes, to write or read various stream values:


LJ_chSTREAM_SCAN_FREQUENCY
LJ_chSTREAM_BUFFER_SIZE           //UD driver stream buffer size in samples.
LJ_chSTREAM_WAIT_MODE
LJ_chSTREAM_DISABLE_AUTORECOVERY
LJ_chSTREAM_BACKLOG_COMM          //Read-only.  0=0% and 256=100%.
LJ_chSTREAM_BACKLOG_UD            //Read-only.  Number of samples.
LJ_chSTREAM_SAMPLES_PER_PACKET    //Default 25. Range 1-25.
LJ_chSTREAM_READS_PER_SECOND      //Default 25.
LJ_chAIN_STREAM_SETTLING_TIME     //0=Auto.  See Section 5.2.12.

With the wait mode special channel above, the following constants are passed in the value parameter to select the behavior when reading data:


LJ_swNONE         //No wait.  Immediately return available data.
LJ_swALL_OR_NONE  //No wait.  Immediately return requested amount, or none.
LJ_swPUMP         //Advanced message pump wait mode.
LJ_swSLEEP        //Wait until requested amount available.

The backlog special channels return information about how much data is left in the stream buffer on the U6 or in the UD driver. These parameters are updated whenever a stream packet is read by the driver, and thus might not exactly reflect the current state of the buffers, but can be useful to detect problems.

When streaming, the processor acquires data at precise intervals, and transfers it to a buffer on the U6 itself. The U6 has a small buffer (512-984 samples) for data waiting to be transferred to the host. The LJ_chSTREAM_BACKLOG_COMM special channel specifies how much data is left in the U6 buffer (COMM and CONTROL are the same thing on the U6), where 0 means 0% full and 256 would mean 100% full. The UD driver retrieves stream data from the U6 in the background, but if the computer or communication link is too slow for some reason, the driver might not be able to read the data as fast as the U6 is acquiring it, and thus there will be data left over in the U6 buffer.

To obtain the maximum stream rates documented in Section 3.2, the data must be transferred between host and U6 in large chunks. The amount of data transferred per low-level packet is controlled by LJ_chSTREAM_SAMPLES_PER_PACKET. The driver will use the parameter LJ_chSTREAM_READS_PER_SECOND to determine how many low-level packets to retrieve per read.

The size of the UD stream buffer on the host is controlled by LJ_chSTREAM_BUFFER_SIZE. The application software on the host must read data out of the UD stream buffer fast enough to prevent overflow. After each read, use LJ_chSTREAM_BACKLOG_UD to determine how many samples are left in the buffer.

Since the data buffer on the U6 is fairly small a feature called auto-recovery is used. If the buffer overflows, the U6 will continue streaming but discard data until the buffer is emptied, and then data will be stored in the buffer again. The U6 keeps track of how many packets are discarded and reports that value. Based on the number of packets discarded, the UD driver adds the proper number of dummy samples (-9999.0) such that the correct timing is maintained. Auto-recovery will generally occur when the U6 buffer is 90-95% full.

In stream mode the LabJack acquires inputs at a fixed interval, controlled by the hardware clock on the device itself, and stores the data in a buffer. The LabJackUD driver automatically reads data from the hardware buffer and stores it in a PC RAM buffer until requested. The general procedure for streaming is:

  1. Update configuration parameters.
  2. Build the scan list.
  3. Start the stream.
  4. Periodically retrieve stream data in a loop.
  5. Stop the stream.

Following is example pseudocode to configure a 2-channel stream.


//Set the scan rate.
AddRequest (lngHandle, LJ_ioPUT_CONFIG, LJ_chSTREAM_SCAN_FREQUENCY, scanRate, 0, 0);

//Give the UD driver a 5 second buffer (scanRate * 2 channels * 5 seconds).
AddRequest (lngHandle, LJ_ioPUT_CONFIG, LJ_chSTREAM_BUFFER_SIZE, scanRate*2*5, 0, 0);

//Configure reads to wait and retrieve the desired amount of data.
AddRequest (lngHandle, LJ_ioPUT_CONFIG, LJ_chSTREAM_WAIT_MODE, LJ_swSLEEP, 0, 0);

//Define the scan list as singled ended AIN2 then differential AIN0-AIN1.
AddRequest (lngHandle, LJ_ioCLEAR_STREAM_CHANNELS, 0, 0, 0, 0);
AddRequest (lngHandle, LJ_ioADD_STREAM_CHANNEL, 2, 0, 0, 0);
AddRequest (lngHandle, LJ_ioADD_STREAM_CHANNEL_DIFF, 0, 0, 1, 0);

//Execute the requests.
GoOne (lngHandle);

Next, start the stream:


//Start the stream.
eGet(lngHandle, LJ_ioSTART_STREAM, 0, &dblValue, 0);

//The actual scan rate is dependent on how the desired scan rate divides into
//the LabJack clock.  The actual scan rate is returned in the value parameter
//from the start stream command.
actualScanRate = dblValue;
actualSampleRate = 2*dblValue;

Once a stream is started, the data must be retrieved periodically to prevent the buffer from overflowing. To retrieve data, add a request with IOType LJ_ioGET_STREAM_DATA. The Channel parameter should be LJ_chALL_CHANNELS or a specific channel number (ignored for a single channel stream). The Value parameter should be the number of scans (all channels) or samples (single channel) to retrieve. The x1 parameter should be a pointer to an array that has been initialized to a sufficient size. Keep in mind that the required number of elements if retrieving all channels is number of scans * number of channels.

Data is stored interleaved across all streaming channels. In other words, if two channels are streaming, 0 and 1, and LJ_chALL_CHANNELS is the channel number for the read request, the data will be returned as Channel0, Channel1, Channel0, Channel1, etc. Once the data is read it is removed from the internal buffer, and the next read will give new data.

If multiple channels are being streamed, data can be retrieved one channel at a time by passing a specific channel number in the request. In this case the data is not removed from the internal buffer until the last channel in the scan is requested. Reading the data from the last channel (not necessarily all channels) is the trigger that causes the block of data to be removed from the buffer. This means that if three channels are streaming, 0, 1 and 2 (in that order in the scan list), and data is requested from channel 0, then channel 1, then channel 0 again, the request for channel 0 the second time will return the same data as the first request. New data will not be retrieved until after channel 2 is read, since channel 2 is last in the scan list. If the first get stream data request is for 10 samples from channel 1, the reads from channels 0 and 2 also must be for 10 samples. Note that when reading stream data one channel at a time (not using LJ_chALL_CHANNELS), the scan list cannot have duplicate channel numbers.

There are three basic wait modes for retrieving the data:

  1. LJ_swNONE: The Go call will retrieve whatever data is available at the time of the call up to the requested amount of data. A Get command should be called to determine how many scans were retrieved. This is generally used with a software timed read interval. The number of samples read per loop iteration will vary, but the time per loop iteration will be pretty consistent. Since the LabJack clock could be faster than the PC clock, it is recommended to request more scans than are expected each time so that the application does not get behind.
  2. LJ_swSLEEP: This makes the Go command a blocking call. The Go command will loop until the requested amount of is retrieved or no new data arrives from the device before timeout. In this mode, the hardware dictates the timing of the application ... you generally do not want to add a software delay in the read loop. The time per loop iteration will vary, but the number of samples read per loop will be the same every time. A Get command should be called to determine whether all the data was retrieved, or a timeout condition occurred and none of the data was retrieved.
  3. LJ_swALL_OR_NONE: If available, the Go call will retrieve the amount of data requested, otherwise it will retrieve no data. A Get command should be called to determine whether all the data was returned or none. This could be a good mode if hardware timed execution is desirable, but without the application continuously waiting in SLEEP mode.

The following pseudocode reads data continuously in SLEEP mode as configured above:


//Read data until done.
while(!done)
{
    //Must set the number of scans to read each iteration, as the read
    //returns the actual number read.
    numScans = 1000;

    //Read the data.  Note that the array passed must be sized to hold
    //enough SAMPLES, and the Value passed specifies the number of SCANS
    //to read.
    eGetPtr(lngHandle, LJ_ioGET_STREAM_DATA, LJ_chALL_CHANNELS, &numScans, array);
    actualNumberRead = numScans;

    //When all channels are retrieved in a single read, the data
    //is interleaved in a 1-dimensional array.  The following lines
    //get the first sample from each channel.
    channelA = array[0];
    channelB = array[1];

    //Retrieve the current U6 backlog.  The UD driver retrieves
    //stream data from the U6 in the background, but if the computer
    //is too slow for some reason the driver might not be able to read
    //the data as fast as the U6 is acquiring it, and thus there will
    //be data left over in the U6 buffer.
    eGet(lngHandle, LJ_ioGET_CONFIG, LJ_chSTREAM_BACKLOG_COMM, &dblCommBacklog, 0);

    //Retrieve the current UD driver backlog.  If this is growing, then
    //the application software is not pulling data from the UD driver
    //fast enough.
    eGet(lngHandle, LJ_ioGET_CONFIG, LJ_chSTREAM_BACKLOG_UD, &dblUDBacklog, 0);
}

Finally, stop the stream:


//Stop the stream.
errorcode = ePut (Handle, LJ_ioSTOP_STREAM, 0, 0, 0);

5 comments

What is the recovery procedure if one encounters a STREAM_CHECKSUM_ERROR? Do we have to completely re-initialize the LabJack connection and start over, or is there a way to reset just the stream and then keep going?

I've tried to do STOP_STREAM/CLEAR_STREAM_CHANNELS/ADD_STREAM_CHANNEL/START_STREAM, but that does not seem to have any effect, and the next read using GET_STREAM_DATA still fails.

This is using a single channel stream, reading AIN0 on a UE9.

This was over USB, and just doing STOP and START STREAM did not recover. It is using latest UD driver, with C# code.

I did upgrade both communication and control firmware after posing the question here, and I have not (yet) seen it occur again.

I am using the LJ_chSTREAM_BACKLOG_UD and LJ_chSTREAM_BACKLOG_COMM to find out when it makes sense for me to transfer data from my LabJack U6(HW 2.0) to the computer. During that some questions arose:

1. Reading the COMM-channel always shows me 0 even when there are samples in the buffer. What could be the reason for that? Does the COMM-channel not work yet?

2. What does the value in the UD-channel exactly mean? Is it the number of samples, the number of scans or the number of bytes in the buffer? I was not able to clearly find that out on my own.

In some special configuration I also got an error 1001. I could not figure out why this happens. But I will open a new thread on that. I will also provide some code so that you can try to reproduce this on your own.

I use Windows 7 64-bit but all the problems also appear on Windows XP 32-bit. I have the latest driver(SW 3.25) and firmware(FW 1.35) installed. I am using Qt and the wrapper some other user already provided to use the DLL. Compiler is MSVC 2008.

Hope that you are able to give me some hints. Thanks in advance.

The LJ_chSTREAM_BACKLOG_COMM shows the backlog currently buffered on the U6 waiting to be sent to the PC.  This is typically zero unless you are streaming very fast and the PC is blocked.  The buffer on the U6 isn't the largest (due to not having a ton of extra memory) and thus it's designed to offload that data to the PC as fast as possible so problems don't occur.  The driver is very efficient at this, thus why it's almost always zero.  It shouldn't typically be larger than that under normal operation.

LJ_chSTREAM_BACKLOG_UD is the driver's internal buffer position.  It's the number of samples currently inside the buffer, available to be read from the driver.  So technically it's the number of samples (doubles) that can be read out at the current moment.