Script that logs data for 10 seconds every 10 minutes


bbosse

Recommended Posts

Hello there!

I'm trying to log data for 10 seconds every 10 minutes - each 10 seconds of data saved to its own separate file.

So I have 2 problems with my script pasted below:

1) For some reason the file will have the first 6 or 7 seconds of data and not the entire 10 seconds

 

2) but even more annoying is that when the 10 minutes is up and it is supposed to switch to a new file, for some strange reason it records the first second or two in the previous file before finally making the switch to a new file.  It's like it starts logging in the file from 10 minutes ago for a second or two before finally opening a new file to begin logging.

 

I've tried troubleshooting this in a variety of ways but still have been unable to get 10 seconds of data to save in its own file every 10 minutes.

Any help is appreciated, thank you.

 

Logging.DataLog.FileName = "D:\ALT DAQFactory Saves\ALT_" + formatDateTime("%y%m%d_%H%M%S", systime()) + ".csv"
beginlogging(DataLog)

// Loop to read stream data and append it to the DF Channels.
while(1)
   dataIn = LJM_eStreamRead(identifier)
   
   if (systime() <= timestamp + 10) // If the first 10 seconds of every 10 minutes, log the data - otherwise don't read the data

      data = insertTime(dataIn.data.AIN0, st, 1 / scanRate)
      Channel0.AddValue(data)  // Needs to match channel defined in "Channels".
      
      data = insertTime(dataIn.data.AIN1, st, 1 / scanRate)
      Channel1.addValue(data)
      
      data = insertTime(dataIn.data.AIN2, st, 1 / scanRate)
      Channel2.addValue(data)
      
      data = insertTime(dataIn.data.AIN3, st, 1 / scanRate)
      Channel3.addValue(data)
      
      data = insertTime(dataIn.data.AIN4, st, 1 / scanRate)
      Channel4.addValue(data)
      
      data = insertTime(dataIn.data.AIN5, st, 1 / scanRate)
      Channel5.addValue(data)
      
      data = insertTime(dataIn.data.AIN6, st, 1 / scanRate)
      data = data*55.45 - 17.78 // Convert channel voltage from mV to degrees C
      Channel6.addValue(data)
      
      data = insertTime(dataIn.data.AIN7, st, 1 / scanRate)
      data = data*55.45 - 17.78 // Convert channel voltage from mV to degrees C
      Channel7.addValue(data)
      
   endif
   
   
   if (systime() >= timestamp + 600) // dont log data for next 10 minutes (600 seconds)
      // From the DAQFactory Manual: "changing the file name while the logging or export set is running will close the current file and open the new file"
      // We change the filename for each new 10 minute interval of data
      Logging.DataLog.FileName = "D:\ALT DAQFactory Saves\ALT_" + formatDateTime("%y%m%d_%H%M%S", systime()) + ".csv"
      
      timestamp = systime() // 10 minutes complete - reset timestamp to current system time
      saveCount += 1 // 10 minutes completed! - update saveCount
   endif
   
   st += scansPerRead / scanRate
   backlog = dataIn.ljmscanBacklog
   timeLeft = 600 - (systime() - timestamp)
endwhile

 

Link to comment
Share on other sites

Logging sets aren't going to be a good fit for this application.  For optimization reasons, to allow for logging of high speed data, the logging sets cache data before writing to disk.  This is why you see several seconds of latency between the start/stop and the actual time stamps in the file.  Logging sets are really designed to be started and left running.

For your application I would either use an export set where each line has [0] so it logs just one line of data, then trigger it from a loop in your script, or do as I do and simply use the File. functions to log the data exactly how you want it, again within a loop in your script.  The File. functions will give you the most control, basically allowing you to log data however you want.

Fortunately you are already in a loop in your data acquisition, so you would put that script right into that loop, probably inside your if().

Link to comment
Share on other sites

Cool.  Thanks.  I have switched to using the direct File writing.

Code pasted below in case it helps someone else.

Let me know what you think :)

// Loop to read stream data and append it to the DF Channels.
while(1)
   dataIn = LJM_eStreamRead(identifier)
   
   if (systime() <= timestamp + 10) // If the first 10 seconds of every 10 minutes, log the data - otherwise don't read the data
      data = insertTime(dataIn.data.AIN0, st, 1 / scanRate)
      Channel0.AddValue(data)  // Needs to match channel defined in "Channels".
      
      data = insertTime(dataIn.data.AIN1, st, 1 / scanRate)
      Channel1.addValue(data)
      
      data = insertTime(dataIn.data.AIN2, st, 1 / scanRate)
      Channel2.addValue(data)
   
      data = insertTime(dataIn.data.AIN3, st, 1 / scanRate)
      Channel3.addValue(data)
      
      data = insertTime(dataIn.data.AIN4, st, 1 / scanRate)
      Channel4.addValue(data)
      
      data = insertTime(dataIn.data.AIN5, st, 1 / scanRate)
      Channel5.addValue(data)
      
      data = insertTime(dataIn.data.AIN6, st, 1 / scanRate)
      data = data*55.45 - 17.78 // Convert channel voltage from mV to degrees C
      Channel6.addValue(data)
      
      data = insertTime(dataIn.data.AIN7, st, 1 / scanRate)
      data = data*55.45 - 17.78 // Convert channel voltage from mV to degrees C
      Channel7.addValue(data)
   endif
   
   if (systime() > timestamp + 10 && notsavedyet == 1)
      // Open File and get handlePrivate 
      Private.handle = File.Open("D:\ALT DAQFactory Saves\ALTfile_" + formatDateTime("%y%m%d_%H%M%S", systime()) + ".csv",0,1,1,1)

      // Put all the channel data into one array
      myArray[][0] = Channel0.Time
      myArray[][1] = floor(Channel0 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][2] = floor(Channel1 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][3] = floor(Channel2 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][4] = floor(Channel3 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][5] = floor(Channel4 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][6] = floor(Channel5 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][7] = floor(Channel6 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      myArray[][8] = floor(Channel7 * 1000 + 0.5) / 1000 // rounds to 3 decimals
      
      // Write the array to the file
      File.WriteDelim(Private.handle, myArray, ",", chr(10))
      
      // once saved, dont repeat until next read
      notsavedyet = 0
      
      // close the file after it has been written
      File.Close(Private.handle)
   endif
   
   
   if (systime() >= timestamp + 600) // dont log data for next 10 minutes (600 seconds)
      timestamp = systime() // 10 minutes complete - reset timestamp to current system time
      saveCount += 1 // 10 minutes completed! - update saveCount
      notsavedyet = 1
   endif
   
   st += scansPerRead / scanRate
   backlog = dataIn.ljmscanBacklog
   timeLeft = 600 - (systime() - timestamp)
endwhile

 

Link to comment
Share on other sites

Now that I'm saving data directly to file, I'm trying to figure out how to save the time points with microsecond precision. 

I see from the manual the the data logging (which I was previously using) has the option to change the Sig Figs:

" Time Sig Figs: Determines how many significant figures are used to write time values. A value of 9 yields timeprecise to the second, 12 to the millisecond and 15 to the microsecond. The maximum value is 15. "

But now that I'm using the File.WriteDelim function to save my data, how do I specify that I want the time to be saved with 14 sig figs?

 

Thank you!

Link to comment
Share on other sites

Use the format() function.  It's not really sig figs, but rather # of digitals after the decimal.  So, for, say, microsecond precision, you'd do:

format("%.6f", mytime[0])

Format() pretty much follows the rather universal C printf() statement, so there is a lot of examples on the web.

Link to comment
Share on other sites

Hmm, okay well it worked in the sense that I can adjust the number of decimal places saved (e.g. I tried format("%.1f", Channel0.Time)  and got 1 decimal place).

But when I tried format("%.6f", Channel0.Time) I still only got 3 decimal places, which leads me to believe that the Channel0 is only generating 3 decimal places so there's nothing to show beyond 3.  I looked into the Channel settings to see if I could specify the number of decimals, but didn't see anything.

image.png.1ed69c0da027df60dbf53ab6eb2ba560.pngimage.png.e65409c381b0d9d54ce4ab7f5fd2a8af.png

Is there a way I can specify that Channel0 saves up to 4 decimal places?

Thank you!

Link to comment
Share on other sites

I don't know.  It works for me when I create a channel, then in Command / Alert do:

? format("%.6f", myChannel.time[0])

It prints something like:

1556486246.000109

The time is stored internally as a 64 bit floating point value, so you should be able to get 6 decimals places.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.