Help with hex

[silogarrett]

Hello, I have a noob question, I'm afraid:

I have a serial device that is streaming data in every 50ms. It requires a 'stop' command to shut off the firehose of data. The RS-232 command is SO0#0A hex, and the one to resume is SO1#0A.

The commands work great on my various terminal emulators. I can stop and start no problem.

I cannot, however, get either monitor of DF to stop the stream. I've tried all kinds of combinations of hex and ASCII translations, including \x53\x4F\30\x23\x30\x41\x0A, with and without the 'req'. The Tx occurs, but has no effect.

Please help. Is it something to do with the configuration?

Mike

[AzeoTech]

My guess is you want:

SO0#\x0a

Basically that 0A in your description is actually a line feed character.

Its also possible that its just:

SO0\x0a

I believe in VB they use # for hex, and perhaps your docs for your device was written by a VB guy so used the rather non-standard #0a for line feed.

[silogarrett]

I found the problem. For me, someone with little experience, I find most of my problems always boil down to syntax.

SO0#0A should have been expressed as SO0\r\n. It took me awhile to figure out I needed the CR+LF.

So, in decimal, it ended up being 837948\r\n.

Works like a charm. Sometimes you just don't know what you don't know.

Thanks for your help.

[silogarrett]

I need your help. I have tried everything I can find on the forum and in the manual, to no avail.

The device I'm trying to comm with is a Ludlum 12-channel radiation counter. It streams counts and other info out every 50ms. The Ludlum software that controls it sums those counts in the various channels and presents the user with an accumulated count of whatever time period you wish - one second, 10 seconds, etc., for each channel.

Now that I have the right RS-232 stop/start commands for it, and com configurations, I can see the stream on both DF comm windows, and I can see the data as hex and/or decimal values no problem. I just can't seem to find a way to get the data into a channel that's readable. The best I can do is NaN using the examples in the manual.

The Ludlum manual (attached) states the unit can be controlled by various RS232 commands, but the only ones I can make work in a recognizable way are the stop and start, SO0\r\n and SO1\r\n (in decimal form, such as 837948\r\n).

I'm sure I can sum counts in specified time increments if I can just get the data into channels! My channel set up is Chan1, Chan2, and so on to Chan12. I've found that the asynchronous ASCII protocol will give me 'input' as a I/O option, where null gives me no I/O option at all. It's so frustrating.

The data that the Ludlum software writes to a .csv file looks like this:

SerialNumber,Group,Channel,CountTime,Count,HV,LLD,ULD,Efficiency,Date

254492,01,01,00:00:05.000,150,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,02,00:00:05.000,182,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,03,00:00:05.000,230,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,04,00:00:05.000,226,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,05,00:00:05.000,340,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,06,00:00:05.000,1065,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,07,00:00:05.000,5704,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,08,00:00:05.000,24834,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,09,00:00:05.000,2291,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,10,00:00:05.000,621,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,11,00:00:05.000,316,0900,0100,3300,99.9,07/11/2010 14:19:03

254492,01,12,00:00:05.000,0,0800,0100,3300,99.9,07/11/2010 14:19:03

It looks like a simple parsing problem, but I can't for the life of me make it work. Trying to parse the 50ms output locks up DF real fast.

What can I try?

Mike

[silogarrett]

I couldn't upload the manual. Trying again.

Says: "You did not select a file to upload" even though I did, six times. Oh well, just my luck.

The link to the document is here:

http://ludlums.com/images/stories/product_..._April-2009.pdf

[AzeoTech]

Can you just post a snippet from the comm monitor? Pause it if you have to before copying.

[silogarrett]

Wow, I never expected someone to answer a post at 2200 hours on a Sunday! Thank you!!!! What great customer service!!!

HERE IT IS IN DECIMAL/ASCII

Tx: SO01310

Tx: SO11310

Rx: 000001000000000002000000000000000002000001000004000000000000000001000000\128\128\128\128\128\128\128\128\128\128\128\1281310
000002000001000003000003000003000003000003000000000003000003000003000000\128\128\128\128\128\128\128\128\128\128\128\1281310
000000000001000002000000000002000004000001000000000003000000000000000000\128\128\128\128\128\128\128\128\128\128\128\1281310


AND HERE IT IS IN HEX

Tx: SO0\x0D\x0A

Tx: SO1\x0D\x0A

Rx: \x00\x00\x02\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x01\x00\x00\x04\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x01\x00\x00\x01\x00\x00\x00\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x0D\x0A
\x00\x00\x01\x00\x00\x03\x00\x00\x02\x00\x00\x01\x00\x00\x01\x00\x00\x01\x00\x00\x02\x00\x00\x00\x00\x00\x02\x00\x00\x01\x00\x00\x01\x00\x00\x00\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x0D\x0A
\x00\x00\x02\x00\x00\x02\x00\x00\x04\x00\x00\x01\x00\x00\x03\x00\x00\x01\x00\x00\x04\x00\x00\x01\x00\x00\x00\x00\x00\x01\x00\x00\x02\x00\x00\x00\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x80\x0D\x0A

Your help is much appreciated!

Mike

[AzeoTech]

Well, sunday night is monday in much of the world, and more importantly, most places DAQFactory is used don't stop running on weekends and holidays!

Anyhow, you have two choices: you can try and split the data into individual channels, which, depending on your PC, it might not be able to do at 50hz, or you can create a single Spectrum channel to hold all 50 elements. Spectral data has its advantages, depending on what you want to do with it. For example, you can easily do waterfall graphs, do graphs of all 50 channels at once (single time), etc.

To read into spectrum, create a spectral test channel with Timing = 0. Then create a little sequence that looks like this:

private string datain
// get us lined up:
device.mydevice.purge()
datain = device.mydevice.readuntil(10)
while(1)
   try
	  datain = device.mydevice.readuntil(10)
	  myspecChannel.addvalue(tranpose(asca(left(datain,50)),1))
   catch()
	  ? strLastError
	  delay(0.5)
	  // realign:
	  device.mydevice.purge()
	  datain = device.mydevice.readuntil(10)
   endcatch
endwhile

I obviously couldn't test this, but that's about what it should be. If you want to go the other way (individual channels), it'd look similar, but the data processing part would be different, and you'd likely have to put the delay() and realignment outside the catch() so it happened every time to avoid stalling the computer.

Make sure you save before running the sequence because its possible the sequence will stall the PC trying to keep up with the data. If you find that the DF displayed data is lagging behind reality, its because the PC can't keep up. You can likely unfreeze the PC if you hang it by unplugging the serial port.

[silogarrett]

It didn't hang the box - it's a quad core, so maybe that helped.

It did give me some errors (see below) and the channel data alternates between a huge single column of NaN and one row of the data, at seemingly random times.

I wonder if getting the data into individual channels might be better? Here's the errors:

C1127 Infinite loop detected. 100 steps executed in 0.846952 seconds

C1127 Infinite loop detected. 100 steps executed in 0.846500 seconds

07/12/10 09:20:54.508

C1136 Timeout: test Line 7 - Uncaught error in sequence test

Even with this result we've made huge progress...

[AzeoTech]

First: turn off the sequence loop check. Its off by default. It under File - Settings. That's why your PC didn't hang (though it still might with it off.

Second: most likely you have some old code running that's processing the data too and generating those NaN's. Try starting from a blank document.

Third: it looks like when I made the post I missed a line when bracketing it for "CODE" on the forum. The first line was there, but outside the block so you probably missed it. I've corrected it. It is the declaration for datain. Because of this my code wouldn't do anything, so I'm guessing there is other code elsewhere that's confusing you. Start from a blank document...

[silogarrett]

Blank document, sequence loop check off, I caught the datain thing. Set protocol to NULL and I get a good spectral array of data and it looks to be near real time.

I put a small radioactive source under one of the detectors (which increases count rate, and is how we check channel response) and suddenly I only get random numbers of rows - sometimes only one, sometimes more, mostly only up to 27 or so.

An increase in counts on one value is wigging out the rest of the data! Yikes.

I'm also finding that the reason I have 50 bytes is that the count for all twelve channels is stored in 3 bytes. From the manual:

Byte 01

[AzeoTech]

I'm assuming by rows you actually mean columns. A row in DAQFactory would be the equivalent of a single reading from the device (a set of 12 measurements). Anyhow, if you are getting shorter columns you have a different issue. First, try and see if you can see the traffic from the device and see if it is still coming through whole. If its not, then we have one issue, if it is, we have another.

Either way, you are probably going to have to slow the thing down because of the time it will take to merge those bytes and there isn't a built in DF function that will do it quickly. The code will change to something like:

private string datain
private data
private out
// get us lined up:
while(1)
   try
	  // get us lined up:
	  device.mydevice.purge()
	  datain = device.mydevice.readuntil(10)
	  // and read a full line:
	  datain = device.mydevice.readuntil(10)
	  if (getlength(datain) != 50)
		 ? "didn't get a whole data packet!"
		 delay(0.5)
		 continue
	  endif
	  data = asca(left(datain,36))
	  for(private i = 0, i < 36, i+=3)
		 out[0][i/3] = data[i] * 65536 + data[i+1] * 256 + data[i+2]
	  endfor
	  out.time = systime()
	  myspecChannel.addvalue(out)
   catch()
	  ? strLastError
   endcatch
   delay(0.5)
endwhile

Something like that (off the cuff again). This will only read a data point every 0.5 seconds and will throw away the rest of the readings (the purge() / readuntil(10) alignment code). It also does the rest of the processing. I still made it use a spectra because I think you'll want your data that way.

[silogarrett]

No, it was definitely rows, but the new code straightened things out right away! I tested each detector with a source, I saw the rise in count rate in each one, and no streaming problems.

What could be the other issue? Data quality, because we're sampling instead of summing?

Holy cow - I'm miles ahead. Thank you for working through this with me!!

[AzeoTech]

What other issue?

[silogarrett]

You said "try and see if you can see the traffic from the device and see if it is still coming through whole. If its not, then we have one issue, if it is, we have another."

I assumed that because the new code and the calculation works, that there was another issue. Maybe I'm mistaken.

I'm going to try to push the half second to a faster interval - do you see a problem or a limit to that?

I'm curious - is there a way to read the .csv that the Ludlum software is writing?

Thank you again for your help. DF makes me look like a pro - but its the support we get from you that makes it so.

[AzeoTech]

I didn't mean you had two issues, but rather one or the other. It appears we've solved that one issue, so we're good to go.

The 0.5 delay was largely arbitrary. Don't go faster than 0.04 (25hz). Anything faster won't help anyway because of the way the script is written.

You probably can read that file using File. functions. It depends if they are locking it against reads.

[silogarrett]

Gotcha. Thank you again.

There is a line in the code:

out[0][i/3] = data * 65536 + data[i+1] * 256 + data[i+2]

Is the expression [i/3] dividing the sum of the three byte sets?

[AzeoTech]

No, its just indexing into the spectra, so [0][0] is the sum of i = 0 to i=2, [0][1] is the sum of i = 3 to i=5, etc.

[silogarrett]

Awesome support. Thanks!!!

[silogarrett]

I was able to flash the counter with new firmware that outputs counts every second in the following format:

"07/13/10","4:13:42 PM",1,0,1,0,0,0,0,0,0,0,0,0,0,0,0

Where

Field 1 is the Date,

Field 2 is the Time

Field 3 is the Count Time

Field 4 is the Time Range

Field 5 is the Output Time

Field 6 is Channel 1 Count

Field 7 is Channel 2 Count

ad nauseum until

Field 17 is Channel 12 Count

A sample of the monitor window shows this:

Rx: 00001100001600002300001500001800002900001500001800001700002300002500000000000001

1\100010000001310

Rx: 00001700002100002600002400002800002700001800002100002500001500002500000000000001

1\100010000001310

Rx: 00001700002100002600002900003000001900001600003100002000001800002000000000000001

1\100010000001310

My feeble attempt at code is this:

private string datain
private data
device.M4612.purge()
while(1)
   try  
	  // data should be streaming, read a packet:
	  datain = device.M4612.readuntil(48)  //while I actually only count 47
	  private data = parse(datain,1,",")
	  // convert the packet from a string to an array of bytes:
	  data = asca(datain)
	  data.time = systime()
	  CHAN1.addvalue(strtodouble(datain[10,12]) // from char 10 to 12, and so on
	  CHAN2.addvalue(strtodouble(datain[13,15])
	  CHAN3.addvalue(strtodouble(datain[16,18])
	  CHAN4.addvalue(strtodouble(datain[19,21])
	  CHAN5.addvalue(strtodouble(datain[22,24])
	  CHAN6.addvalue(strtodouble(datain[25,27])
	  CHAN7.addvalue(strtodouble(datain[28,30])
	  CHAN8.addvalue(strtodouble(datain[31,33])
	  CHAN9.addvalue(strtodouble(datain[34,36])
	  CHAN10.addvalue(strtodouble(datain[37,39])
	  CHAN11.addvalue(strtodouble(datain[40,42])
	  CHAN12.addvalue(strtodouble(datain[43,45])
   catch()
	  ? strLastError
	  delay(1)
   endcatch
endwhile

I get the C1000 Channel or function not found: TEST Line 12. I thought the flash would make things easier to read (and have better data), but I'm lost once again. Can you help me out of the ditch once again?

Mike

[AzeoTech]

First, that output doesn't look anything like what you said it would look like. It still looks binary, where your reflash should have converted the output to an ASCII string of real characters.

Second, you can't subset a string like that to get certain characters. datain[10,12] doesn't give you the 11th through 13th characters (remember everything is numbered from 0), but instead gives you the 11th through 13th array element of an array of strings. Since datain is a single string (array length = 1), there is no 11th through 13th array element, and thus the error.

Once you get the output to actually look like what you said, you'll want to use the parse() function on datain to split it by comma. This is a pretty standard output format and I'm pretty sure there are sample scripts on doing it in this forum.

[silogarrett]

I understand. Thank you. I'll see what I can do.

[silogarrett]

Hello again.

Is

datain = device.mydevice.readuntil(13+10)

valid syntax for reading until the CR+LF?

I'm getting situations where '10' randomly occurs in my one second data inputs and that particular packet is not being logging into the channel. I'm thinking that forcing DF to look for the CR+LF might cure that.

[AzeoTech]

No. It only looks for one character. You are actually getting a binary 10 in your packet? Didn't you switch to using an ASCII packet? timestamp, data, data, dataCRLF?

I have two things to say about that:

1) if you are using ascii, getting a mistaken LF character in the wrong place means you have noise in the line since LF should only be at the end of the packet. A counter value of 10 would be sent as ASCII 1 and ASCII 0, not binary 10.

2) if you are using binary, and you can get a LF character (binary 10) in the data stream, then the company that designed the protocol made a serious mistake. Sure, we can check to make sure you have both a CR and an LF, but what happens when your counts hit 3338 (which is 13 * 256 + 10)? This is a valid value, but will be miscontrued as the end of line terminator by any software. The DF1 protocol has ways around this, but I'm guessing your hardware isn't that sophisticated, and this is simple oversight by the engineer. If this is a possibility, you are going to have to use fixed frame widths instead of CRLF. You'd probably use CRLF to find the end of the first line, and then do read(52) to read in 50 characters (plus CRLF) each time. Of course, with our latest method, this won't work because we are purging each time.

So, you need to make sure its CRLF, and then make sure the result has the 50 characters. Then throw it away if you don't (missed reading). To do this, do:

datain = device.mydevice.readUntil(13)
private lfchar = device.mydevice.read(1)
if ((lfchar == chr(10)) && (getlength(datain) == 50))
   // valid CRLF pair, and proper length, process
   ....
endif

If this second one is true, and CRLF could exist in the middle of the packet, don't forget to complain to your hardware manufacturer about the missed 3338 counts. We actually had someone find a bug in a large manufacturer's new PLC's Modbus implementation by using DAQFactory, so this wouldn't be the first time. Fortunately, the manufacturer was quite receptive and fixed it.

[silogarrett]

Hi again, and thanks once more for your help!

I think you're correct about noise - I cleaned up power line and serial cable paths around the plant and things seem much more stable - in fact, no problems today at all.

After reading your reply, I thought maybe I should avoid depending on receiving packet length - maybe that's wrong, but hey, I'm a newbie and this is all new territory. So, I searched the manual and your posts and developed a hybrid sequence that seems to be working well - I'm sure it could use some polish:

private string datain
private data
private out
device.MODEL4612.purge()
while (1)
   try
	  // read until CR+LF
	  datain = device.MODEL4612.readuntil(Chr(013) + Chr(010))
			data = asca(left(datain,33))
	  for(private i = 0, i < 33, i+=3)
		 out[0][i/3] = data[i] * 65536 + data[i+1] * 256 + data[i+2]
	  endfor
	  out.time = systime()
	  RADDATA.addvalue(out)
   catch()
	  delay(0.1) // just in case
   endcatch
endwhile

I sure as heck don't want to put bad code on the forum, so please hammer it hard if it's wrong.

Mike