Most efficient way to populate channels Modbus

[Sanchez 0]

Hi Azeotech Guru:

I am using modbus RTU to read about 700 registers from 3 PLCs. Here is the sample code of how I am doing it:

Private.Data_Integer1 = 0

while(1)

try
   Private.Data_Integer1 = Device.ModbusRTU.ReadHoldingRegistersSSI(1,1,125)
catch("D")
endcatch


CTSU_DIFF_START.AddValue(Private.Data_Integer1[0][0])
CTSU_DIFF_STOP.AddValue(Private.Data_Integer1[0][1])
CTSU_DIFF_100.AddValue(Private.Data_Integer1[0][2])
CTSU_DIFF_NORMAL.AddValue(Private.Data_Integer1[0][3])
CTSU_INTERVAL.AddValue(Private.Data_Integer1[0][4])

//More Channels here

endwhile

I am showing only 5 channels, but I do the same for the aprox 700 channels that I have.

The problem is that this is slow, and the values on my screens don't update quickly, sometimes it takes 4 seconds or more to update. Is there a way to make it more efficient?

I have paused the alarms before reading.

I have set the Thread Priority to 0, otherwise the computer becomes too slow.

I have put a switch statement to execute only the AddValues of the channels displayed on the screen that is active.

Thanks.

[AzeoTech 0]

700 tags really shouldn't be a problem for DAQFactory even if you read them all at once instead of just when the page is displayed. The total refresh rate is largely determined by how fast your device can reply to the queries, and so reducing the number of queries by only querying values when a particular page is displayed will certainly help.

My guess is that your problem is that you have script in some of the Channel's events that is bogging things down. A readholding..() and bunch of ..AddValue() should run fast and not take much processor power.

[Sanchez 0]

I didn't put any script in the channel event tabs, unless it happened by accident, I have to check one by one.

Another thing that I used is execute instrucctions.

//Zones++++++++++++++++++++++++++++++++++++++++++++++++++++

private.zone_counter1 = 45
private.zone_counter2 = 62
for(private.i = 1, private.i < (var.numberzones + 1), private.i++) 
   //Zone Error 
   execute("IPS_Zone_" + ...) 
   //Active Zone 
   execute("IPS_Zone_" + ...) 
   private.zone_counter1 = private.zone_counter1 + 2 
   private.zone_counter2 = private.zone_counter2 + 3 
endfor

I have groups of channels with the same name, just differentiated by an index, like:

IPS_Zone_1_Error

IPS_Zone_2_Error

IPS_Zone_3_Error

etc...

Up to 12

So, I tried to be efficient and use a for loop, instead of typing the 12 AddValue instructions.

Would this slow down the sequence significantly?

[AzeoTech 0]

Yes, most definitely. While DAQFactory doesn't compile down to assembly code, it does actually parse and compile script into psuedo code. That is because parsing a line of script is actually quite slow. The system has to read through character by character and figure out what you have written. With execute() we don't know what the string will have when the script is compiled, which is of course the purpose, but it means that DAQFactory has to compile the line of script before it can run it, and it has to do it every time execute() runs. This is slow. For many applications, its not an issue because you are only doing 1 or 2 of them, and the loops are long, but in yours, it is probably the source of the problem.

Instead, I would consider using the Channel.AddValue() function. You can see it in use in the serial/ethernet guide, though in a slightly different way. It is a generic function that does not use the channel name, but instead uses the device type, device number, i/o type and channel number to specify which channel it should go into. You could then make all your blocks of channels have consecutive channel numbers, and simply increment the channel parameter of this function. The function is described in the help under Channels and Conversions -> Channel List functions.

[Sanchez 0]

Ok, the Channel.AddValue Function looks interesting, I am going to try it, although it's going to take some time.

Another question that I have is: Does having too many channels slow down DAQFactory? I ask because our application is modular, meaning that it can control systems of several sizes, so I have declared Channels to cover all sizes. However most of the times, the systems are small, so there are several channels that are not necessary. I wonder if having idle channels reduces performance.

Thanks

[AzeoTech 0]

Not if they have a timing of 0 and you aren't stuffing any data into them with AddValue(). Channels just sitting there do nothing except take up a little memory. If you want to save memory, you may consider setting the history length of all unused channels to 1 by default. Note that if you change it dynamically, the actually history won't change until a single value is read. So, if you want to do it dynamically after the app starts, you should set the history to 1, then use AddValue() to stuff some random value into the channel and reset the history. Doing ClearHistory() does not actually clear the memory as DAQFactory preallocates the history.

[Sanchez 0]

I have tested the Channel.AddValue() function to read my 700+ channels all at once and it works very well, is not lightening fast but is more efficient, that's the way to go.

[Andreschrosa 0]

I'm having a similar issue with my application:

I'm creating the channels dynamically using execute statements, but once created they just begin reading modbus TRU read input floats.

In my testings I'm using two equipments with about 30 values each, netting 60 channels being read.

The problem I'm facing is it sometimes gives this message:

"C1038 Timing lag, data acquisition stalled. Resetting. Timing: 1.000, Offset: 0.000"

As you can see, I'm using timing 1.00 and offset 0 in every channel. I'm considering setting offset to something other than zero, but I think it wont help much when there are like 600 channels (which is the approximate network size for which this application is being designed) and 300 or so have an offset of 1 or 2 second. Also, the application user will want to see data read and logged every second.

We have other applications developed to read those values from the same equipments that can get the readings every 200 ms, so I think the hardware is not slow in sending data.

I've already disabled broadcast in the document settings and in each channel individually when they are created. Each channel also has history set to 3600 and persistence set to 0.

What could be done to get a better performance of the readings, can this be caused by the timing interval too short or history setting?

[AzeoTech 0]

Well, to start, create a single channel on one device, then look in the monitor window with the Display Time of Tx/Rx checked and see what the difference is between the Tx and Rx. This is the round trip time. Next, are your channels consecutive so DAQFactory can use fewer queries? Are the two devices on the same serial port? If not, then you can use an offset on one device to allow DF to query both devices simultaneously.

[Andreschrosa 0]

Here is a part of the comm monitor with time for all channels I'm testing:

Tx (10:02:14.397): \x01\x04\x00\x1B\x00\x02\x01\xCC
Rx (10:02:14.431): \x01\x04\x04G\xB8\x87\x81\xCC\x85
Tx (10:02:14.432): \x01\x04\x00\x1C\x00\x02\xB0\x0D
Rx (10:02:14.464): \x01\x04\x04D\xE3q-\xFA\xCF
Tx (10:02:14.466): \x01\x04\x00"\x00\x02\xD1\xC1
Rx (10:02:14.535): \x01\x04\x04@"\xFD/O\x02
Tx (10:02:14.535): \x01\x04\x00#\x00\x02\x80\x01
Rx (10:02:14.573): \x01\x04\x04\x00\x00\x00\x00\xFB\x84
Tx (10:02:14.573): \x01\x04\x00$\x00\x021\xC0
Rx (10:02:14.606): \x01\x04\x04B\xE8"\xE9\xB6\xE6
Tx (10:02:14.607): \x01\x04\x00%\x00\x02`\x00
Rx (10:02:14.640): \x01\x04\x04\x00\x00\x00\x00\xFB\x84
Tx (10:02:14.641): \x01\x04\x00&\x00\x02\x90\x00
Rx (10:02:14.674): \x01\x04\x04\x00\x00\x00\x00\xFB\x84
Tx (10:02:14.675): \x01\x04\x00'\x00\x02\xC1\xC0
Rx (10:02:14.706): \x01\x04\x04\x00\x00\x00\x00\xFB\x84

Here is the tx/rx for one channel:

Tx (10:08:35.000): \x01\x04\x00\x00\x00\x02q\xCB
Rx (10:08:35.033): \x01\x04\x04C\x03\x17\xCD\xD0e
Tx (10:08:36.000): \x01\x04\x00\x00\x00\x02q\xCB
Rx (10:08:36.033): \x01\x04\x04C\x03\x1D\xC3W\x01
Tx (10:08:37.000): \x01\x04\x00\x00\x00\x02q\xCB
Rx (10:08:37.033): \x01\x04\x04C\x03\x1D\xC3W\x01
Tx (10:08:38.000): \x01\x04\x00\x00\x00\x02q\xCB
Rx (10:08:38.041): \x01\x04\x04C\x03!~\x86p
Tx (10:08:39.000): \x01\x04\x00\x00\x00\x02q\xCB
Rx (10:08:39.032): \x01\x04\x04C\x03$\xCA\x85W

I'm setting an offset between devices as you suggested, my devices are set up on TCP/IP connections, each device on an IP address.

The channels are all in the same device for this particular test, some cant be sequential, some can. The channels are the values checked by the user to be read from the equipment, so sequence can not aways be achieved.

[AzeoTech 0]

Well, it looks like it takes between 30 and 40 ms to poll the channel. Multiplied by 30, thats 900 to 1200 ms. But since its never consistantly 30ms, we are definitely talking more than 1 second to poll all the channels if there are no blocks. I'd need to see you channel table to see if there actually are blocks. The snippet of traffic you sent is all single requests.

[Andreschrosa 0]

I exported the channel table from this particular test and it's attached. Please take a look as I might be configuring the channels inefficiently since it's my first project using DAQFactory.

Maybe if I set timing to 1s, that would work better. Since I'm storing those in the database, I must figure the best way to configure DAQFActory so it wont generate timing lags and stops recording.

To make the system scaleable for more equipments in the network, ie. more channels, it should make a little calculation to determine the timing, using 1s offset between devices.

Thanks for the help!

channels_table.txt

[AzeoTech 0]

Isn't your timing already 1s?

Are you sure the monitor window you sent is for this set of channels? Can you resend, making sure "display all ascii chars as codes" is checked. You should only need 4 queries to read all these channels since you have two blocks of channels on each device.

[Andreschrosa 0]

Tx (08:45:22.497): 010400020002\20811
Rx (08:45:22.531): 0104046708\163\1758778
Tx (08:45:22.531): 010400030002\129\203
Rx (08:45:22.581): 01040467100836\201\217
Tx (08:45:22.581): 0104000400024810
Rx (08:45:22.613): 01040462\15093\141\238\181
Tx (08:45:22.613): 01040005000297\202
Rx (08:45:22.647): 01040462\146\177679832
Tx (08:45:22.647): 010400060002\145\202
Rx (08:45:22.681): 01040462\150\183\21697\234
Tx (08:45:22.682): 010400070002\19210
Rx (08:45:22.733): 01040462\153\175\1432623
Tx (08:45:22.733): 010400080002\24009
Rx (08:45:22.767): 01040460\168\20213\22581
Tx (08:45:22.768): 010400090002\161\201
Rx (08:45:22.801): 0104046032\123\115\14911
Tx (08:45:22.802): 01040010000281\201
Rx (08:45:22.835): 01040459\113\229\196\237\184
Tx (08:45:22.835): 0104001100020009
Rx (08:45:22.867): 010404616600\13123\157
Tx (08:45:22.867): 010400120002\177\200
Rx (08:45:22.901): 010404643114\195\155\179
Tx (08:45:22.901): 010400130002\22408
Rx (08:45:22.935): 010404624628983176
Tx (08:45:22.936): 0104001400021608
Rx (08:45:22.971): 010404\18966\124\1729565
Tx (08:45:22.972): 01040015000265\200
Rx (08:45:23.005): 0104046403\251\157\156\221
Tx (08:45:23.005): 010400160002\11214
Rx (08:45:23.037): 01040466\240\221\13054\254
Tx (08:45:23.037): 01040017000233\206
Rx (08:45:23.110): 010404662756\19812\105
Tx (08:45:23.110): 010400180002\209\206
Rx (08:45:23.143): 0104046632\174\223\211\206
Tx (08:45:23.144): 010400190002\12814
Rx (08:45:23.177): 0104046637\21196\16347
Tx (08:45:23.178): 01040020000249\207
Rx (08:45:23.211): 01040466\24118\193\11563
Tx (08:45:23.211): 0104002100029615
Rx (08:45:23.243): 01040466275768\141\152
Tx (08:45:23.243): 010400220002\14415
Rx (08:45:23.277): 0104046632\199\237\12475
Tx (08:45:23.277): 010400230002\193\207
Rx (08:45:23.311): 01040466383682\148\202
Tx (08:45:23.312): 010400240002\241\204
Rx (08:45:23.345): 01040466\11221\160\224\207
Tx (08:45:23.346): 010400250002\16012
Rx (08:45:23.423): 01040468\235\2190505\179
Tx (08:45:23.424): 0104002600028012
Rx (08:45:23.457): 010404\189\1475473\249\147
Tx (08:45:23.457): 01040027000201\204
Rx (08:45:23.491): 01040471\190\229\247\13202
Tx (08:45:23.491): 010400280002\17613
Rx (08:45:23.525): 01040468\235\2190505\179
Tx (08:45:23.525): 010400340002\209\193
Rx (08:45:23.559): 0104046440\131\11214\152
Tx (08:45:23.560): 010400350002\12801
Rx (08:45:23.593): 01040400000000\251\132
Tx (08:45:23.594): 01040036000249\192
Rx (08:45:23.627): 01040466\240\233\212\16000
Tx (08:45:23.627): 0104003700029600
Rx (08:45:23.661): 01040400000000\251\132
Tx (08:45:23.661): 010400380002\14400
Rx (08:45:23.695): 01040400000000\251\132
Tx (08:45:23.695): 010400390002\193\192
Rx (08:45:23.729): 01040400000000\251\132
Tx (08:45:23.729): 020400000002\113\248

Just back from vacation.

Attached is a new table I exported just now, and above is a part of the comm monitor window.

It keeps giving the timing lag and time out message, still cant figure what I'm doing wrong.

channels_table.txt

[Andreschrosa 0]

I also tried reducing screen resolution to 800x600 to reduce memory load. It still lags, but much less than before.

I doubt, but could this be a lack of system memory? I'm running this on my own working machine, which is a pentium core 2 duo 2.4MHz with 1 G RAM memory.

Also, should I remove all other devices from device configuration other than my devices? I think there is some configuration problem I'm not yet aware of... I tested it just now using a single device connected via serial cable and it still lags like I described above.

[AzeoTech 0]

What protocol are you using? It looks like Modbus ASCII? I'm not completely sure our ASCII driver optimizes reads. Why aren't you using RTU like your previous posts?

[Andreschrosa 0]

I'm using modbus RTU over ethernet. I also disabled broadcast in document settings and in ethernet configuration timeout is set to 3500.

[AzeoTech 0]

The long, last monitor capture you posted is not RTU (unless you changed it). Most of the characters are ASCII and it looks like Modbus ASCII. This is especially weird since all your other monitor windows are showing RTU.

[Andreschrosa 0]

I can assure you, I didn't change the protocol, just in case I checked then on device configuration. The only difference is I checked "display all chars as ASCII codes" like you asked for in the last post before that.

We where using a virtual serial port software for our older softwares, so I thought maybe that was causing the lags, but no good, disabling then also didn't stop the lagging.

Are there any know issues with any windows application or service causing something like this?

You think there is a possibility it's something in the hardware's modbus configuration that may cause that?

[AzeoTech 0]

Weird. For some reason somewhere along the line the is getting stripped out. For example you are saying the monitor is:

020400000002\113\248

when in fact it should be:

020400000002\113\248

It didn't strip it when you posted the previous because you had the "display all codes in hex" checked and so instead of it was \x. I'm not really sure where it got stripped. Either in the clipboard or this forum software perhaps. Anyhow, it does not matter to the question at hand.

It appears that DAQFactory is not optimizing the reads and is in fact using a separate query for each channel. I think the reason is that you have either selected the wrong data type, or you selected a 32 bit data type and forgot that the tag numbers must jump by 2 (because it takes two modbus registers to create a 32 bit value). I see a query like this:

010400020002\20811

which says read 2 holding registers starting at address 2, which means its going to read holding registers #2 and #3.

But then the next query is:

010400030002\129\203

which says read 2 holding registers starting at address 3, which means its going to re-read holding register #3 and also read #4.

This is why DF can't optimize it. So, you need to figure out exactly what data type your data is. If your data is a 16 bit word value (U16 or S16), then you need to change all the I/O types to match. If your data is a 32 bit value, either integer or float, then you need to increment your channel #'s by two. If you have a 32 bit I/O type selected, you cannot increment your channel #'s by 1. Well, you can, but DF won't optimize and the results won't make any sense.

If your device for some reason returns 32 bit values, but numbers their modbus addresses by 1, then you need to go back to them and tell them that their modbus implementation is incorrect. A modbus address is, by definition, a 16 bit value. 32 bit values don't exist in modbus and the only way to get them is to combine two consecutive 16 bit values. This is why it must be numbered by 2.

[Andreschrosa 0]

That was it, the data type was right, but I was using the addresses incremented by 1, incrementing by 2 it worked right.

However, there still is some timeout, probably normal considering ethernet connection... maybe this is not a problem.

I want the application to store data at the MySQL database at a rate of one read every second or one read every 5 seconds, it dosent need to be read at every second. Maybe if I configure the logging set right, I can say, have the channels read every second, and record every five seconds. I believe it's possible to do, yes?

[Andreschrosa 0]

Another thing that must be helping to give those timeouts, our equipments's modbus has a maximum frame size of 120 bytes, which will amount to about 30 floats. I'm requesting 35 floats from each equipment. Think I'll have to script it somehow, probably set the timing of every channel to zero, and have a sequence execute a read on every channel once per second, divided by the number of bytes our protocol supports.

[AzeoTech 0]

Logging sets have to run at the same speed as the data coming in. That's the whole idea behind them. If you want to log at 1/5 the acquisition speed, you'll have to use an export set (with [0] at the end of all the expressions so you get the most recent point) and trigger it from a sequence running every 5 seconds.

As to your 120 byte limit, you can do grouping in the channel list by changing the offset. So, if channels 0-29 are at timing 1, offset 0, and you want to put channels 30-59 in a different modbus query block, just put them at timing 1 offset 0.01. You can repeat this as much as you need to create different blocks, or just stagger them between offset 0 and offset 0.01.

[Andreschrosa 0]

Tried setting the offset of the last 10 values to 0.01, but then it gives me port locked. You have any idea why?

Might give a try to the read channel idea, but it might require a lot more of work.

[AzeoTech 0]

What is your timeout set to? Are you getting timeout errors? If the stuff at offset 0 takes longer than the timeout, than you'll get a port locked from the stuff at offset 0.01 because it can never get the port. If you get a timeout error, then the same sort of thing is happening.