Import loggging file

[Sid]

Hello

I used your sample for create and read logging file and it seems to work

The logging part and the change logging filename on daybreak works OK! The file open part works OK and now I would like to ask something!

What would be the best way to import the data back to DAQ? I would like to show it on a graph!

It's like a long term history. The client would just choose for which day (izbrani_cas()) he wants to see the trend and that's it! That way I could turn off persist completly! The client chooses a day (y/m/d) and DAQ finds the filename.. This works!

........
........
........
Global izbrani_cas
Global string strLOGfile

strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())


Private FileHandle = File.Open(strLOGfile,1,0,0,1)
Private string strIn

while (1)
// read a line
strIn = File.Read(FileHandle)
// This checks for an empty string
if(strIn == "")
// Breaks out if empty string
break
endif

// This code Parses the String and sets the output channel
x_out = StrToDouble(Parse(strIn,1,";"))	  //value
TimeH = StrToDouble(Parse(strIn,0,";"))	  //time

// Reads 1 second at a time
delay (0.001)
endwhile

// Closes the file
File.Close(FileHandle)

But I can't get it to store the value with time, so I could show the result (the readout from the file) on a graph. I have two variables, would it be better if I were to store to an array (but how....I tried and failed) with time? The logging uses DAQ time not Excel time!

If I try to trend a graph with the x-axis = TimeH and y-axis = x_out it does something but with DAQ time (I scaled the bottom axis from izbrani_cas to izbrani_cas+500)! What would be the best way?

And another thing.. DAQ saves the logging files from the beginning to the end. When I import the file with the upper function the file is imported from the end to the beginning! That means that the first value in the variable is actually the last logged value in the file! Could that be a problem with trending? I changed the code to fix this, is it OK?

Global i
i=60			 //a file with 60 records

while (1)
// read a line
strIn = File.Read(FileHandle)
// This checks for an empty string

if(strIn == "")
// Breaks out if empty string
break
else
i--	  
endif

// This code Parses the String and sets the output channel
x_out[i] = StrToDouble(Parse(strIn,1,";"))	  //value
TimeH[i] = StrToDouble(Parse(strIn,0,";"))	  //time



// Reads 1 second at a time
delay (0.5)
endwhile

// Closes the file
File.Close(FileHandle)

Thank you for your reply, you rule!

[AzeoTech]

Graphs prefer time data associated with the actual data, so where you have:

x_out = StrToDouble(Parse(strIn,1,";")) //value

TimeH = StrToDouble(Parse(strIn,0,";")) //time

you should do:

x_out = StrToDouble(Parse(strIn,1,";")) //value

x_out.time = StrToDouble(Parse(strIn,0,";")) //time

Also, graphs require data to be sorted properly from newest to oldest data point, which, as you pointed out, is the reverse of the logging set. Fortunately, we have the sorttime() function:

x_out = sorttime(x_out)

which will take care of that for you.

[Sid]

Hi

Thanks for the previous reply Guru!

But it seems that when I use your code my timeline changes..

I checked it in the Watch monitor and when I start the sequence and try to read the desired file, the x_out.time gets associated with the system time (each point from the history/logging file gets a new time value from systime when being read from file)...

What can I do?

[Sid]

A bit more..

When I use the code:

// This code Parses the String and sets the output channel
TimeH = StrToDouble(Parse(strIn,0,";")) //time

x_out.Time = StrToDouble(Parse(strIn,0,";")) //time

x_out = StrToDouble(Parse(strIn,1,";")) //value

The time is written in the tags as in the picture.. The same happens if I write use only TimeH or only x-out.Time in the upper code. TimeH gets the right timeline and x_out.time gets SysTime..

[AzeoTech]

Did you do it in the exact order I did? I.e. x_out = first then x_out.time = ? .time must always be second, as myvariable = overwrites the entire variable, both time and value, where myvariable.time only overwrites the time portion (unless there is no value)

[Sid]

Hi

The order makes no difference in my case. It's the same as before.. My new code is as follows:

// Opens the file sets the variable FileHandle

x_out.ClearHistory(1)	  //clear previous

Global string strLOGfile

strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())

Private FileHandle = File.Open(strLOGfile,1,0,0,1)
Private string strIn

while (1)

// read a line
strIn = File.Read(FileHandle)
// This checks for an empty string

if(strIn == "")
// Breaks out if empty string
break
endif

x_out = StrToDouble(Parse(strIn,1,";")) //value

x_out.Time = StrToDouble(Parse(strIn,0,";")) //time

// Reads 0.1 second at a time
delay (0.1)



endwhile

// Closes the file
File.Close(FileHandle)


x_out = sorttime(x_out)

This code does not save the correct time with the readout value! Could the problem be that x-out channel is DF test channel set to D to A?

I also tried with a virtual channel (w/ 3600 history points) and strange things happened

// Opens the file sets the variable FileHandle

V.x_out.ClearHistory(1)	  //clear previous

Global string strLOGfile

strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())


Private FileHandle = File.Open(strLOGfile,1,0,0,1)
Private string strIn

while (1)

// read a line
strIn = File.Read(FileHandle)
// This checks for an empty string

if(strIn == "")
// Breaks out if empty string
break
endif

V.x_out = StrToDouble(Parse(strIn,1,";")) //value

V.x_out.Time = StrToDouble(Parse(strIn,0,";")) //time

// Reads 0.1 second at a time
delay (0.1)
endwhile

// Closes the file
File.Close(FileHandle)

V.x_out = sorttime(V.x_out)

It basically saved the time and the value in the same channel (didn't separate them ..see attached JGP)! But this time the time was the correct one!! What do you think?

In the attachment is the logging file if you want to try..just change the strLOGfile to the MyData_09_12_14.csv and change the path..

[AzeoTech]

That is absolutely the problem. MyChannel.Time does not work, period, and mychannel = sorttime(mychannel) is going to act weird too. I thought it was a variable, not a channel. When you do mychannel = x it calls the driver. For the test device D to A, it simply echos back the value set, but it does change the time to the current time as a real device would. So a few things:

1) why use a channel? Why not a variable. That will work properly

2) also, instead of using a loop, just use the readdelim() function to read and parse the delimited file in one go.

3) if you need a channel for some reason, you can use the AddValue() function instead of = to put data into the channel with your preset timestamp. You'd have to set the timestamp into a variable with the data point before calling addvalue(). You'd also need to use readdelim() to read the whole file and sorttime() before doing addvalue(). You can use addvalue() with an array, but again, if you have it in a nice variable array, why the channel?

[Sid]

Hello

After a bit of trying and testing I finally got it to work!

this is the code:

//declarations
Global string strLOGfile
Private x1
Private string strIn

//get filename
strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())

// Opens the file sets the variable FileHandle
Private FileHandle = File.Open(strLOGfile,1,0,0,1)


while (1)
// read a line
strIn = File.Read(FileHandle)
// This checks for an empty string
if(strIn == "")
// Breaks out if empty string
break
endif

//save value and time from string to the variable
x1 = StrToDouble(Parse(Private.strIn,1,";"))
x1.Time = StrToDouble(Parse(Private.strIn,0,";"))

//add to channel (and channel history)
V.testni.AddValue(x1)

// Reads 0.001 second at a time
delay (0.001)

endwhile

// Closes the file
File.Close(FileHandle)

OK, this code with the rest of the program works fine! I can save a file each day for the current logging and after that open it (or even while it is writing into it!)! Thank you for your previous help Guru!

Sorry, but I have more questions...

1.

But this code is slow (reads a line every 0.001 seconds..)...what will happen if I have a file with about 60000 points? This would take 60000 * 0.001 seconds = 60s. What if I delete the delay? Will the CPU overload, or will it just freeze for a couple of seconds?

What would be the best way to read such a file in your opinion?

2. What about the ReadDelim() function, is it any faster? I've tested it and I think it is.. Could you please tell more about it (not much written about it on this forum and in the manual)?

I've used it with this code, but can't seem to get the Time included with the value array (x1)..

Global x1
Global string strLOGfile

strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())
Private FileHandle = File.Open(strLOGfile,1,0,0,1)

x1 = file.ReadDelim(FileHandle,1,";",chr(10),0,0) //number of lines= 0 za vse vrstice

x1.time = file.ReadDelim(FileHandle,0,";",chr(10),0,0) 


delay (0.001)
V.testni.AddValue(x1)

// Closes the file
File.Close(FileHandle)

The correct value gets stored in the array, but the time portion is always 0.. Where did I go wrong? Can ReadDelim function read DF time and Excel time? I'm still using DF Time!

Thank you for your reply!

[Sid]

Hi, it's me again..

I managed to get the ReadDelim code working correctly!!!

Private x1
Global string strLOGfile
Private x2


strLOGfile = "c:\DAQFactory\LOG\MyData_" + FormatDateTime("%y_%m_%d.csv",izbrani_cas())
Private FileHandle = File.Open(strLOGfile,1,0,0,1)

x2 = file.ReadDelim(FileHandle,-1,";",chr(10),0,0) //number of lines= 0 for all lines in the file

x1 = x2[][1]
x1.time = x2[][0]

x1 = sorttime(x1) //arrange x1.time

V.testni.AddValue(x1) //add to virtual channel

// Closes the file
File.Close(FileHandle)

I'm still interested in the questions I posted before and I'd like to get your opinion about this code! Do you think it's better/faster than the code with the While loop and parse function?

And another question.. Using ReadDelim() function I always seem to get the latest value written as NaN and time as 0(...1970 etc.). That's probably because of the first line (header).. How can I omitt it from reading (don't read it with this function), because I'd really like to keep the first line in case the customer wants to reuse it in another program (like Excel)!

Thank you

[AzeoTech]

1) readdelim() is significantly faster than using a loop, probably 10,000 times if not more. You generally want to avoid doing loops in DAQFactory because its a semi-interpretted language. ReadDelim() and all the other functions in DF run in low level C, and so run about as fast as your computer could possibly perform the given task, so you should always take advantage of them when possible.

If you have to do a loop, remove the delay() and run the sequence at priority 0.

2) you can get rid of the first line simply by doing something like: x1 = x2[1,10000000][1] instead of x1=x2[][1]

[Sid]

Hi

I don't understand your anwser.. Where am I supposed to put the following line? In the first program with the RealDElim function (which didn't work) or the second (which works) and why?

2) you can get rid of the first line simply by doing something like: x1 = x2[1,10000000][1] instead of x1=x2[][1]

And again

And another question.. Using ReadDelim() function I always seem to get the latest value written as NaN and time as 0(...1970 etc.). That's probably because of the first line (header).. How can I omitt it from reading (don't read it with this function)...

Thank you

[AzeoTech]

In the second, working version. ReadDelim() reads the whole file, can't do anything about it. But you can exclude the data when you assign it to x1. So where you have:

x1 = x2[][1]

x1.time = x2[][0]

you want:

x1 = x2[1,1000000][1]

x1.time = x2[1,1000000][0]

Replace 1000000 with a number bigger than all the possible rows. I suppose you could also do:

x1 = x2[1,numrows(x2)][1]

x1.time = x2[1,numrows(x2)][0]

[Sid]

Hi,

works like a charm.. Thank you for all your help!

I'll be posting a follow up in the networking thread because I have new questions which don't fit in this thread!