String And Number To/from String Functions

[EOlsen]

In some ways the scripting language has an impressive number of string functions, but one string function it is missing is the ability to create a string of desired length filled with a repeated value.

 

The function would be something like String(n, 32) or String(n, " "). The first parameter is the length of the returned string. The second parameter is the character to fill the string with. If it is a number, it is the ASCII value of a character. Or it can be a single character represented as a string.

 

 

Sometimes writing data to disk in binary format is an advantage. It is available directly in DAQFactory only when writing to logging files. As mentioned in help, file I/O gives more flexibility than logging. Plus without the ability to easily read binary formatted files, there isn't much you can do with binary logged data from within DAQFactory.

 

The language has a long list of to/from functions for converting to/from numbers to byte arrays. Then there are also a couple of functions to convert byte arrays to/from strings which can be written to disk. Fairly complicated process. I'd like to see functions for easily writing and reading numbers to disk in binary format. At the very least a set of functions that converted from a number in memory to a string and the reverse in a single function call. Maybe something like BinaryToString(type, number) where type identifies the type of number (integer, long, float, double) and number is the value to convert. Then the reverse for reading would be StringToBinary(type, string) or BinaryFromString(type, string).

 

It would be really nice to be able to read logged data and display it on a graph and not be restricted to just history data. For writing and reading logging style binary files where the records contain multiple columns repeated throughout the file it would be nice to have something like the export setup or printf setup where you identify the type of each element (column), call a function with the column format and the data.

 

At the very least there should be a data retrieval function for reading data back from an exported log file into a form that can be used to display graphs.

 

 

[AzeoTech]

There are lots of parts here.  

 

1) creating a string of repeated characters.  You can do that with chra() and fill().  For example, to create a string with the letter "a" repeated 32 times (and assuming you didn't know the ascii code for "a") you'd do:

 

chra(fill(asc("a"),32))

 

asc() converts a string character to its ascii code.  Fill() creates an array with the specified value repeated n number of times.  chra() takes an array and creates a string of characters where each element in the array is one character.  asca() does the reverse.  Chr() does the reverse of asc() and applies to just one character.

 

2) You can read binary formatted files using file.readInter().  I don't know what you mean by converting a number in memory to a string.  You can convert a number to a string containing the bytes using the appropriate from. function combined with chra().  For example, here's a round trip to string and back to the original number:

 

to.Double(asca(chra(from.Double(3.4234))))

 

The only problem is that this only works on one number at a time.  You can't take an array of numbers and processing them in one call.  But if you are logging, you probably don't need to do that.

 

As for reading it back in, readInter() will do that.  You can then use the necessary to. function to convert the 2d array returned by readInter() right back into the desired numbers.

 

3) read logged data: you can of course use persist to extend the history.  As for reading back actual logged data, you can do that using a little script, either as described above if you really want binary, using file.ReadDelim() for csv files, or using SQL queries and db.queryToClass() if you are using an ODBC database (by far the easiest and most powerful way).  Then just plot the variables you load back.

 

BTW: I generally recommend against using binary data formats.  They are very non-portable, and very susceptible to damage.  Lose a sector and you pretty much lose the whole file.  Lose a sector in a csv file, and you really just lose that part.  The non-portable part is more important.  You have to store or remember somewhere the actual format of the data, and if you lose it, then you just have a bunch of random numbers.  CSV allows for headers right in the file, and of course databases have field names already.  The only advantage of binary is speed and compactness, and that is less and less of an issue as processor power is way ahead of disk access, and disk space is dirt cheap.  If speed really is an issue, a database is going to be almost as fast to read from.

 

There are some exceptions where you can really eek out speed in binary files.  That's certainly what we do in the Persist files.  But unless you are trying to create a high performance server application, its rarely worth the trouble, or the risks just mentioned.  I know its hard to let go of optimizations that 10-20 years ago were really required for a reasonable application.

[EOlsen]

One big advantage of binary files is the ease at which you can jump to a particular record for reading and writing. This is especially important if it is a file in which any of the data will need to be changed. In binary mode it is easy to calculate the file location and either read or write your fixed length record at that point. With CSV you typically have to read the entire file into memory or least up to the portion you want to use because records are of an uneven length. They are also longer so you have to read even more data. Writing mid file is even more involved. Separate header files solve the problem of general data file format specs for binary files. For other files the data format spec is built right into the program - you never need or want to see the data outside the context of the program like channels in DAQFactory for example. Without headers in a CSV file you still just have a bunch of numbers, they take up several times the amount of disk space, and use up processor cycles converting back and forth between numbers and strings. Lots of extra steps with a net loss instead of a gain in terms of just about everything - program complexity, storage space, processing time.

 

Yes, I forgot about ReadInter() function. Looks like it does get a step closer. Wasn't clear how the interval parameter worked if you are reading fixed length records. If you read all bytes in a record, would the interval be set to zero or to the size of records, or does it even matter. Is there an example somewhere of it in use? Especially nice would be an example of fetching data from a binary logged data file to use with graphs on a page.

 

Maybe I am old school. I believe good code is easily readable (good formatting and style), easy to understand (to follow the program's logic, liberal use of comments helps), and makes good use of computing resources. It seems today instead of attempting to write efficient code, the solution is to just to get a new computer with more memory, faster processor, and bigger drive. But just think how much more your existing computer could do and how much faster if there was more emphasis on code that made excellent use of available computing resources. And often efficient use of computing resources is a result of better program organization which in turn is easier to follow.

[AzeoTech]

True, binary files are easier to search and change, but then so are SQL databases, and databases don't have the pitfalls I mentioned, plus are MUCH simpler to query.  They've done all the work for you.

 

You would not use readInter() to read a record.  readInter() is designed to read a field across all records.  To read a record, just use read(), specifying the size of the record, after jumping to the right place in the file.

 

I am old school too, but many old school techniques actually don't result in cleaner code, or in this case, cleaner logging.