Count function for valid data

[ekanderson]

function ala ooo calc's count() that would count VALID (non-Nan()) data for a channel or variable array.

would be equivilent to numrows() - numnrt og NaN entries

ek

[AzeoTech]

You can actually do this now:

sum(x != nan())

you can sum any criteria this way. Its boolean math.

[ekanderson]

equivilent to numrows() - numnrt og NaN entries

should have been

equivilent to numrows() - number of NaN entries

anyway

At first glance

sum(x !=NaN()) would appear to take the total of x[] by adding all x values which are not equal to NaN() which is what sum() now does (I think, and hope).

The ooo calc's COUNT does

counter =0

for y =0 to numrows(x)

if (x[y] = a valid numeric value // i.e., not a string, not an empty cell, not a NaN()

counter +=1

endif

next y

return(counter)

Don't see how a sum() would give a COUNT of all x's that are not equal to NaN(), but I'll try it.

thanks

ek

[AzeoTech]

Its boolean math:

x is an array of values

x != nan() is an array of 1's and 0's, with a 1 every where x does not equal nan() and a 0 every where it does equal nan()

the sum of x != nan() is a sum of the 1's and 0's, so gives you a count of the number of elements in the array of x that don't equal nan(). You can do this sort of thing to count any sort elements.

The sum of the values of x that don't equal nan() is different:

sum(x * (x != nan())

once again, x != nan() is an array of 1's and 0's. Multiply that by x, and you get an array with all the nan()'s converted to 0. Then we take the sum of that.

Understanding this will give you a lot of power to manipulate arrays without using loops, which will run incredibly fast and makes for simpler code.

[ekanderson]

aahhhaa ! neat

ek

[ekanderson]

Some sort of unified results with values that are NaN().

Wrote a sequence

//  Calculate dew point from temp & RH	Last Mod  11 Aug 09
// from Chris Arndt e-mail  10 Aug 09  PHP code
//	and wikipedia Dew_point entry printed out 09 Aug 09
//
// 11 Aug 09 eka	Convert to DF code and got it to give ~ matching values for
//			wunderground values for a range of stations Fresno, Paso, M.B., SLO
//	at first glance, fails dimensional analysis.  Perhaps some of constants have dimensions?????
//		Probably a barometric factor would clean up dimensional problem
//	wikipedia/NOAA has
//		actual vapor press =saturated v.p. -barom (T -Tw)*0.00066[1 +(0.00115Tw)]
//	pass temperature in F, and RH as percent
function dewpoint_calc (Temp_f, RHumid)
	// Calculate the Dew point

// Get the vapor pressure. Needs temp in C

	// Convert temp in F to C.
Private Temp_c = (Temp_f - 32)* 0.5555555556 	// save a division	5/9)
//RHumid =Rhumid *0.01       // convert to fraction
//? temp_c

//Private vapor_pressure = Rhumid *0.01 *6.112 * exp((17.62 * Temp_c) / (Temp_c + 243.12))

		// constants from wikipedia/NOAA
Private vapor_pressure = Rhumid * 0.01 * 6.112 * exp((17.67 * Temp_c) / (Temp_c + 243.5))
	// debug
// ? vapor_pressure

//Calc the dewpoint in C

//	Original	Way off  possibly related to DF or PHP log function or ?????
//		tried it with   ln instead of log, still ng
//Private dewpoint = (243.12 * log(vapor_pressure) - 440) / (19.43 - log(vapor_pressure))

	// this adapted from Wikipedia NOAA formulae printed out 09 Aug 09
Private dewpoint = (243.5 * ln(vapor_pressure/6.112)) / (17.67 - ln(vapor_pressure/6.112))

// Convert it to F
//? dewpoint

dewpoint = (dewpoint * 1.8) + 32

//dewpoint = round ($dewpoint, 0)	// no need to round here

//? "dp= " +doubleToStr(dewpoint))
//? dewpoint

return (dewpoint)

When passed dewpoint_calc(NaN(), NaN(), 1) in command window has returned NaN good and predictable

and '-1.#IND'.

NOTE: The RdtoDig routine (which uses format(), and filters the results was posted somewhere on this forum.

When passed channel data of RdToDig(dewpoint_calc(temp_f[], Rhum[]), 1) that contained matching NaN() values AND real data.

Yesterday it returned -1.#IND for pairs that were NaN() indeterminate?????

Today with same data -1.$ ditto god only knows

I've also seen -1.?J, etc. with other code that was processing NaN() values the ? cannot remember what it was.

If the return values are just tested for '-1.', then one has no idea if it is potentially valid numeric data with out testing for all possilbe combinations of the next character. A search of your forum for '-1.#', no results.

If it was always '--1.' or some other totally unique value +whatever, that had no relationship to any possible real results, then the results could be easily filtered.

[AzeoTech]

You really should check for NaN() in the beginning of the function and return a predefined value. The reason you get varying results is because you try and do math with NaN() and the OS is returning different things depending on where in an overflow occurred in your calcs. I don't recommend trying to figure it out, but rather just put a couple if's at the beginning to check for valid input parameters. This is good programming practice anyway.

[ekanderson]

I've tried ifs, and they only seem to work when single values are passed and not work when arrays are passed.

Any hints on where I'm going wrong.

ek

[AzeoTech]

I forgot. Did we ever try using the filter() function?

x = filter(x, x != nan())

[ekanderson]

no filter() yet, but from DF help section, don't know if it will do what I want, i.e., it return a "sparse" array not aligned with all my other data.

The usual situation is all channels have the same missing data (tagged with NaN) for times when the system is down for hardware/software maintenance. The gotcha, is that several channels can have values of NaN() if the sensor's "excitation" voltage is out of spec, or if a solar irradiance sensor goes 0 during daylight hours, etc.

Don't know what affect using filter() would have for export files that output channel_x[0,9] using filter or others that use channel_y[time1, time2], but I can mess with it.

Thanks again.

ek

[ekanderson]

Got filter() to work with mean(), max(), min(). Only problem I had was when filter returned a null array, but by doing "x =filter()" then checking if x is empty, AOK.

Thanks.

ek

[ekanderson]

Add to "Help" AND "forum" some key words that will direct people to "System.entryDialog()".

when searched for "* keyboard input" where * is user, operator, etc.

For about a year I have searched for those terms with no luck. Stumbeled across "System.entryDialog()" somehow just yesterday.

ek