Specifying variable's precision programatically


okinelin

Recommended Posts

Posted

Hello there,

I am using direct file access to log calculated variables. I want to control the number of decimal points for the log variables.

I have tried format("%.2f", CalculatedVariable) but that doesn't seem to work.

Can someone please help out?

Thanks

Posted

Below is the variable;

Scaled_Pressure_1 = Pressure_1[0]*Slope_Pressure_1 + Intersect_Press_1

In order to write to a file I used the following code

DoubleToStr(format("%.0f", Scaled_Pressure_1))

and that did not work. Is there anything wrong in the code?

Posted

No, provided all those variables actually have values and are numeric. There is certainly nothing wrong with this line:

DoubleToStr(format("%.0f", Scaled_Pressure_1))

Though technically you don't need the doubleToStr() since the format() function makes a string.

You might add a:

? Scaled_Pressure_1

after the calc to see what the result of the calc is, or use the debugger.

Posted

Thanks Guru.

It's working. I didn't realize the format() returns a string. I removed the DoublToStr() and everything is fine.

I appreciate your support

Archived

This topic is now archived and is closed to further replies.