Filled graphs with variable colour

[Gammatech]

Hello,

I have tried using the "stick" type plot with boolean math to produce either a value or NaN() for each colour but the resultant display is not really satisfactory unless there is space around each "stick". As my data is produced each second, to get a 5 minute scrolling graph requires the "sticks" to be set too thin for adequate colouration. If I use thicker "sticks" then there are often overlapping colours or sometimes random vertical spaces traversing along the graph. Area plotting seems to give some peculiar effects of one colour block joining to the last of that colour and overwriting the intervening colour. What I need is:

If the value is at the user adjustable target plus or minus a user adjustable band then flood fill with green.

If the value is higher than (target + band) or lower than (target-band) then flood fill with yellow.

If the value is less than the lower limit or greater than the upper limit then flood fill with red.

Is there any way of doing this?

In an answer some time ago you suggested that the graphing was going to be updated from the current third party code you currently use at a future release - when will this be done

[AzeoTech]

Yes there is, but it requires a little script and the Canvas component. The canvas component allows you to use lower level drawing commands to script up your own displays. I'd still probably use a graph as a background to draw the scaling and the grid, but then you can overlay a Canvas component to draw the actual graph the way you want it. Here's the code that will draw the "test" channel in green and red depending on its value:

private numpoints = 60
private rangemin = -2
private rangemax = 2
private totalrange = rangemax - rangemin
private rect = Position
private width = Position[1][0] - Position[0][0] 
private height = Position[1][1] - Position[0][1]
private barwidth = width / numpoints
for (private x = 0, x < numpoints, x++)
   if (test[x] < 0.5)
	  Draw.SetBrush(RGB(255,0,0))
   else
	  Draw.SetBrush(RGB(0,255,0))
   endif
   // right side of rect
   rect[1][0] = Position[1][0] - x * Barwidth 
   // left side
   rect[0][0] = rect[1][0] - Barwidth
   // top
   rect[0][1] = rect[1][1] - (test[x] - rangemin) / totalrange * height 
   // bottom is already set
   Draw.FillRect(dc,rect)
endfor

Its not terribly complicated. I used a bunch over variables at the top for clarity. The rect variable is a rectangle, which is stored as a 2x2 array. [0][0] is the x coord of the top left, [0][1] is the y coord, [1][0] is the x coord of the bottom right, and [1][1] is the y coord. Most of the math is determining the top of the box.

A little tie-in to the scaling of the graph and the rest of your app and it should be quite robust. You can of course add more colors. The only real problem with this is that the graph is not anti-aliased and will show steps, but that's only an issue if the numpoints variable is smaller than the actual width of the canvas control. For you, numpoints is going to be 300, so the aliasing shouldn't be bad.

[Gammatech]

Thanks for that. I did get some peculiar "shuffling" motion that may well have been aliasing but by experimenting with page update time got a good display with three colours in five bands.

The next problem comes because, as I want four scrolling graphs on screen at the same time, it is much better to have them scrolling vertically rather than horizontally. (so that I can take advantage of the aspect ratio of the screen)

The array addressing to change to vertical defeats me, perhaps I'm Dim but I can't make it scroll down the screen!

I have tried to put the time (Say in one minute intervals) scrolling down as well but using Draw.TextOut() in the For/EndFor loop seems to always put it at a fixed location or everywhere along the top of the graph so I must be getting the array addressing wrong here too.

How can the existing Graph component be set up for vertical scrolling?

[AzeoTech]

I've attached a sample showing all three graph types displayed here. I mention it now for those that just want to see it. There's one graph per page.

OK, the following code will do the same as the previous code, but draw a vertical graph (moving upwards), add labels to x and y, draws a box around the graph, and doesn't display an error when not enough data is available:

// number of data points across y axis (similar to time width, but in units of datapoints)
private numpoints = 60  
// x axis scaling:
private rangemin = -2
private rangemax = 2
// convenient calc:
private totalrange = rangemax - rangemin
// preset rect:
private rect = Position
// point variable for label
private p
// calc info on component location
private width = Position[1][0] - Position[0][0]
private height = Position[1][1] - Position[0][1]
// determine size of bars
private barwidth = height / numpoints
// how often to draw a label on y axis (in data points)
private labelinterval = 10
// offset from y axis
private labelwidth = 120
// how often to draw a label on x axis (in x axis units)
private xlabelinterval = 0.25
// vertical offset
private xlabelheight = 10
// horizontal offset to center text
private xlabeloffset = 15

// draw box around graph:
draw.SetPen(rgb(0,0,0))
draw.FrameRect(DC,Position)

for (private x = 0, x < numpoints, x++)
   if (isempty(test[x]))
	  break
   endif

   if (test[x] < 0)
	  Draw.SetBrush(RGB(255,0,0))
   else
	  Draw.SetBrush(RGB(0,255,0))
   endif
   // right
   rect[1][0] = rect[0][0] + (test[x] - rangemin) / totalrange * width
   // left is already set
   // top side of rect
   rect[0][1] = Position[1][1] - x * Barwidth
   // bottom side
   rect[1][1] = rect[0][1] - Barwidth
   Draw.FillRect(dc,rect)
   if (x % labelinterval == 5)
	  // draw x label:
	  p[0][0] = position[0][0] - labelwidth 
	  p[0][1] = rect[0][1]
	  draw.TextOut(dc,p,formatdatetime("%c",test.Time[x]))
   endif	  
endfor
// now x axis:
for (x = 0, x <= totalrange, x+= xlabelinterval)
   p[0][1] = position[1][1] + xlabelheight
   p[0][0] = position[0][0] + x/totalrange * width - xlabeloffset
   draw.TextOut(dc,p,format("%.2f",x + rangemin))
endfor

I tried to name all my constants at the top so you should be able to tweak it.

Now, you can make the graph a bit smoother if you use fillpolygon() instead of fillrect(), but its slightly more complicated. The other problem is that it doesn't split the color perfectly. You'd have to decide how you wanted that determined. To make this post readable, I'm trimming the script from the top and the bottom. See the attached sample for the full working code:

.... (init code)
// these coords remain the same:
rect[0][0] = position[0][0]
rect[1][0] = position[0][0]
for (private x = 1, x < numpoints, x++)
   if (isempty(test[x]))
	  break
   endif
   if (test[x] < 0)
	  Draw.SetBrush(RGB(255,0,0))
   else
	  Draw.SetBrush(RGB(0,255,0))
   endif
   // top side of rect
   rect[0][1] = Position[1][1] - x * Barwidth
   // bottom side
   rect[1][1] = rect[0][1] - Barwidth
   // and again
   rect[2][1] = rect[1][1]
   // and top again
   rect[3][1] = rect[0][1]
   // right
   rect[2][0] = rect[0][0] + (test[x] - rangemin) / totalrange * width
   // right again for poly
   rect[3][0] = rect[0][0] + (test[x-1] - rangemin) / totalrange * width
   // left is already set
   // draw the polygon
   Draw.FillPolygon(dc,rect)
   // draw y labels
   ...
endfor
... (draw x labels)

Now, for those that want a regular line graph that runs vertical, here is the modified code to draw lines instead of shaded areas, here is the code. The difference between this and above is that we collect all the points for a polyline (a bunch of lines connected together) in the loop, then draw them at the end. We could use moveto() and lineto() but this is slower. Again, I've trimmed the script here to make it readable. The full working script is in the attached sample.

... (init code and frame)
for (private x = 0, x < numpoints, x++)
   if (isempty(test[x]))
	  break
   endif
   // y
   rect[x][1] = Position[1][1] - x * Barwidth
   // x
   rect[x][0] = position[0][0] + (test[x] - rangemin) / totalrange * width
   // left is already set
   ... (y axis labels)
endfor
// draw the line
draw.Polyline(dc,rect)
... (x axis labels)

shadegraphsample.ctl

[Gammatech]

Hi,

As V6.0 is delayed, I revisited this version of your multicoloured vertical scrolling graph. On a faster machine than before it performs much more smoothly. However I have a problem, the graph scrolls UP the page rather than DOWN which I think is aestheticaly more pleasing. I have tried changing the coordinates and the plotting loop but to no avail, I either get a complete mess or no change, so how do I make it scroll down the page rather than up? I seem to have great difficulty with the array addressing! Please put me out of my misery.

Martin

[Gammatech]

Hi,

I've got another problem. One of the channels I want to draw is a deviation from target and is likely to vary either side of zero so I want to draw this about an origin set at the midpoint of the width. I feel sure that it is a case of setting both right AND left sides of the Rect array rather than only setting the right side and leaving the left at the left side of the canvas. Trouble is I cannot get my frazzled brain round the array addressing.

I have overlayed the canvas on a graph so that it has a nice grid and titling is easy. I have that scrolling down the page which is part of the reason for the canvas drawing to go down the page as well.

TIA

Martin

[AzeoTech]

First, scrolling the other way. You simply adjust the y position calcs. Currently its:

// top side of rect

rect[0][1] = Position[1][1] - x * Barwidth

// bottom side

rect[1][1] = rect[0][1] - Barwidth

Make it into:

// top side of rect

rect[0][1] = Position[0][1] + x * Barwidth

// bottom side

rect[1][1] = rect[1][1] + Barwidth

Remember, Position[1][1] is the bottom Y coord, while Position[0][1] is the top Y coord. You'll need to flip the labels too. Change this line:

draw.TextOut(dc,p,format("%.2f",x + rangemin))

to:

draw.TextOut(dc,p,format("%.2f",totalrange - x + rangemin))

Getting it to draw bars +/- from the center is simply a matter of setting rect[0][0] to the middle. Its currently set to the side (= position). So, in the header you have:

private rect = Position

just add right under this:

rect[0][0] += (Position[1][0] - Position[0][0]) / 2

or move the width declaration above rect and just do:

rect[0][0] += width / 2

[Gammatech]

Hi,

Thanks for the help.

I changed the two lines as you suggested and it plotted out of the defined canvas. With your reminder as to the coordinates, I changed the line for the top side of Rect as you suggest but left the line for the bottom side as it was and that works fine, plotting within the defines of the canvas.

To get the deviation from target to work the right side of Rect has to be :

Rect[1][0] = Rect[0][0] +Test[x]/TotalRange * Width

Rather than:

Rect[1][0] = Rect[0][0] + (Test[x] - RangeMin)/TotalRange * Width

I Also added clamps on the values so that they can not plot beyond the bounds set by RangeMin and RangeMax.

Now, as I understand it, the Canvas replots all of the "Graph" every page update and this puts quite a load on the processor. I know you have often said that the performance tab of TaskManager is not a good guide to processor usage but it does give an indication of the relative processor usage. With my application, including the canvas plotting, the CPU usage is double that without the canvas plotting (I have two scrolling graphs on the page) when displaying the graphics page.

This begs the question would it be less processor intensive if it were possible to plot a bar for the value at time t, then at time t+1 move this bar down one bar thickness and plot a new bar in the original position followed at time t+2 by moving both bars down one bar thickness and ploting a new bar etc. If this is possible it would save plotting out the entire graph every page update. would it be possible to use the methods of animation for the scrolling down?

An interesting aside. I have run the full aplication on a Dell Optiplex 2.8GHz with 2Gb memory running XP Professional and the processor usage is around 53% This is with a licenced copy of DaqFactory. On a Fujitsu Esprimo E2200 2.2GHz with 1Gb memory running Vista Business with a trial copy of DaqFactory it shows 22%CPU usage. Can you shed any light as to why the CPU usage should be so different. I have a Fujitsu Esprimo E4400 2.0Ghz with 1Gb memory running XP professional that has a Runtime licence for DaqFactory. Is it worth trying that with the full application to see whether the machine or the operating system is the common link? Both the tested systems have the latest DaqFactory version. I can update the runtime licence from the current 5.79 build 1573.

TIA

Martin

[AzeoTech]

First, the reason I say task manager is unreliable is because some things in DAQFactory run in the idle thread priority. This includes graphs and the canvas component. The idle thread priority is the lowest priority and therefore only gets CPU time when everything else in the system is done with it. So, even though the task manager may show 53%, 40% of that might be idle time used to draw graphs, so really you have 87% left for other apps. You can really see this if you create a reasonably complex graph and then set the page refresh to something like 0.1. You'll see the CPU usage spike to 100%, but the system will still remain quite responsive because most of that is in the idle thread to redraw the graphs ten times a second.

There is no real way to optimize this in script as the page is redrawn completely with every interval. I would just not worry about it until you actually see the computer slow down.

As to why the "slower" computer actually uses less processor power, you have to remember that there is a lot more to making a computer "fast" than just CPU ghz rating. Processor architecture (i.e. xeon's run faster than consumer CPUs), memory speed, motherboard architecture, hard drive speed, video card and much more determine the overall speed of the system. In this case, it could be as simple as a different video card. I don't know how much the Windows API passes to the card, but its probably a fair bit. The capabilities of the card to do 2D graphics would then determine how much CPU power was used to perform the functions that a better video card could handle on its chip.

[Gammatech]

Having built the main page that has two of the Canvas scrolling graphs, one being three coloured and the other being a two coloured deviation from target, I find on my effectively fastest machine that a graph display of longer than one minute (60 points to plot) is not possible as the time for a button press to respond on the same page comes up to a couple of seconds. In another application where the two canvas graphs are both three colour the maximum time for the graphs is closer to 30 seconds (30 points to plot) before the response to a button press becomes unacceptably slow. I would like the graphs to display 300 points and thus show the last 5 minutes but at that setting the whole thing just grinds to a halt. With the effect that clicking on the properties of the canvas component brings up the lower help screen but the actual code window never appears. When this happens the only way out is to use task manager to stop DaqFactory as with half the properties page popped up no other control is possible. The canvas filled graph is exactly the sort of display I want but it is just too slow/processor intensive.

Any other ideas for a faster filled graph display.

Presumably the graph component is coded down at machine level and so runs fast enough. When I tried the original stick plotting in the graph component it was OK up to 600 seconds and more but gave curious shuffling actions and occasional blank lines in the graph as the bar thickness does not seem to be related to the length of the graph.

TIA

Martin

[AzeoTech]

Post your current .ctl doc so we can see where you are at.

[Gammatech]

OK.

I have found part of the problem in that on the most troublesome page the refresh rate was set too high from an earlier test.

The full .CTL document is 400k and will not work without the serial input to feed the channels that are displayed in the canvas graphs. would it be better to send a simplified version that uses simulated data so that the canvas graphs can have data as there is a break out of the plotting loop if there is no data.

If you want the full document I would rather send it off forum.

Martin

[AzeoTech]

The whole doc is probably best. I can load it in safe mode. Feel free to send it direct.

[Gammatech]

Right I'll send the whole document direct to you. I have attached the simplified simulation as it shows up some slightly curious behaviour.

As the chart periods are increased a button that has a quick sequence to reset the accumulated data seems to be barely affected.

Buttons that have a SetTo action take increasing times to respond until eventually they do not respond at all.

It has three sliders to set the simulation values that have limits for the possible values. An auto run sequence initialises all the variables including the starting simulation values. Unless there is a delay in setting one of the values it seems to take the minimum limit from the slider when initially run?

If the chart period is fairly long so that the plotted bars are very thin their thickness seems to vary as the chart scrolls down. Is this an effect of aliasing? I have the screen update set to 1 second and the new data is available each second.

Thanks

Martin

shadedGraphSample_12_.ctl

[AzeoTech]

There is a lot of code in there. Its not surprising it runs kind of slow, and unfortunately, canvases get drawn in the main UI thread, unlike graphs. So, you need to make it fast, and as such need to optimize when you are doing complex stuff like what you are doing. DF script is not particular fast, especially compared to C, so its especially important.

Personally, I'd probably create a sequence that runs at priority 0 and precalcs everything for you and have the canvas draw whatever the last iteration of the sequence setup. Just make sure you do all your calcs on privates, and then move to globals that your canvas accesses at the end.

That said, there are a few optimizations that can be done in the code no matter what. This is all optimization 101 and applies to any programming language:

0) when looking for places to optimize, always start at the inner most loop and work out

1) you do:

for (private x = 0, x < numpoints, x++)
   // trap out if channel entry not yet available
   if (isempty(Ash[x]))
	  break
   endif
   ...
endfor

You should instead adjust numpoints to numrows(ash) if numrows() is < numpoints. Otherwise you are checking that if() with every iteration, and it only actually evaluates to true once, if at all.

2) Precalc mean(ash[x,ResponseTime+x]) to a private before the switch/case. Otherwise it gets recalced for each case until one matches.

3) replace simple if()'s with boolean math:

If(Speed[x] > 0.05)
	  // set local flag
	  PlotVal = 1
   EndIf

with

PlotVal = Speed[x] > 0.05

4) you have:

For(Private x = 0, x < NumPoints, x++)
// possible way to draw axis lines. Counter is incremented once each second so axis line
// is plotted one barwidth further down on each successive screen update
   if (x % GridSpace == Counter)
	  ...
   endif
endfor

which means the loop iterates a bunch, but only does something every Gridspace. Very wasteful. Instead do:

for (private x = counter, x < numpoints, x+=Gridspace)

[Gammatech]

Thanks for that, I have tried all your suggested changes and it speeds up the simplified demo quite a bit (in that I can have a longer chart period before the system response becomes too sluggish) In the full document it does not speed up enough to make the chart length acceptable so I suppose I'll just have to wait until the improved graphics becomes available.

I have the system time showing on the main screen to the second so have the refresh rate set at 1 second. If I have the refresh rate of a screen for the chart down at say 5 seconds then although the graph obviously updates only once every 5 seconds on its own, popping up the chart screen onto the main screen overrides the 5 second refresh of the pop up to the 1 second of the main screen. If I Reduce the resolution of the screen then I shall need to redesign all of the layout as most of the layout is then off screen.

I have used a freeware program to bench mark a number of the desk top computers here and it would seem that the graphics card is generally the weak link particularly with one manufacturer. The slower clock speed machine that appeared to perform much better than my highest clock speed machine had a far superior graphics system.

Until Version 6 (or an updated graphics version) I shall just have to backtrack to the basic graphs and forgo shaded charts unless there is some way of avoiding the relatively slow script, a great pity.

Martin

[AzeoTech]

Well, you could implement a delayed graph yourself, using the technique I mentioned. This is probably your best way of getting the graph to work well. Basically you calc everything in a background sequence running at priority 0, then at the end, you copy the results into some global (a class perhaps?) and then the Canvas uses the results to do the drawing. The canvas script would then be much shorter and therefore faster. You'd also want to do things to avoid the ifs and case statements. For example, you'd want to precalc the colors for each block too, so the loop gets reduced to basically:

for()
   Draw.setbrush()
   Draw.FillRect()
endfor

In fact your entire canvas script should be reduced to just some Draw. functions with a couple for() loops and 3 if()s. You shouldn't have any private variables except the counters for the loops and all your ifs() should just be if(x) where x is some variable precalced to true or false.

For this, I'd use a class/object to store the various settings, if only to avoid cluttering the namespace. It would also be easier to debug, because you could put all the drawing code in the class, and simply have the canvas call that code on a particular object. This would also make it so you could reuse the class for both of your graphs.

[Gammatech]

This sounds like a whole new learning curve for me, a died in the wool linear programmer! I'll read through all the help and documentation for Object/Class constructs and get back to you when I can grasp some of it.

Martin

[Gammatech]

Me again.

Having retrogressed to two vertical graphs I have noticed a couple of points with the graphs:

In the Line annotations, selecting 'Medium Solid Line' gives a line that is the virtually the same thickness as the 'Thin Solid Line'

On the Axis tab, selecting any plotting method apart from 'Line' gives some very peculiar effects on vertical graphs.

On the Colors + Fonts tab, one needs to set the font scaling low to get rid of the DaqFactory time scrolling down the side. This makes the Titles rather difficult so I have used a simplified canvas sequence to do the axis and title and then grouped the two components together so that movement is convenient.

Martin

[Gammatech]

I've searched the documentation and the help for some information as to defining a Class/object without success. Can you point me in the right direction?

I think I can shift all of the calculations required for each draw of the canvas to a sequence but then I need to store all the data into some sort of record (is this what you're referring to?) so that the canvas code can access all it needs.

I experimentally tried a declaration - 'Global Class CanvasData' which was not rejected by the compiler. If that is the route how do I define the structure? As a Pascal programmer I have no experience of Objects so again can you give me some pointers, the most sophisticated data structure I'm used to is the Pascal Record!

If the calculation sequence is running continuously I would need to synchronise the availability of new data for each draw of the canvas. Otherwise the end of the canvas drawing could begin the calculation sequence ready for the next draw.

How many points can a filled polygon have? Unless I build the data for the entire image for each plot I don't see how it can be much faster than the current code.

Thanks,

Martin

[AzeoTech]

The OOP documentation is in the forum here:

http://www.azeotech.com/board/index.php?&showtopic=3878

or, just search for "class"

You can either use the class simply as a structure, or you can put the calc routine as a member function of the class. Its up to you.

If the calculation sequence is running continuously I would need to synchronise the availability of new data for each draw of the canvas. Otherwise the end of the canvas drawing could begin the calculation sequence ready for the next draw.

This is why I said you have to do all the calcs to private variables and then update a global variable set at the end.

Polygons can have as many points as needed, as long as the resulting shape is a polygon (i.e. doesn't result in crossed lines).

Believe it or not, removing all that extra code will make a big difference. The slow part is not the rendering, but the execution of each line. If you eliminate 50% of the lines inside your inner-most and longest loop, you'll probably increase the speed of execution by 100% (2 times). That said, if you can make it as one big polygon, all the better.

[Gammatech]

Hi,

While I am waiting on Version 6 (and boning up on object programming) I set up two vertical graphs in place of the canvas graphs.

The vertical graphs show up a couple of peculiarities.

1) Only the line plotting works as all of the filled plotting tries to plot vertically rather than horizontally.

2) When trying to use multiple traces with boolean math to give colouration to the trace as the Y expression which is in fact time (Test.Time) needs to be repeated for each trace I have had to use Test.Time,Test.Time +0, Test.Time+0 +0, Test.Time-0 etc so that the graph component would accept them as different traces.

My actual data is derived from a radiological process and as such is very noisy second by second. For this reason I use the Smooth function {Smooth(Ash,ResponseTime)} where Ash is the noisy channel and ResponseTime is user adjustable around 10 to 30 seconds. When one tries to add the boolean math to select the plotting colour the comparison has to be Mean(Ash[0,ResponseTime])> SetPoint which makes the expression very long winded:

Smooth(Ash,ResponseTime) * (Mean(Ash[0,ResponseTime]) > HighLimit) + (NaN() * !(Mean(Ash[0,ResponseTime]) > HighLimit))

for values above the upper limit which is to be plotted in red.

and:

Smooth(Ash,ResponseTime) * (Mean(Ash[0,ResponseTime]) < HighLimit) * (Mean(Ash[0,ResponseTime]) > (Target + Band)) + (NaN() * !((Mean(Ash[0,ResponseTime]) < HighLimit)) * (Mean(Ash[0,ResponseTime]) > (Target + Band)))

for values below the upper limit but above the Target + Band which is to be plotted in yellow

and:

Smooth(Ash,ResponseTime) * (Mean(Ash[0,ResponseTime]) < (Target + Band)) * (Mean(Ash[0,ResponseTime]) > (Target - Band)) + (NaN() * !((Mean(Ash[0,ResponseTime]) < (Target + Band))) * (Mean(Ash[0,ResponseTime]) > (Target + Band)))

for values around the target +/- Band which is to be plotted in green.

Add in the other two trace expressions for red below the lower limit and yellow above the lower limit but below the target less the band and one has a potential nightmare just waiting for mistakes.

After entering all these expressions, the width of the vertical graph changes marginally as each different colour is plotted AND the whole trace changes for the period that the value is within a particular colour band.

Have you any suggestions how to get around this?

Quite often there are diagonal connecting lines between the last portion of a coloured trace and the next portion of the same coloured trace if there is any condition that allows more than one colour on the graph at the same time. I have tried juggling with the Line Gap Threshold without success.

Martin