Threaded Mode | Linear Mode

Lock-n-Code · 02-15-2009 05:13 PM

Hey folks, glad to see the forum back up.

I have some rather elaborate treeviews in my latest project and the largest takes approx. 6 seconds to populate. I'd like to speed this up. My short routine works perfectly to any depth desired by simply adding %paths (and changing a few obvious values), but as I said, it's slow:

Code:

alias treefill {

  ;// %treedir and %treeID are supplied by the calling routine

  ;// halt if tree has already loaded once

  if (%treeID isin %done.tree) { halt }

  var %line = 1

  while (%line <= $lines(%treedir)) {

    var %read = $read(%treedir,%line)

    if ((!%read) || ($numtok(%read,187) < 14)) { inc %line | continue }

    var %br = $gettok(%read,1,187)

    %parm = 2

    while (%parm <= 15) {

      %p [ $+ [ %parm ] ] = $gettok(%read,%parm,187)

      inc %parm

    }

    ;//  %p2 = flags, %p3 = icon, %p4 = sicon, %p5 = overlay, %p6 = state, %p7 = integral

    ;//  %p8 = textcolor 1, %p9 = textcolor 2, %p10 = textcolor 3, %p11 = %bgcolor 1, %p12 = bgcolor 2

    ;//  %p13 = bgcolor 3, %p14 = item text, %p15 = tooltip text

    ;// R indicates Root

    if (%br == R) { inc %path1 | %path = 2 | unset %path2 %path3 %path4 %path5 %path6 %path7 }

    elseif (%br isnum 2-6) {

      inc %path [ $+ [ %br ] ]

      %path = %br + 1

      %pun = %path

      while (%pun <= 7) {

        unset %path [ $+ [ %pun ] ]

        inc %pun

      }

    }

    elseif (%br == 0) { inc %path [ $+ [ %path ] ] }

    xdid -a dialog %treeID $+(%path1 %path2 %path3 %path4 %path5 %path6 %path7,$chr(9),+ $+ %p2 %p3 %p4 %p5 %p6 %p7 $rgb(%p8,%p9,%p10) $rgb(%p11,%p12,%p13) %p14,$chr(9), $+ %p15)

    inc %line

  }

  %done.tree = %done.tree $+ %treeID $+ ,

  unset %tree* %p*

}

So, is there anything you'd suggest I do to increase speed, short of using huge hash tables? One reason I'm using text files for all the data is ease of editing/debugging.

***twig*** · 02-15-2009 10:45 PM

1. reading from a file is quite slow
2. instead of using lots and lots of variables %path[1,2,3...etc], take a look at the /tokenize function in the mirc help file
3. or if you dont want to use tokenize, use 1 %path variable and append to it where necessary, rather than creating lots of variables

depending on the size of your text file, the reason why it is slow is the length of the file (how many lines) and how many paths it has to process per line.
unless there is a reason for you to do this dynamic loading, i suggest you hardcode the structure or use DCXML

Lock-n-Code · 02-16-2009 08:38 AM

(02-15-2009 10:45 PM)twig Wrote: 1. reading from a file is quite slow

depending on the size of your text file, the reason why it is slow is the length of the file (how many lines) and how many paths it has to process per line.

Yes, I noticed load time is proportionate to file size. The file in question is quite large (932 lines).
In case anyone's wondering why:

This project is a completely new trivia bot to replace my previous one, and with it come many new features. Unfortunately, this also means the dialogs are far more elaborate now -- you can say bloated; I won't be offended because I agree. I have trees for configuring everything from basic channel and question processing stuff to setting up team, tournament, ladder games, as well as bot personality (rudimentary AI). My purpose for this project was to design the most versatile and feature-rich trivia bot that I could for irc. To accomplish that, there must be a certain degree of excess.

The tree in question is a combination user stat/config box. It contains the various statistics (and there are a lot) for each user/player as well as a plethora of configurable parameters for each user (such as handicaps, filters, remote bot access grant levels, etc.).

The reason I put this in a tree view is so that the bot operator can drill down to view only those desired items, as opposed to stuffing a separate listview with many many columns, requiring a lot of scrolling.
My current design populates a listview on the left side, from which the operator selects a user; that user's stats/settings are then loaded into the treeview at right. It turns out that this is actually performed quickly. The lag occurs with the initial loading of the treeview control itself. Once the trees have been loaded/populated once, I can switch back and forth between tabs with zero display delay.

One file holds all user data, with each user/player occupying a single line -- admittedly, bloated tokenization.
But, from my experimentation with different dialog designs for this project, this tree method definitely looks much cleaner and is easier to use.

Forgive the long-winded reply, but I wanted to explain why my trees are so massive.
I can try certain things to streamline processing, such as your suggestion to tokenize, but I can't do anything about the quantity of data w/o sacrificing versatility or level of control over the bot (and the game).

***twig*** · 02-16-2009 08:41 AM

if thats the case, i'd look into finding a DLL or script that handles XML for you to convert your trivia information into a DCXML treeview so it can be loaded.

that way, you have some degree of readability in your file and it'll load faster than the processing being done here

Lock-n-Code · 02-17-2009 07:01 PM

I ditched the variables and tokenized to $identifiers. Still takes 6 secs to build that tree -- I also watched it filling RAM (task manager) as the trees were built. I wonder how much speedier it'd be if a binary file were used, assuming a perfected tree structure that no longer requires modification. Would DCXML be quicker still?

***twig*** · 02-18-2009 12:00 AM

not sure.
save a copy of your current code and attempt it in DCXML.
wont know till you try i guess

Travis · 02-18-2009 08:20 PM

Just my 2 cents ... Using a text file is the slowest wat you can go.

**Mpdreamz** · 02-20-2009 12:08 PM

Using the dataset feauture of the treeview or using the <item> markup directly with DCXML should be significantly faster since they call internal methods within dcx to add the items. This means that c++ doesn't have to report back to mirc and wait before mirc sends the next script line to be parsed by DCX.

Lock-n-Code · 02-21-2009 10:58 AM

I ended up hardcoding them into .mrc files, since this was a simple matter of pasting from the text files into mIRC editor and easily find/replacing the tokens with the required dcx strings. This resulted in a 3x performance boost (i.e., the largest tree takes 2.10 secs to build now). I may try writing it up in DCXML sometime just to play around with it, but I doubt that would shave much more time off, if any.

Unfortunately, all this tinkering has screwed my bot and/or Windows. DCX refuses to omit individual checkboxes in the tree now. IOW, every item now has a checkbox. I tried creating a new folder and building a virgin test bot with all-new, freshly-unzipped installs of mirc.exe, dcx.dll, the various .mrc loads, etc. And I tried different versions of dcx.dll. I also tried going back to the original routine as per my original post in this thread.

I even shut down the pc and unplugged the power cord for a while, since I've experienced weird Windows probs before if the mobo is left in standby.

Nothing fixes this. The only way I can now get rid of any checkbox is to eliminate the "checkbox" style from the dialog table -- which of course removes them all. I could understand maybe a file or two being corrupted, but I tried fresh replacements, ones which originally worked, and now they don't work! What gives?

Lock-n-Code · 02-21-2009 09:06 PM

I just tried a new test dialog and this time the checkbox style works, but only with dcx v1.3.7

I did install the correct matching dcx_tools for v1.4.0 when I tried it.

Threaded Mode \| Linear Mode populating a large treeview
Author	Message