I use a virtual list view to show data from a database.

This is now the structure of the "refresh" procedure

    [*]Empty listview
    [*]Select data from the database
    [*]Create an array of structures, to hold the data from the database
    [*]Fill the array with the data from the database

    If the database is large ( > 1000 rows) it takes some seconds to show the data.

    Since I have a virtual listview I can show the data as soon as I read them from the database.
    But to do this I have to create a separate thread.

    I want to create a procedure like this

      [*]Empty listview
      [*]Select data from the database
      [*]Create an array of structures, to hold the data from the database
      [*]Create a thread that read data from the database and fill the array

      This structure seems ok but I have to syncronize the 2 thread.

      My idea is this:
      I add a byte to each structure that holds data.
      In the beginning this byte is zero. When the worker thread retrieve a row from the database, it writes this row in the structure and set the

      special byte to ONE.

      When the main thread get LVN_GETDISPINFO, it checks the special byte and enter in a loop until this byte is ONE
      @@:
      
      invoke SwitchToThread
      cmp special_byte, 1
      jne @B

      But SwitchToThread exist only in NT/2k/Xp, and I need a program that works also under 9x/Me

      Any idea????
Posted on 2004-04-10 10:16:50 by greenant
use events, sleep, and waitforsingleobject
Posted on 2004-04-10 12:36:48 by comrade
Yes, I can use Sleep, 1, but I don't like it

How can I use events?
I mean, how should I use events in my situation?
Posted on 2004-04-10 14:03:18 by greenant
okay, I thought a bit about your problem, and I'm not sure I understand it entirely - but I assume you're reacting to the virtual callbacks and don't want the UI to block while waiting for 1000 results to be returned from the database, and that you can't cache results?

Perhaps something like this, then... create an array to hold your data items, plus a dword to indicate how many items in the array are valid. Create a CRITICAL_SECTION object to protect access to this structure. Also, create an EVENT object to signal "data is ready".

When you're about to refresh the view, start the worker thread before sending the repaint to the listview. The worker thread wait for data from the database, and as soon as some is received, it enters the critical section, adds the items to the array, updates the ValidItems counter, and leaves the critical section. Then it signals the data-ready event, and waits for more data from the database.

When your wndproc receives a listview virtual message:
LOOP:
Enter the critical section for the array. If the requested index lies within the valid range, it can be served. Leave critical section. If the requested item was NOT within the valid range, you'll have to WaitForSingleObject on the data-ready even, and JMP LOOP.

Of course this is just one way of doing it, it might not work, it might not be what you want, and there might be better methods :) - but it was what I could come up with off the top of my head.
Posted on 2004-04-10 14:44:26 by f0dder
I don't need the critical_section to protect the array.

I have found the bottleneck, it isn't in the database fetch routine.
I don't need anymore this help.
Thanks
Posted on 2004-04-11 03:57:18 by greenant

I don't need the critical_section to protect the array.

If you are doing multithreaded programming with two threads accessing the array, you need either a CRITICAL_SECTION or another synchronization primitive. Don't even doubt it.

But well, nice that you found the bottleneck - what was it?
Posted on 2004-04-11 06:47:47 by f0dder
Repainting of listview I guess.
Posted on 2004-04-11 06:57:13 by roticv
events for synchronization of threads, as opposed to "special_byte"
CreateEvent, SetEvent, ResetEvent

Sleep(0) is actually used to switch threads, it is not a hack, so there is no reason for dislike
Posted on 2004-04-11 12:36:30 by comrade
In my database I have 3 tables.
In the main table, 2 columns are 2 foreign key.
When I read from the database, I keep two representation of my data, one is numeric and the other is string.
Imagine 2 array, one that contains number (the ones read from the database) and the other is the "printable" rapresentation of the first array. When I get LVN_GETDISPINFO i read from the second array.

This is the list of things I do when I read from the database
[*] Read database
[*] Populate first array (numeric)
[*] Convert first array (numeric) into second array (string)

My two foreing key are ID (dword values) that I use as Primary Key into other two different tables to retrieve the associated string.
So I use "SELECT name FROM name_table WHERE ID = ?"
And I should execute this query twice (one time for each foreing key) for every row. If I have 1000 rows I waste so much time.
So I cached the query response.
Sice name_table has only a few rows (about ten), I have created a cache.
When I have to translate ID into a name, first I look into the cache. If I find it I read the name from tha cache, but if I have a cache miss, I execute the SQL query on the database and then I insert the value into the cache.
Now I have no delay.

-------

I can't replace "special bytes" with Events or Critical Sections, because I need a special byte for each row.
If I had 1000 rows, I should create 1000 critical sections or 1000 events, and this is a waste of CPU time, memory and handles

-------

If I use special byte I don't need critial sections to syncronize two threads, because one thread (the worker thread) has a WRITE-ONLY access to this special byte, while the other thread (the one that reply to LVN_GETDISPINFO) has a READ-ONLY access.
I don't have race conditions.

In the beginning the special byte is zero.
The GETDISPINFO thread enter a loop and wait for the byte becoming one.
The worker thread have exclusive access to the piece of array to update, because the GETDISPINFO is locked.
Then is read from the database, insert data into the array, and then, at the end of the update, when it doesn't need anymore exclusive access to that piece of the array, it set the special byte.

Doing so, GETDISPINFO thread "wakes up" and read the data from that row.

I cannot have race conditions
Posted on 2004-04-12 03:02:47 by greenant