[V4RB] 1.x Open() Delay in Win32

[Russ Tyndall]

On 4/6/06 4:48 PM, "Ruslan Zasukhin" <sunshine at public.kherson.ua> wrote:

> On 4/6/06 6:35 PM, "Russ Tyndall" <fitzbew at nc.rr.com> wrote:
> 
> Hi Russ,
> 
>> I did some more research on this (have to reboot to get back into "slow"
>> state), and also poked around on the 'Net to make sure I am interpreting the
>> Task Manager readings correctly.
>> 
>> In Win XP Home (SP2), 2.5ghz Celeron D with 512 mb of physical RAM, it takes
>> 24 wall-clock seconds for V4RB 1.x to open a 155mb encrypted database the
>> *first* time after a startup/reboot of the PC.  Subsequent openings are very
>> speedy.
>> 
>> The amount of memory used on the PC increases by only 3mb during the Open()
>> call.  This is true during each Open(), not just the first. So, I do not
>> think the delay is due to the OS caching the database. (System cache and
>> Kernel memory hardly change at all.)
>> 
>> The first time the DB is opened, this 3mb of memory is consumed very slowly
>> during the 24 secs.  On subsequent openings, this 3mb is consumed quickly.
> 
> Actually we have see that on some not-clear reason Windwows XP do much
> slower flushed of HDD then do MAC OS X using practically the same HDD
> hardware. 
> 
> We have se this effect about 3-4 years ago also.
> 
> The reason open takes so long time in your case is that you have in db a
> lots of VarChar and index files. When Valentina 1.x open such PAGE-Files it
> needs read their header. So this cause many random access to HDD. And this
> not require a lots of RAM as you see.
> 

I only have 6 varChar fields spread across 10 tables. Only 1 varChar field
is indexed. But some of my tables do have several indexes (mostly strings).

So, if I follow you (in a general sense) Valentina 1.x (and 2.x, based on a
subsequent comment) kind've "pre-scan" the database during Open(), gathering
info about the db that stays persistent until the PC reboots.  This
information somehow stays persistent on the PC, in system cache or such. But
not the entire contents of the 155mb db, just some smaller amounts of data
which I presume are used to enable those stunningly fast queries.

I do not fully understand all the details, but what I do understand makes
sense to me and matches what I have seen with my own eyes.

>> My (virtual) Win98se PC with 128mb of RAM does not hesitate at all during
>> Open().
> 
> Sounds like Win98 was faster of XP on random access.
> 
>> Even more disappointing, Vstudio suffers this slowdown also when opening an
>> unencrypted converted version of the same 155mb database so I don't think
>> migrating to 2.x will help with this. It also rules out a Realbasic cause.
>> (Also, that would seem to rule out a problem relating to the encryption.)
> 
> I have not catch. Valentina Studio open your 1.x daatbase. Right ?
> But this means that 1.x engine is used.

Vstudio seems to open my 1.x db, but it never appears in the interface.  So,
I take the unencrypted version of my full-sized db and convert it to 2.x
using the option in the Utilities menu.  Then, I can open the converted file
in Vstudio successfully --- but first time is very slow just like 1.x.
Subsequent openings within the same session are ok. So, I conclude migrating
to 2.x will not help me any.

> 
> To see work of 2.x engine, you need do Convert_1_2 of your db into 2.x
> format. Then open this new 2.x database using Valentina Studio. Then only
> will work 2.x engine.
> 

Yes, after conversion to 2.x, my db appears in Vstudio correctly, just slow
on first open.

>> The delay *does* appear to be related to db size.  I experimented with a
>> trimmed down version of the db (28mb), and it opened quickly every time.
>> Also interesting, the Open() call on this smaller db does not cause *any*
>> noticeable memory to be consumed.
> 
> Hmm. What cache size you use then ?
> Try set it bigger
> 

I believe I tried larger caches first thing after I noticed this problem,
but I shall try again.  Right now, I am allocating about 160mb for cache.
But if the core problem is XP poorly reading the db file on the hard drive
during the "pre-scan", *can* a larger Valentina cache really help?  No
matter, it's easy to test.

>> I have another 388mb db that uses an alternative engine that opens swiftly
>> *every* time on the same machine.
>> 
>> I'm quite puzzled about what to do here, but I don't think any of us would
>> expect either 1.x or 2.x to take so long to simply open a 100-200mb db.
> 
> Agree. 
> 
> We was going make lazy access to header of PageFiles to not do this on open.

If I understand correctly, you hope to one day make this "pre-scan" go away.
Which, I gather would fix my problem.

Ok!  At least I have a sense of what is happening and why it appears to only
be happening to me.  Now, I have to figure out what to do about it.

Thank you for taking the time to explain in more detail!

-- 
Russ Tyndall
Wake Forest, NC