Indexing or sorting ordered arrays.

[Ed Kleban]

On 11/20/05 2:16 AM, "Ruslan Zasukhin" <sunshine at public.kherson.ua> wrote:

> On 11/20/05 9:55 AM, "Ed Kleban" <Ed at Kleban.com> wrote:
> 
>> On 11/20/05 1:39 AM, "Ruslan Zasukhin" <sunshine at public.kherson.ua> wrote:
>> 
>>> On 11/20/05 9:31 AM, "Ed Kleban" <Ed at Kleban.com> wrote:
>>> 
>>>> When you set the fIndexed EVFlag, then V2 builds an index, correct?
>>>  
>>>> 
>>>> 5) I'm totally clueless because you're simply not indexing or sorting in
>>>> any
>>>> of these ways.
>>> 
>>> Actually nature of DBMS indexing is sort of inserts.
>> 
>> Doh!  Of course.  It's incremental to deal with a single record add at a
>> time.  Makes perfect sense for adding individual records.  So then
>> re-indexing an already or mostly sorted array would be very fast.  Yep, yep.
> 
>  
>> But that also suggests that if I'm going to supply a very large array of
>> randomly ordered elements and then ebabke the "Unique" or "Indexed" flag,
> 
>> that I might be better off presorting them first myself, since an insertion
>> sort in that circumstance could take much longer... that is if I happened to
>> have a plugin or RB-written sort routine that could compete with the speed
>> of your C++ compiled insertion sort.  True?
>>  
>> Doesn't much matter.  Most of the arrays I'm concerned with right now all
>> happen to be pre-sorted.
> 
> Ed, Valentina take care self to do best things.
> 
> Valentina do very complex indexing. We have size of code related to
> indexing, bigger than SqlLite itself is. :-)
> 
> 
> Again I will repeat. IT is good idea at first make prototype using simple
> logic. And ONLY if you will see some performance problems there is sense
> start think on improvements. Most probably you will not see them at all.
> 

That is very good advice, and well worth repeating.  It is good advice for
any programming project to do a decent design, make a rapid prototype, meter
the heck out of it, and tune the performance where and if needed.  You get
far more progress in far less time.

That said, I ask such questions for several reasons, including:

1) I have an inherent curiosity not only about how to use Valentina, but how
it works.  

    You're certainly under no obligation to share that information -- it may
well even be confidential.  That's fine!  I don't want to press you
regarding the recipe for the "special sauce" that distinguishes Valentina
from its Rivals.  To the extent you are willing to answer such questions
however, it helps me better understand the capabilities, limitations, and
design tradeoffs in using the product as tool.  To the extent you prefer not
to answer or simply don't know (and don't even care about knowing) I can
write and run some tests if it's so important to me that I really need to
know -- typically it's not.
 
2) I'm not writing code from scratch.  Rather I'm tacking on Valentina to a
very large existing code base which gives me a large number of options I
normally wouldn't have if implementing a design from scratch.

    As an example, when I want to populate a table, I may well have the data
available to do so in two or three different forms -- some random, some
already sorted, some hashed for faster access than Valentina will ever be
able to match.  Better understanding of Valentina's capabilities and
limitations helps me to make better choices for using or transferring these
sources to V2.

3) As previously discussed, we share a common addiction and I simply can't
help myself :-)


======

Sounds like I'm probably at the point of diminishing returns at this point
and am rapidly becoming more of a pest than a list contributor.  I'll
endeavor to tighten the clamp on my curiosity and make less demands on your
generous penchant for sharing information.  I certainly don't want to wear
out my welcome with one of the best tutors on structural efficiency I've run
into in a very long time.

In that vein, I'll stop short of listing reasons 4), 5), and 6).