Need advice

[Brendan Murphy]

Ruslan wrote:
>>>> With version 2.3, the internals have changed, so I am assuming my
>>>> previous measurements don't apply. Starting over...
>>>
>>> In fact must be the same as 1.x
>>> What exactly you have to find?
>>
>> With 1.1 file sizes exploded proportionally with segment size.
>
>> With 2.3 I would characterize it as leveling off as the the number
>> of records increases.
>
>> In other words, there is no significant
>> difference in file sizes for 100,000 records using segments sizes
>> of 4k, 8k, 16k, and 32K.
>
> And THIS IS correct and expected result.
> Hmm, I believe 1.x did work in the same way, Brendan.

The numbers don't lie. Way back when I first measured 1.11 the
file sizes exploded across the board in proportion to the segment
size. For 2.3 it does not explode (which is a good thing).

>> There is a significant difference (percentage wise) when there is  
>> only 10,
>> 100, 1000, and 1000 records. So this is definitely a different  
>> behavior from
>> version 1.1.
>
> Still not very clear, but I think not very important.
>
> Again, initial size of db can be even few MB. ONLY WHEN you insert  
> few MB of
> data it will start grow.

Here are the measurements for version 2.3 for the different
segment sizes. The exact same input data was entered for each
database for the corresponding record sizes.

4K segment size:
10: 0.668 MB
100: 0.708 MB
1000: 1.1 MB
10000: 5.8 MB
100000: 52.7 MB

8K segment size:
10: 1 MB
100: 1 MB
1000: 1.4 MB
10000: 6.1 MB
100000: 52.9 MB

16K segment size:
10: 1.6 MB
100: 1.6 MB
1000: 2.1 MB
10000: 6.7 MB
100000: 53.5 MB

32K segment size:
10: 3.2 MB
100: 3.2 MB
1000: 3.6 MB
10000: 8.3 MB
100000: 55 MB

There is not a linear growth pattern across the segment sizes.
Understanding why it behaves this way can help me avoid any
pitfalls (if they exist). Since what you have stated for
the behavior for version 2.3 is slightly different from the actual
measurements, there is something going on which you are not aware.
Perhaps there is some kind of optimization you can make in a
future release.

For me 4K segment size visibly slows performance down. 16K segment
size imposes a 50% file size increase for typical file sizes (at
the extreme record size there is no difference).

>> I notice a slow down when using 4k segment sizes for very large
>> (100,000 records) files, but no difference in speed for 8k, 16k,
>> and 32k segment sizes. So it looks like 8k seems to be optimal.
>
> I think you have not come to barrier when 8K also will be visibly  
> slower of
> 16K  :-)

It is not visibly slower.