Collection Object -- I need to know your needs.

[Shaun Wexler]

On Aug 1, 2006, at 3:09 AM, Ruslan Zasukhin wrote:

> On 7/31/06 9:42 PM, "Shaun Wexler" <dev at macfoh.com> wrote:
>
>> FWIW, I am creating an EOF layer in VX for highly-efficient object
>> access, without any intervening SQL.
>
> Shaun, I did expect to hear from you on this issues :-)
> Let me know please what you think.

I am developing this EOF exclusively for Valentina "VX", and in the  
future might add import/export to other db types, but for now this is  
left as an application convention.  There are some important  
differences between my VX project and Apple/NeXT EOF.  I do not use  
Cocoa's built-in Key-Value coding/observation; instead I use the  
efficient SKWObject for the base class, and this provides  
notifications via non-retained observation, as well as efficient  
cache-aligned object, structs and ivars for memory optimization, plus  
fine-grained locking (mutex, r/w, etc), shared memory between  
processes, networking and other SKWBase classes, and it's thread-safe  
and the majority of methods are reentrant for concurrent thread  
access.  VX is also a client/server system, but the client side is  
pure ObjC and all of the C++ is contained in the server process (VX)  
and data is efficiently converted and/or bridged to the client  
natively, rather than as their C++ counterparts; the client does not  
[need to] link to any C++ libraries, nor to VX.  If the client and  
server processes are on the same host, they utilize shared memory and  
API's to communicate (and behave as if they are a single app),  
otherwise transmitting packets via TCP.


>> A BLOB of OIDs is called a "bag", CFBag, which is an unordered array
>> that allows duplicate values (OID's).
>
>> A NSSet is an unordered array of UNIQUE values; no two OID's are  
>> the same.
>
> Yes good point.
>
>> A NSCountedSet is similar to a bag; a ref-counted NSSet.
>
>> CoreFoundation and Foundation  collection classes are toll-free- 
>> bridged, and
>> use the same abstract  structure for their objects, however they are
>> class-clusters and use  various concrete subclasses for their  
>> underlying
>> implementations; you  ONLY access them thru their abstract base  
>> class methods.
>
>> You can store a set of RecID from a single table as a BLOB, using a
>> BitSet, but in most cases (especially for small sets), it will cost O
>> (n) to lock & walk the existing array for insertion and removal, to
>> maintain an actual set manually.
>
> Hmm, I think BitSet is not good for this. Better use ArraySet.
> Actually more correct to say:
>         array of ulongs.
>
>
> Shaun, now return to above my text. I see that Valentina still can  
> get few
> new features to much better fit into job that you and Philip want  
> to do.
>
> For example, right now, yes you CAN store array of ulongs into BLOB.
> But you have no way build index on this ulong values.
>
> As workaround can be store them into TEXT field as coma separated  
> values,
> and mark this as IndexByWords...Also assign in Collator - order as  
> numbers.
> But this is already looks somehow less efficient.
>
> You see?

What would satisfy this requirement is a Red-Black tree structure  
stored as BLOB and thus has no need to be indexed.  You can fit it  
into 128 bytes per record (require power-of-two, and is G5 cacheline  
sized), using a 64-bit OID, 23-bit parent index whose high bit  
denotes the node's color, and two 24-bit children indexes, where all  
indexes are relative to BLOB bufferstart + index * nodesize.   
Searching can walk the tree, or you can quickly copy all objects into  
an array, bitset, etc.  Also is possibility to store objects and tree  
index as two buffers which each grow by multiples of page size (64  
objects), and that is efficient for copyless memmove by page  
remapping (only needed when modifying stored BLOB).

> So I think you and Philip should try find such issues and push us  
> to add
> them into Valentina...
>
> I understand of course that you develop commercial product, and  
> need fast
> time, so are forced to use what you have right now...but still I  
> need have
> info from you both WHAT IS REQUIRED to make Valentina better.
>
> Tell me your dreams in regard to db engine!

I'm working on it...  ;-)
-- 
Shaun Wexler
MacFOH
http://www.macfoh.com