Table structure?
Joakim Schramm
joakim at astrocalc.com
Fri Apr 28 19:10:58 CDT 2006
Hi,
I would like to ask for advice in a db/table structure question, if my idea
I have about would matter in any sense to Valentina (VCOM).
I want to set up a db to import a lot of data from it's current source and
have decided to go for binary links right away although it's completely new
to me. It looks briliant though at least on paper.
Anyhow, my dataset basically consist of 2 parts, now stored in 2 tables. The
first table has about 220 000 records and the second table has about 500 000
records. As my current datasourse isn't relational in a true sense I link
that bit in code now and the second table serve as something like a "lookup"
table for items of the first table. But each item in T1 only look for data
within a limited range of the whole data in T2, and several items can look
for same data so one could say it's something of a M:M relation. Important
to mention as well is that the T2 data need to be in a certain sequencial
order, either in the db or as it's returned for evaluation (like in an array
as it's returned now).
So my question is; I can devide the 500 000 records in to about 300
"logical" groups related to certain records in T1, would it be an idea to
split up this data in say 300 tables to easily exclude most of the
irrelative data to lookup in?
The lookup data is really sort of a DateTime value in 16 character string
where each DT has a result value in a second field, were DT mark a boundury
to set a value. I was thinking of using IVDateTime but the examples in the
docs (at least VCOM) are really sparse so not sure if this would do? Instead
of just a single line or two shoing how to asign and read a value, it would
be really bice (and helpful) if the was at least a small piece of code
showing a typical context to use it together with other calls. It exists in
a few places, but far to few. I know, you are probably aware of it, but
ponting it out anyway. The documentation is also a bit contradictional on
this, while the VCOM doc show you can set date format, kernel doc say it's
depends on system settings (internal storage), what matter to me of course
is what I get when need to use this value in my other code?
Just to give as an example, my data looks something like this with | as
field separator
19971026030000|-1
19980329020000|-2
19981025030000|-1
19990328020000|-2
19991031030000|-1
20000326020000|-2
20001029030000|-1
Lets say if I make a lookup using the value 19990429073000 (29 April 1999
07:30:00) it will return me -2 as it's more then 19990328020000 but not as
much as 19991031030000, but it's really not the comparison I am after here
but the best way to structure the data.
The above is just a small shunk from one set of about 2000, which is further
grouped into about 300 subsets. If I can use DateTime it would be great as
it would cut the needed storage space with about 50% if I have understood it
right, or even more if the string is UTF-16. Question is just if I can
effectively make a lookup with all the data in one table (I need to first
pick the correct group of 300, then the correct subgroup holding the data
above and determine were in the range the the given parrameter fits)?
Maybe this was all just too many words confusiong the whole thing, but... I
am just asking Valentina up for the first dance ;-)
Joakim
More information about the Valentina
mailing list