From info at vallemediatime.com Tue Nov 1 13:32:36 2005 From: info at vallemediatime.com (info@vallemediatime.com) Date: Tue Nov 1 06:32:57 2005 Subject: V4MD crash can delete data of db References: Message-ID: <000801c5dee0$57becd90$2e5b3897@vmtrm4p9ipbkv2> Hi Ruslan, ----- Original Message ----- From: "Ruslan Zasukhin" To: Sent: Thursday, October 27, 2005 7:31 AM Subject: Re: V4MD crash can delete data of db > On 10/27/05 12:35 AM, "info@vallemediatime.com" > wrote: > > Hi Paolo, > > > Hi Ruslan, > > from two days sometimes I get a new problem: > > if I stopped my movie after some commands I get a crash and the size of vdb > > file become very little and the data are deleted. > > (Btw, I save a copy of db in start session) > > > > For the crash I understand that it's possible because I'm testing some new > > methods for me, but why this can delete the 90% of data in vdb file? > > Hmm, so file was e.g. 1 MB in size and become 100Kb ??? > > Exists way to reproduce ? I've tried to reproduce it, the error is in a SQL query, if I repeat the same query two or three times I get the crash and/or the db corruption... btw: first to repeat a query I always set gCursor = 0 Two days ago I've download the new beta 2.1b1: the same query always works and the crash is missing... I'm happy for this. Probably in the 2.0.5 version it was a bug somewhere about this, but I don't full understand where. Paolo > > > I use this: > > > > on stopMovie > > gCursor = 0 > > Valentina.Shutdown() > > Valentina = VOID > > end > > -- > Best regards, > > Ruslan Zasukhin > VP Engineering and New Technology > Paradigma Software, Inc > > Valentina - Joining Worlds of Information > http://www.paradigmasoft.com > > [I feel the need: the need for speed] > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina > > > > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.361 / Virus Database: 267.12.5/147 - Release Date: 24/10/2005 > > From sunshine at public.kherson.ua Tue Nov 1 15:09:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 1 07:10:04 2005 Subject: V4MD crash can delete data of db In-Reply-To: <000801c5dee0$57becd90$2e5b3897@vmtrm4p9ipbkv2> Message-ID: On 11/1/05 2:32 PM, "info@vallemediatime.com" wrote: >> Exists way to reproduce ? > I've tried to reproduce it, the error is in a SQL query, if I repeat the > same query two or three times I get the crash and/or the db corruption... > > btw: first to repeat a query I always set gCursor = 0 > > Two days ago I've download the new beta 2.1b1: the same query always works > and the crash is missing... I'm happy for this. We too. So now first query return not 0 but some cursor with records? > Probably in the 2.0.5 version it was a bug somewhere about this, but I don't > full understand where. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From info at vallemediatime.com Tue Nov 1 14:27:51 2005 From: info at vallemediatime.com (info@vallemediatime.com) Date: Tue Nov 1 07:28:12 2005 Subject: V4MD crash can delete data of db References: Message-ID: <000801c5dee8$0f86f950$2e5b3897@vmtrm4p9ipbkv2> ----- Original Message ----- From: "Ruslan Zasukhin" To: Sent: Tuesday, November 01, 2005 2:09 PM Subject: Re: V4MD crash can delete data of db > On 11/1/05 2:32 PM, "info@vallemediatime.com" > wrote: > > >> Exists way to reproduce ? > > I've tried to reproduce it, the error is in a SQL query, if I repeat the > > same query two or three times I get the crash and/or the db corruption... > > > > btw: first to repeat a query I always set gCursor = 0 > > > > Two days ago I've download the new beta 2.1b1: the same query always works > > and the crash is missing... I'm happy for this. > > We too. > > So now first query return not 0 but some cursor with records? Yes, but this in 2.0.5 also the strange and dangerous problem happens (first of 2.1) when I repeat twice o more time the same query... the query was like this (if I good remember).. now I have new code, sorry for my not too precise information. query = "SELECT\ Appuntamenti_ID,\ Appuntamenti_Giorno || '-' || Appuntamenti_Mese || '-' || Appuntamenti_Anno,\ Appuntamenti_Ore,\ Persone_Cognome,\ Appuntamenti_Mediatore\ FROM Appuntamenti JOIN Persone\ ON Appuntamenti_PersoneDoc_Ref = Persone_ID" Paolo From jordan at digi-net.com Tue Nov 1 14:35:31 2005 From: jordan at digi-net.com (Ken Jordan) Date: Tue Nov 1 13:33:54 2005 Subject: [V4RB] BLOB problems Message-ID: <4367C383.9020209@digi-net.com> I have a very small program to test storing and retrieving binary data using a VBlob. (2.0.4 - OS X.3.9) app.open: --------------- Valentina.Init(10 * 1024 *1024) Valentina.DebugLevel = 3 Valentina.ThrowExceptions = true Window1.PushButton1.Action: ------------------------- dim f as FolderItem = GetFolderItem("vtest.vdb") if f.Exists then f.Delete() dim test_database as VDatabase = new VDatabase test_database.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) dim file_table as VTable = test_database.CreateTable("ftable") call file_table.CreateBLOBField( "file", 1024 ) f = GetFolderItem("file.rar") file_table.BlobField("file").FromFile(f) f = GetFolderItem("saved_file.rar") file_table.BlobField("file").ToFile(f) test_database.Close() test_database = nil app.Close: ----------------------- Valentina.Shutdown() file.rar is a 1.8 MB rar containing Valentina examples. Two problems: 1) "diff file.rar saved_file.rar" yields "Binary files file.rar and saved_file.rar differ" Stuffit will open file.rar, will not open saved_file.rar 2) Program crashes at test_database.Close() Any ideas what I am missing?? TIA, Ken From sunshine at public.kherson.ua Tue Nov 1 21:48:26 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 1 13:48:49 2005 Subject: [V4RB] BLOB problems In-Reply-To: <4367C383.9020209@digi-net.com> Message-ID: On 11/1/05 9:35 PM, "Ken Jordan" wrote: Hi Ken, > I have a very small program to test storing and retrieving binary data > using a VBlob. (2.0.4 - OS X.3.9) > > app.open: > --------------- > Valentina.Init(10 * 1024 *1024) > > Valentina.DebugLevel = 3 > Valentina.ThrowExceptions = true > > > Window1.PushButton1.Action: > ------------------------- > dim f as FolderItem = GetFolderItem("vtest.vdb") > if f.Exists then f.Delete() > > dim test_database as VDatabase = new VDatabase > test_database.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) > > dim file_table as VTable = test_database.CreateTable("ftable") > call file_table.CreateBLOBField( "file", 1024 ) Actually here should be also: > f = GetFolderItem("file.rar") file_table.SetBlank() > file_table.BlobField("file").FromFile(f) file_table.AddRecord() NOW YOU HAVE record in table on disk. > f = GetFolderItem("saved_file.rar") > file_table.BlobField("file").ToFile(f) > > test_database.Close() > test_database = nil > > app.Close: > ----------------------- > Valentina.Shutdown() > > file.rar is a 1.8 MB rar containing Valentina examples. > > Two problems: > 1) "diff file.rar saved_file.rar" yields "Binary files file.rar and > saved_file.rar differ" > Stuffit will open file.rar, will not open saved_file.rar > 2) Program crashes at test_database.Close() > > Any ideas what I am missing?? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jordan at digi-net.com Tue Nov 1 15:10:46 2005 From: jordan at digi-net.com (Ken Jordan) Date: Tue Nov 1 14:09:10 2005 Subject: [V4RB] BLOB problems In-Reply-To: References: Message-ID: <4367CBC6.5040409@digi-net.com> Ruslan Zasukhin wrote: >On 11/1/05 9:35 PM, "Ken Jordan" wrote: > >Hi Ken, > > >>I have a very small program to test storing and retrieving binary data >>using a VBlob. (2.0.4 - OS X.3.9) >> >>app.open: >>--------------- >> Valentina.Init(10 * 1024 *1024) >> >> Valentina.DebugLevel = 3 >> Valentina.ThrowExceptions = true >> >> >>Window1.PushButton1.Action: >>------------------------- >> dim f as FolderItem = GetFolderItem("vtest.vdb") >> if f.Exists then f.Delete() >> >> dim test_database as VDatabase = new VDatabase >> test_database.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) >> >> dim file_table as VTable = test_database.CreateTable("ftable") >> call file_table.CreateBLOBField( "file", 1024 ) >> >> > >Actually here should be also: > > >> f = GetFolderItem("file.rar") >> >> > > file_table.SetBlank() > >> file_table.BlobField("file").FromFile(f) >> >> > > file_table.AddRecord() > >NOW YOU HAVE record in table on disk. > > >> f = GetFolderItem("saved_file.rar") >> file_table.BlobField("file").ToFile(f) >> >> test_database.Close() >> test_database = nil >> >>app.Close: >>----------------------- >>Valentina.Shutdown() >> >>file.rar is a 1.8 MB rar containing Valentina examples. >> >>Two problems: >>1) "diff file.rar saved_file.rar" yields "Binary files file.rar and >>saved_file.rar differ" >> >> >> Stuffit will open file.rar, will not open saved_file.rar >>2) Program crashes at test_database.Close() >> >>Any ideas what I am missing?? >> >> Thanks, Ruslan. That cures problem 2, but the files are still not the same. ??? From sunshine at public.kherson.ua Tue Nov 1 22:24:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 1 14:25:09 2005 Subject: [V4RB] BLOB problems In-Reply-To: <4367CBC6.5040409@digi-net.com> Message-ID: On 11/1/05 10:10 PM, "Ken Jordan" wrote: > Thanks, Ruslan. > That cures problem 2, but the files are still not the same. ??? Hmm, please try call file_table.CreateBLOBField( "file", 8 * 1024 ) Or 16 Kb So how file differ? It have the same size but differ internally? Or size also is other? Can you try other files? Smaller? E.g 20 Kb 100 Kb -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jordan at digi-net.com Tue Nov 1 15:58:20 2005 From: jordan at digi-net.com (Ken Jordan) Date: Tue Nov 1 14:56:41 2005 Subject: [V4RB] BLOB problems In-Reply-To: References: Message-ID: <4367D6EC.5080509@digi-net.com> Ruslan Zasukhin wrote: >On 11/1/05 10:10 PM, "Ken Jordan" wrote: > > > >>Thanks, Ruslan. >>That cures problem 2, but the files are still not the same. ??? >> >> > >Hmm, please try > > call file_table.CreateBLOBField( "file", 8 * 1024 ) > >Or 16 Kb > > >So how file differ? It have the same size but differ internally? >Or size also is other? > >Can you try other files? Smaller? E.g > 20 Kb > 100 Kb > > Works at 1Kb for 20 Kb and 160 Kb files, needs at least 4 Kb for 1.8 MB file. Thanks again, Ruslan. Ken From jordan at digi-net.com Tue Nov 1 16:14:26 2005 From: jordan at digi-net.com (Ken Jordan) Date: Tue Nov 1 15:12:48 2005 Subject: [V4RB] BLOB problems In-Reply-To: <4367D6EC.5080509@digi-net.com> References: <4367D6EC.5080509@digi-net.com> Message-ID: <4367DAB2.2090504@digi-net.com> Ken Jordan wrote: > Ruslan Zasukhin wrote: > >> On 11/1/05 10:10 PM, "Ken Jordan" wrote: >> >> >> >>> Thanks, Ruslan. >>> That cures problem 2, but the files are still not the same. ??? >>> >> >> >> Hmm, please try >> call file_table.CreateBLOBField( "file", 8 * 1024 ) >> >> Or 16 Kb >> >> >> So how file differ? It have the same size but differ internally? >> Or size also is other? > Size is the same, differ internally. (sorry, forgot to answer in last post) >> >> Can you try other files? Smaller? E.g >> 20 Kb >> 100 Kb >> >> > Works at 1Kb for 20 Kb and 160 Kb files, needs at least 4 Kb for 1.8 > MB file. > > Thanks again, Ruslan. > > Ken From sunshine at public.kherson.ua Tue Nov 1 23:29:54 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 1 15:30:15 2005 Subject: [V4RB] BLOB problems In-Reply-To: <4367D6EC.5080509@digi-net.com> Message-ID: On 11/1/05 10:58 PM, "Ken Jordan" wrote: >> Hmm, please try >> >> call file_table.CreateBLOBField( "file", 8 * 1024 ) >> >> Or 16 Kb >> >> >> So how file differ? It have the same size but differ internally? >> Or size also is other? >> >> Can you try other files? Smaller? E.g >> 20 Kb >> 100 Kb >> >> > Works at 1Kb for 20 Kb and 160 Kb files, needs at least 4 Kb for 1.8 MB > file. So if set segment-size 4Kb then your 1.8mb file is written ok, right ? Interesting, I do not see reason why 1Kb do not work then. We need test this. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jordan at digi-net.com Tue Nov 1 17:30:18 2005 From: jordan at digi-net.com (Ken Jordan) Date: Tue Nov 1 16:28:42 2005 Subject: [V4RB] BLOB problems In-Reply-To: References: Message-ID: <4367EC7A.8030002@digi-net.com> Ruslan Zasukhin wrote: >On 11/1/05 10:58 PM, "Ken Jordan" wrote: > > > >>>Hmm, please try >>> >>> call file_table.CreateBLOBField( "file", 8 * 1024 ) >>> >>>Or 16 Kb >>> >>> >>>So how file differ? It have the same size but differ internally? >>>Or size also is other? >>> >>>Can you try other files? Smaller? E.g >>> 20 Kb >>> 100 Kb >>> >>> >>> >>> >>Works at 1Kb for 20 Kb and 160 Kb files, needs at least 4 Kb for 1.8 MB >>file. >> >> > >So if set segment-size 4Kb then your 1.8mb file is written ok, right ? > > Right. >Interesting, I do not see reason why 1Kb do not work then. >We need test this. > > From rjb at robelko.com Wed Nov 2 00:06:31 2005 From: rjb at robelko.com (Robert Brenstein) Date: Tue Nov 1 17:11:56 2005 Subject: LoadDump files not selectable Message-ID: > > On 10/25/05 9:38 PM, "Chris Sheffield" wrote: >> >>> That is not the case, Ruslan. I've had this same problem with VApp. >>> It won't let you select any type of file. I've tried playing with >>> the type/creator codes using ResEdit and had no luck. I've only been >>> able to load a dump using Valentina Studio (but even with that, it's >>> necessary to have another database open first, which I think must be >>> a bug). >> >> I think it needs only set text/text. > >Vapp doesn't LoadDump with TEXT,TEXT text/TEXT TEXT,text or text/text... > >Sigh :-( > Have you tried setting the creator to VALA and type to TEXT? The creator should not matter, actually. I would check whether the xml file is complete. From delong at redcort.com Wed Nov 2 08:25:31 2005 From: delong at redcort.com (Keith DeLong) Date: Wed Nov 2 10:26:00 2005 Subject: LoadDump files not selectable In-Reply-To: Message-ID: >>> On 10/25/05 9:38 PM, "Chris Sheffield" wrote: >>> >>>> That is not the case, Ruslan. I've had this same problem with VApp. >>>> It won't let you select any type of file. I've tried playing with >>>> the type/creator codes using ResEdit and had no luck. I've only been >>>> able to load a dump using Valentina Studio (but even with that, it's >>>> necessary to have another database open first, which I think must be >>>> a bug). >>> >>> I think it needs only set text/text. >> >> Vapp doesn't LoadDump with TEXT,TEXT text/TEXT TEXT,text or text/text... >> >> Sigh :-( >> > > Have you tried setting the creator to VALA and type to TEXT? The > creator should not matter, actually. I would check whether the xml > file is complete. Thanks for the suggestion. It appears there's something broken in 1.11 as I had no trouble using loaddump when I used vapp 1.10 with V4RB 1.10 data files. Ruslan has indicated that v1 support is dead so I finally just gave up on my user's corrupted data file. It's now time to figure out my options to support my installed user base in the coming months and years. Keith Delong From jordan at digi-net.com Wed Nov 2 12:21:18 2005 From: jordan at digi-net.com (Ken Jordan) Date: Wed Nov 2 11:19:37 2005 Subject: [V4RB] BLOB problems (additional info) In-Reply-To: <4367EC7A.8030002@digi-net.com> References: <4367EC7A.8030002@digi-net.com> Message-ID: <4368F58E.5090900@digi-net.com> Ken Jordan wrote: > Ruslan Zasukhin wrote: > >> On 11/1/05 10:58 PM, "Ken Jordan" wrote: >> >>>> Hmm, please try >>>> call file_table.CreateBLOBField( "file", 8 * 1024 ) >>>> >>>> Or 16 Kb >>>> >>>> So how file differ? It have the same size but differ internally? >>>> Or size also is other? >>>> >>>> Can you try other files? Smaller? E.g >>>> 20 Kb >>>> 100 Kb >>> >>> Works at 1Kb for 20 Kb and 160 Kb files, needs at least 4 Kb for 1.8 >>> MB file. >> >> So if set segment-size 4Kb then your 1.8mb file is written ok, right ? > > Right. > >> Interesting, I do not see reason why 1Kb do not work then. >> We need test this. > 5.5 MB file requires 16K segments. 16K segment size works for files up to 30MB (but not with files > 40MB) From sunshine at public.kherson.ua Wed Nov 2 19:32:27 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 2 11:32:50 2005 Subject: [V4RB] BLOB problems (additional info) In-Reply-To: <4368F58E.5090900@digi-net.com> Message-ID: On 11/2/05 7:21 PM, "Ken Jordan" wrote: > 5.5 MB file requires 16K segments. 16K segment size works for files up > to 30MB (but not with files > 40MB) Okay, but why you want store such big files ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jordan at digi-net.com Wed Nov 2 12:53:46 2005 From: jordan at digi-net.com (Ken Jordan) Date: Wed Nov 2 11:52:11 2005 Subject: [V4RB] BLOB problems (additional info) In-Reply-To: References: Message-ID: <4368FD2A.90706@digi-net.com> Ruslan Zasukhin wrote: >On 11/2/05 7:21 PM, "Ken Jordan" wrote: > > > >>5.5 MB file requires 16K segments. 16K segment size works for files up >>to 30MB (but not with files > 40MB) >> >> > >Okay, but why you want store such big files ? > > Gosh, I'm not sure anybody would want to. Just thought some additional info might be helpful if you were looking into why the 1kb segments weren't working. :-) From jordan at digi-net.com Wed Nov 2 16:57:31 2005 From: jordan at digi-net.com (Ken Jordan) Date: Wed Nov 2 15:55:55 2005 Subject: [V4RB] VText Problem Message-ID: <4369364B.5080008@digi-net.com> Similar to last (BLOB) problem. ---------------------------------------------------- dim f as FolderItem = GetFolderItem( "dbTest.vdb" ) if f.Exists then f.Delete() dim testDB as VDatabase = new VDatabase testDB.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) dim photo_table as VTable = testDB.CreateTable("ptable") call photo_table.CreateTextField( "pic", 1024 ) photo_table.SetBlank() f = GetFolderItem("goofy.jpg") dim p as Picture = f.OpenAsPicture() dim s1 As String = PNGu.GetData(p) photo_table.Field( "pic" ).SetString(s1) call photo_table.AddRecord() dim bRes as Boolean = photo_table.FirstRecord() dim s2 As String = photo_table.Field( "pic" ).GetString() if s1 <> s2 then msgbox "not the same" End If testDB.Close testDB = nil ---------------------------------------------------------- MessageBox displays "not the same" every time. I suspect that its an encoding problem, but I've tried every kind of encoding manipulation that I can think of but haven't been able to get a match. Any ideas?? TIA, Ken From sunshine at public.kherson.ua Thu Nov 3 09:40:04 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 3 01:40:33 2005 Subject: [V4RB] VText Problem In-Reply-To: <4369364B.5080008@digi-net.com> Message-ID: On 11/2/05 11:57 PM, "Ken Jordan" wrote: > Similar to last (BLOB) problem. > ---------------------------------------------------- > dim f as FolderItem = GetFolderItem( "dbTest.vdb" ) > if f.Exists then f.Delete() > > dim testDB as VDatabase = new VDatabase > testDB.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) > > dim photo_table as VTable = testDB.CreateTable("ptable") > call photo_table.CreateTextField( "pic", 1024 ) > > photo_table.SetBlank() > > f = GetFolderItem("goofy.jpg") > dim p as Picture = f.OpenAsPicture() > dim s1 As String = PNGu.GetData(p) > photo_table.Field( "pic" ).SetString(s1) > > call photo_table.AddRecord() > > dim bRes as Boolean = photo_table.FirstRecord() > dim s2 As String = photo_table.Field( "pic" ).GetString() > > if s1 <> s2 then > msgbox "not the same" > End If > > testDB.Close > testDB = nil > ---------------------------------------------------------- > MessageBox displays "not the same" every time. > I suspect that its an encoding problem, but I've tried every kind of > encoding manipulation that I can think of but haven't been able to get a > match. Any ideas?? Hi Ken, I do not understand. You have create TEXT Field, but store into it Picture ? This is wrong. Valentina have Vpicture field type for pictures. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jordan at digi-net.com Thu Nov 3 09:21:26 2005 From: jordan at digi-net.com (Ken Jordan) Date: Thu Nov 3 08:19:46 2005 Subject: [V4RB] VText Problem In-Reply-To: References: Message-ID: <436A1CE6.3070106@digi-net.com> Ruslan Zasukhin wrote: >On 11/2/05 11:57 PM, "Ken Jordan" wrote: > > > >>Similar to last (BLOB) problem. >>---------------------------------------------------- >> dim f as FolderItem = GetFolderItem( "dbTest.vdb" ) >> if f.Exists then f.Delete() >> >> dim testDB as VDatabase = new VDatabase >> testDB.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) >> >> dim photo_table as VTable = testDB.CreateTable("ptable") >> call photo_table.CreateTextField( "pic", 1024 ) >> >> photo_table.SetBlank() >> >> f = GetFolderItem("goofy.jpg") >> dim p as Picture = f.OpenAsPicture() >> dim s1 As String = PNGu.GetData(p) >> photo_table.Field( "pic" ).SetString(s1) >> >> call photo_table.AddRecord() >> >> dim bRes as Boolean = photo_table.FirstRecord() >> dim s2 As String = photo_table.Field( "pic" ).GetString() >> >> if s1 <> s2 then >> msgbox "not the same" >> End If >> >> testDB.Close >> testDB = nil >>---------------------------------------------------------- >>MessageBox displays "not the same" every time. >>I suspect that its an encoding problem, but I've tried every kind of >>encoding manipulation that I can think of but haven't been able to get a >>match. Any ideas?? >> >> > >Hi Ken, > >I do not understand. > >You have create TEXT Field, but store into it Picture ? >This is wrong. > >Valentina have Vpicture field type for pictures. > > Yes, I am aware of the VPicture field type. However, for several reasons it would be advantageous if I could store the pictures in string format. Is there no way to retrieve the string data in its original form? From cmsheffield at gmail.com Thu Nov 3 10:25:16 2005 From: cmsheffield at gmail.com (Chris Sheffield) Date: Thu Nov 3 11:25:41 2005 Subject: error -61 and error 32 Message-ID: Hi all, I'm using Valentina 1.x with Revolution. I have suddenly started getting error -61 (Mac OS X) and error 32 (Windows) when trying to perform a Flush on a local database. The same routine performed on a remote database does not cause the error. Can anyone think of why this might happen. I've been banging my head on my desk for hours now trying to figure this out. The thing that's weird is that the same code used to work. I'm not sure what, if anything, I've changed to make the error occur. All I'm doing is creating a read/write cursor, updating some fields, performing a flush (that's where it errors), and then closing the database. From what I can gather, error -61 is a "write permission" error for OS X, and error 32 is a sharing violation for Windows. Any takers? Thanks, Chris ------------------------------------------ Chris Sheffield Read Naturally The Fluency Company http://www.readnaturally.com ------------------------------------------ From delong at redcort.com Thu Nov 3 10:41:27 2005 From: delong at redcort.com (Keith DeLong) Date: Thu Nov 3 12:41:52 2005 Subject: error -61 and error 32 In-Reply-To: Message-ID: Hi Chris, You're attempting to open the database after it has already been opened. I'm not a revolution user but I can tell you that error -61 on OS X and error 32 on Windows occurs in Valentina (or any file system) when an already open file is being opened/accessed again by another process or application. On OS X, the file opens without error but throws the -61 error when an attempt to access the file is made. On Windows, error 32 occurs when the attempt is made to open the file. HTH, Keith DeLong > Hi all, > > I'm using Valentina 1.x with Revolution. I have suddenly started > getting error -61 (Mac OS X) and error 32 (Windows) when trying to > perform a Flush on a local database. The same routine performed on a > remote database does not cause the error. Can anyone think of why > this might happen. I've been banging my head on my desk for hours > now trying to figure this out. The thing that's weird is that the > same code used to work. I'm not sure what, if anything, I've changed > to make the error occur. All I'm doing is creating a read/write > cursor, updating some fields, performing a flush (that's where it > errors), and then closing the database. > > From what I can gather, error -61 is a "write permission" error for > OS X, and error 32 is a sharing violation for Windows. Any takers? > > Thanks, > Chris From sunshine at public.kherson.ua Thu Nov 3 20:48:29 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 3 12:48:54 2005 Subject: [V4RB] VText Problem In-Reply-To: <436A1CE6.3070106@digi-net.com> Message-ID: On 11/3/05 4:21 PM, "Ken Jordan" wrote: > Yes, I am aware of the VPicture field type. However, for several reasons > it would be advantageous if I could store the pictures in string format. > Is there no way to retrieve the string data in its original form? Of course. Because string is TERMINATED by ZERO. Binary data can have zero inside. You need to use BLOB or Picture -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Wed Nov 2 17:22:20 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 3 13:26:33 2005 Subject: [V4RB] VText Problem In-Reply-To: <4369364B.5080008@digi-net.com> References: <4369364B.5080008@digi-net.com> Message-ID: <2d2666f4c6393f73c3e2ae8406dd8dd1@desuetude.com> On Nov 2, 2005, at 4:57 PM, Ken Jordan wrote: > Similar to last (BLOB) problem. > ---------------------------------------------------- > dim f as FolderItem = GetFolderItem( "dbTest.vdb" ) > if f.Exists then f.Delete() > dim testDB as VDatabase = new VDatabase > testDB.Create(f, 1, 32 * 1024, EVOs.kOSDefault ) > dim photo_table as VTable = testDB.CreateTable("ptable") > call photo_table.CreateTextField( "pic", 1024 ) > photo_table.SetBlank() > f = GetFolderItem("goofy.jpg") > dim p as Picture = f.OpenAsPicture() > dim s1 As String = PNGu.GetData(p) > photo_table.Field( "pic" ).SetString(s1) > call photo_table.AddRecord() > dim bRes as Boolean = photo_table.FirstRecord() > dim s2 As String = photo_table.Field( "pic" ).GetString() > if s1 <> s2 then > msgbox "not the same" > End If > testDB.Close > testDB = nil > ---------------------------------------------------------- > MessageBox displays "not the same" every time. > I suspect that its an encoding problem, but I've tried every kind of > encoding manipulation that I can think of but haven't been able to get > a match. Any ideas?? You should be using StrComp to compare the strings; it will tell you whether they contain the same bytes, I think. My recollection (which may be from v1) is that GetString doesn't work for VBlob and maybe VText. Since pic is a VText field, you should retrieve the data using VText.Value, or VBlob.ReadData. -------------- Charles Yeomans From cmsheffield at gmail.com Thu Nov 3 12:27:11 2005 From: cmsheffield at gmail.com (Chris Sheffield) Date: Thu Nov 3 13:27:36 2005 Subject: error -61 and error 32 In-Reply-To: References: Message-ID: Thanks, Keith and Robert. You were both right. I figured that was probably the case, but I couldn't figure out why. I finally realized that I did have a place where I was opening the database, but not closing it, and then trying to open it again. Thus the error. Too bad it took me so long to track it down. Anyway, thanks again. Chris On Nov 3, 2005, at 11:41 AM, Keith DeLong wrote: > Hi Chris, > You're attempting to open the database after it has already been > opened. > > I'm not a revolution user but I can tell you that error -61 on OS X > and > error 32 on Windows occurs in Valentina (or any file system) when > an already > open file is being opened/accessed again by another process or > application. > > On OS X, the file opens without error but throws the -61 error when an > attempt to access the file is made. On Windows, error 32 occurs > when the > attempt is made to open the file. > > HTH, > > Keith DeLong > > >> Hi all, >> >> I'm using Valentina 1.x with Revolution. I have suddenly started >> getting error -61 (Mac OS X) and error 32 (Windows) when trying to >> perform a Flush on a local database. The same routine performed on a >> remote database does not cause the error. Can anyone think of why >> this might happen. I've been banging my head on my desk for hours >> now trying to figure this out. The thing that's weird is that the >> same code used to work. I'm not sure what, if anything, I've changed >> to make the error occur. All I'm doing is creating a read/write >> cursor, updating some fields, performing a flush (that's where it >> errors), and then closing the database. >> >> From what I can gather, error -61 is a "write permission" error for >> OS X, and error 32 is a sharing violation for Windows. Any takers? >> >> Thanks, >> Chris > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina ------------------------------------------ Chris Sheffield Read Naturally The Fluency Company http://www.readnaturally.com ------------------------------------------ From jordan at digi-net.com Thu Nov 3 15:08:17 2005 From: jordan at digi-net.com (Ken Jordan) Date: Thu Nov 3 14:06:34 2005 Subject: [V4RB] VText Problem In-Reply-To: References: Message-ID: <436A6E31.3040009@digi-net.com> Ruslan Zasukhin wrote: >On 11/3/05 4:21 PM, "Ken Jordan" wrote: > > > >>Yes, I am aware of the VPicture field type. However, for several reasons >>it would be advantageous if I could store the pictures in string format. >>Is there no way to retrieve the string data in its original form? >> >> > >Of course. Because string is TERMINATED by ZERO. > >Binary data can have zero inside. > > Good point! (Although I was receiving a String of the same length, just with vastly different 'characters') >You need to use BLOB or Picture > > Changing to BLOB field and using WriteRawData And ReadRawData gives correct results. Thanks Ruslan and Charles, Ken From sunshine at public.kherson.ua Thu Nov 3 22:35:38 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 3 14:36:04 2005 Subject: [V4RB] VText Problem In-Reply-To: <2d2666f4c6393f73c3e2ae8406dd8dd1@desuetude.com> Message-ID: On 11/3/05 12:22 AM, "Charles Yeomans" wrote: Hi Charles, > You should be using StrComp to compare the strings; it will tell you > whether they contain the same bytes, I think. My recollection (which > may be from v1) is that GetString doesn't work for VBlob and maybe > VText. For VText it works -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Thu Nov 3 16:29:41 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 3 15:30:07 2005 Subject: [V4RB] VText Problem In-Reply-To: References: Message-ID: <1826c71fe9c3526eba2af67ef52f2965@desuetude.com> On Nov 3, 2005, at 3:35 PM, Ruslan Zasukhin wrote: > On 11/3/05 12:22 AM, "Charles Yeomans" wrote: > > Hi Charles, > >> You should be using StrComp to compare the strings; it will tell you >> whether they contain the same bytes, I think. My recollection (which >> may be from v1) is that GetString doesn't work for VBlob and maybe >> VText. > > For VText it works It should work for VBlob as well; isn't VBlob a subclass of VField? -------------- Charles Yeomans From sunshine at public.kherson.ua Fri Nov 4 08:20:59 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 00:21:27 2005 Subject: [V4RB] VText Problem In-Reply-To: <1826c71fe9c3526eba2af67ef52f2965@desuetude.com> Message-ID: On 11/3/05 11:29 PM, "Charles Yeomans" wrote: >> For VText it works > > It should work for VBlob as well; isn't VBlob a subclass of VField? It can be called. Actually yes, REALstring is able keep binary data. But in general this is not correct. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Fri Nov 4 13:12:38 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 4 12:09:16 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes Message-ID: <436BA496.7030503@mediamacros.com> Every time I try to inspect a file to see what version it is Director crashes hard (fatal exception, etc) Valentina = xtra("Valentina").new() Valentina.getDatabaseVersion("G:\file.vdb") BAM - Crash. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer From sunshine at public.kherson.ua Fri Nov 4 20:59:47 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 13:00:15 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BA496.7030503@mediamacros.com> Message-ID: On 11/4/05 8:12 PM, "Chuck Neal" wrote: > Every time I try to inspect a file to see what version it is Director > crashes hard (fatal exception, etc) > > Valentina = xtra("Valentina").new() > Valentina.getDatabaseVersion("G:\file.vdb") > > BAM - Crash. Hi Chuck, I believe that we have such example. Folder Common. Please check it and let us know. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From doppler at derlichtblick.at Fri Nov 4 20:00:46 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 13:01:18 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes Message-ID: <436BAFDE.9010008@derlichtblick.at> i updated (run your installer and copy the new xtra and components in the xtrafolder of dir and all projects) on os x from 203 to 205 becouse i hope to fix a problem with german characters. now director crashes by gDatabase = new (Xtra "VDatabase") i'm very unhappy, on 203 it works not perfect(see special characters) but it starts. what is this? gr thomas -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From doppler at derlichtblick.at Fri Nov 4 20:24:15 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 13:24:42 2005 Subject: test Message-ID: <436BB55F.3010702@derlichtblick.at> -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From chuck at mediamacros.com Fri Nov 4 14:44:46 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 4 13:42:38 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: References: Message-ID: <436BBA2E.9010902@mediamacros.com> I don't think I understand what you are asking/saying. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > Hi Chuck, > > I believe that we have such example. > Folder Common. > > Please check it and let us know. > > From sunshine at public.kherson.ua Fri Nov 4 21:49:43 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 13:50:11 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BBA2E.9010902@mediamacros.com> Message-ID: On 11/4/05 9:44 PM, "Chuck Neal" wrote: > I don't think I understand what you are asking/saying. To check our example which use this method. It seems this is example Db Convert_1_2 It check version of db file before do conversion, If example works fine, then you can compare your code with example. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 4 21:51:10 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 13:51:36 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: <436BAFDE.9010008@derlichtblick.at> Message-ID: On 11/4/05 9:00 PM, "thomas doppler" wrote: > i updated (run your installer and copy the new xtra and components in > the xtrafolder of dir and all projects) on os x from 203 to 205 becouse > i hope to fix a problem with german characters. > now director crashes by > > gDatabase = new (Xtra "VDatabase") > > i'm very unhappy, on 203 it works not perfect(see special characters) > but it starts. > what is this? Try again trash VComponents folder and V4MD_2 to make sure you have clean install. Then install new. You can see in list archive that few developers have confirm that 2.0.5 MAC works for them -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Fri Nov 4 15:06:47 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Fri Nov 4 14:07:14 2005 Subject: [V4RB] VText Problem In-Reply-To: References: Message-ID: <4d434b096ffc26e55cc4dcf6d4eabad9@desuetude.com> On Nov 4, 2005, at 1:20 AM, Ruslan Zasukhin wrote: > On 11/3/05 11:29 PM, "Charles Yeomans" wrote: > >>> For VText it works >> >> It should work for VBlob as well; isn't VBlob a subclass of VField? > > It can be called. > > Actually yes, REALstring is able keep binary data. > > But in general this is not correct. What's not correct? It seems to me that GetString should do more or less the same thing as ReadData; the only question is how text encoding should be handled. -------------- Charles Yeomans From doppler at derlichtblick.at Fri Nov 4 21:19:05 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 14:19:34 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: References: Message-ID: <436BC239.2060901@derlichtblick.at> i did a clean install and it did not work i try on 2 machines with the same result, (systems 10.3.3 and 10.3.4) and i try your example from the installfolder i attach you the systemreport, maybe you can see something gr th Ruslan Zasukhin wrote: > On 11/4/05 9:00 PM, "thomas doppler" wrote: > > >> i updated (run your installer and copy the new xtra and components in >> the xtrafolder of dir and all projects) on os x from 203 to 205 becouse >> i hope to fix a problem with german characters. >> now director crashes by >> >> gDatabase = new (Xtra "VDatabase") >> >> i'm very unhappy, on 203 it works not perfect(see special characters) >> but it starts. >> what is this? >> > > Try again trash VComponents folder and V4MD_2 to make sure you have clean > install. Then install new. > > > You can see in list archive that few developers have confirm that 2.0.5 MAC > works for them > > -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From sunshine at public.kherson.ua Fri Nov 4 22:26:52 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 14:27:20 2005 Subject: [V4RB] VText Problem In-Reply-To: <4d434b096ffc26e55cc4dcf6d4eabad9@desuetude.com> Message-ID: On 11/4/05 10:06 PM, "Charles Yeomans" wrote: >>>> For VText it works >>> >>> It should work for VBlob as well; isn't VBlob a subclass of VField? >> >> It can be called. >> >> Actually yes, REALstring is able keep binary data. >> >> But in general this is not correct. > > What's not correct? It seems to me that GetString should do more or > less the same thing as ReadData; the only question is how text encoding > should be handled. I mean Charles all products, e.g. V4MD, V4REV. They do not handle binary data in strings. So generally speaking this is not correct read BLOB data as strigs -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 4 22:27:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 14:27:57 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: <436BC239.2060901@derlichtblick.at> Message-ID: On 11/4/05 10:19 PM, "thomas doppler" wrote: > i did a clean install and it did not work > i try on 2 machines with the same result, (systems 10.3.3 and 10.3.4) > and i try your example from the installfolder > i attach you the systemreport, maybe you can see something Strange, Can you try then 2.1 b2 build ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Fri Nov 4 15:41:58 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 4 14:38:47 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: References: Message-ID: <436BC796.7030800@mediamacros.com> I found the problem. You have to call init before that command. Where is there info on what the numbers mean? IE I get back 400 but nothing tells me how to translate that into anything useful. Also the docs use... getDatabaseFormatVersion() \But the Xtra says (and uses) getDatabaseVersion() -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/4/05 9:44 PM, "Chuck Neal" wrote: > > >>I don't think I understand what you are asking/saying. > > > To check our example which use this method. > > It seems this is example Db Convert_1_2 > It check version of db file before do conversion, > > If example works fine, then you can compare your code with example. > From sunshine at public.kherson.ua Fri Nov 4 22:49:07 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 14:49:34 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BC796.7030800@mediamacros.com> Message-ID: On 11/4/05 10:41 PM, "Chuck Neal" wrote: Hi Chuck, > I found the problem. You have to call init before that command. Ah, this is logical. ValentinaInit() must be first call to Valentina Absolutely first. Because it loads dlls and init ICU -- unicode library. > Where is > there info on what the numbers mean? IE I get back 400 but nothing > tells me how to translate that into anything useful. Also the docs use... Run Apple calculator, switch to Programmer mode. Convert 400 decimal to HEX. You will see 190, this is version 1.9 of format. > getDatabaseFormatVersion() > \But the Xtra says (and uses) > getDatabaseVersion() This is changed for 2.1. New name is more informative. You use 2.0.5 I think right? So you do not see yet this change Btw, you work on OS X ? Thomas, cannot get working 2.0.5 on some reason ... -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From doppler at derlichtblick.at Fri Nov 4 21:51:05 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 14:51:31 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: References: Message-ID: <436BC9B9.1000404@derlichtblick.at> yes it works, is the beta stabil enough for production? i cannot wait for the final, my deadline for the project is next week. gr th Ruslan Zasukhin wrote: > On 11/4/05 10:19 PM, "thomas doppler" wrote: > > >> i did a clean install and it did not work >> i try on 2 machines with the same result, (systems 10.3.3 and 10.3.4) >> and i try your example from the installfolder >> i attach you the systemreport, maybe you can see something >> > > Strange, > > Can you try then 2.1 b2 build ? > > -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From chuck at mediamacros.com Fri Nov 4 16:23:17 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 4 15:19:49 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: References: Message-ID: <436BD145.2090809@mediamacros.com> Working on Windows for now but I may end up porting this to Mac as well. Is there any way to keep the pointer to the old syntax with 2.1? The one thing that drives me nuts between versions is when command names change and are not backwards compatible. :) As for the version, any possibility of a helper script to more easily convert that? My user's may not want to have to run a calculator to determine if I need to update their database. ;) -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/4/05 10:41 PM, "Chuck Neal" wrote: > > Hi Chuck, > > >>I found the problem. You have to call init before that command. > > > Ah, this is logical. > > ValentinaInit() must be first call to Valentina > > Absolutely first. Because it loads dlls and init ICU -- unicode library. > > > >>Where is >>there info on what the numbers mean? IE I get back 400 but nothing >>tells me how to translate that into anything useful. Also the docs use... > > > Run Apple calculator, switch to Programmer mode. > > Convert 400 decimal to HEX. You will see 190, this is version 1.9 of format. > > > >>getDatabaseFormatVersion() >>\But the Xtra says (and uses) >>getDatabaseVersion() > > > This is changed for 2.1. > > New name is more informative. > You use 2.0.5 I think right? So you do not see yet this change > > > Btw, you work on OS X ? > Thomas, cannot get working 2.0.5 on some reason ... > > From sunshine at public.kherson.ua Fri Nov 4 23:37:20 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 15:37:47 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BD145.2090809@mediamacros.com> Message-ID: On 11/4/05 11:23 PM, "Chuck Neal" wrote: > Working on Windows for now but I may end up porting this to Mac as well. > > Is there any way to keep the pointer to the old syntax with 2.1? The > one thing that drives me nuts between versions is when command names > change and are not backwards compatible. :) I know, this is very rare happens > As for the version, any possibility of a helper script to more easily > convert that? My user's may not want to have to run a calculator to > determine if I need to update their database. ;) But Lingo can show numbers in HEX ? C++, REALbasic, and others can easy I think Lingo also should And I think users should see nothing except dialog: app is in 1.x format. Do you want convert it to 2.0 ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 4 23:38:50 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 15:39:18 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: <436BC9B9.1000404@derlichtblick.at> Message-ID: On 11/4/05 10:51 PM, "thomas doppler" wrote: > yes it works, > is the beta stabil enough for production? > i cannot wait for the final, my deadline for the project is next week. Yes. All our 25,000 tests are DONE. But please test it of course. Btw, what problems you did have in 2.0.3 ? 2 days ago Ivan have fix issue with SQL functions upper() / lower() They did not use locale info. Now do. This is not in b2. Will go into b3 -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Fri Nov 4 16:54:42 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 4 15:51:15 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: References: Message-ID: <436BD8A2.1040504@mediamacros.com> > But Lingo can show numbers in HEX ? Not directly. I can probably use an Xtra to convert but its not built in. I'll search for a script to help convert... -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer From doppler at derlichtblick.at Fri Nov 4 23:29:13 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 16:29:42 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: References: Message-ID: <436BE0B9.2030102@derlichtblick.at> with problem in 203 i mean that special german characters (?,?,...) are not stored in the db. so they are not shown after read out. with the b2 it is ok thanks th Ruslan Zasukhin wrote: > On 11/4/05 10:51 PM, "thomas doppler" wrote: > > >> yes it works, >> is the beta stabil enough for production? >> i cannot wait for the final, my deadline for the project is next week. >> > > Yes. All our 25,000 tests are DONE. > > But please test it of course. > > > Btw, what problems you did have in 2.0.3 ? > > 2 days ago Ivan have fix issue with SQL functions upper() / lower() > They did not use locale info. Now do. > > This is not in b2. Will go into b3 > > > -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From doppler at derlichtblick.at Fri Nov 4 23:33:54 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Fri Nov 4 16:34:21 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BD8A2.1040504@mediamacros.com> References: <436BD8A2.1040504@mediamacros.com> Message-ID: <436BE1D2.2080405@derlichtblick.at> http://www.mcli.dist.maricopa.edu/director/tips/lingo/hex2dec.html gr th Chuck Neal wrote: > > But Lingo can show numbers in HEX ? > > Not directly. I can probably use an Xtra to convert but its not built > in. I'll search for a script to help convert... > > > -Chuck > -------------------------- > Chuck Neal > CEO, MediaMacros, Inc. > chuck@mediamacros.com > http://www.mediamacros.com > -------------------------- > Check out the Developers Mall > Your one stop shop for all your Director Xtra Needs > http://www.mediamacros.net/customer > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From sunshine at public.kherson.ua Sat Nov 5 02:06:50 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 4 18:07:17 2005 Subject: V4MD update 2.0.3 to 2.0.5 crashes In-Reply-To: <436BE0B9.2030102@derlichtblick.at> Message-ID: On 11/5/05 12:29 AM, "thomas doppler" wrote: > with problem in 203 i mean that special german characters (?,?,...) are > not stored in the db. > so they are not shown after read out. > with the b2 it is ok Great! -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From lfredricks at proactive-intl.com Sat Nov 5 10:14:57 2005 From: lfredricks at proactive-intl.com (Lynn Fredricks) Date: Sat Nov 5 12:15:32 2005 Subject: Submit your Products for a Powered by Valentina Gallery Message-ID: <20051105181458.62A4717D036@spatula.dreamhost.com> Hello all, Paradigma Software would like to show off Powered by Valentina products in a site gallery. Such a gallery would have information about your product, plus, a paragraph about your company. This not only promotes the usage of Valentina, but also aids in driving business to you. Such a gallery would be divided up into platforms, such as Director, REALbasic, etc. What do you think about this? Shortly we will set up a form for this. Best regards, Lynn Fredricks President Paradigma Software, Inc Joining Worlds of Information Deploy True Client-Server Database Solutions Royalty Free with Valentina Developer Network http://www.paradigmasoft.com From maxprog at mac.com Sat Nov 5 19:37:54 2005 From: maxprog at mac.com (Stan Busk) Date: Sat Nov 5 12:38:28 2005 Subject: Submit your Products for a Powered by Valentina Gallery In-Reply-To: <20051105181458.62A4717D036@spatula.dreamhost.com> References: <20051105181458.62A4717D036@spatula.dreamhost.com> Message-ID: Really cool!... let us know as soon the form is ready. > Hello all, > > Paradigma Software would like to show off Powered by Valentina > products in a > site gallery. Such a gallery would have information about your > product, > plus, a paragraph about your company. This not only promotes the > usage of > Valentina, but also aids in driving business to you. > > Such a gallery would be divided up into platforms, such as Director, > REALbasic, etc. > > What do you think about this? Shortly we will set up a form for this. > > Best regards, > > Lynn Fredricks > President > Paradigma Software, Inc > > Joining Worlds of Information > > Deploy True Client-Server Database Solutions > Royalty Free with Valentina Developer Network > http://www.paradigmasoft.com > > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina -- ~/Stan Maxprog? From sunshine at public.kherson.ua Sun Nov 6 10:30:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 6 02:31:03 2005 Subject: Number of Records versus Record size In-Reply-To: <258f809eeb04ea4e8cd363d5c155f2b1@shaw.ca> Message-ID: On 11/6/05 12:56 AM, "Charles Cohen" wrote: Hi Charles, > On 5-Nov-05, at 8:17 AM, Ruslan Zasukhin wrote: > > Because Varchar use less disk space. > > String[500] will use 500 bytes ALWAYS, even if you store there 10 chars. > > Back again... > > After our conversation concluding that VarChars are a more efficient > storing method....I adapted two of the easiert tables first... > > On one table it should not have changed the size much as there was only > one field whose data would vary in length....but I switched it to > VVarChar(9)...the longest string that would be written. Look. For SHORT strings, 2-20 bytes, Vstring[N] is the best. For strings as URLs, when can be 200 chars or 10 chars, better to use varChar. > The second table contains 8 feilds.....given all these fields can > contain data of varying length...I switched them all....one was a > vstring field was set to 100...and there are 102 records...seemd like > it should make some difference..it did.... > > I apologize in advance for the conclusion... the database I began with > was 2.4 megs...it is now 3.2 megs..:-(. > I have not done the table we spoke of before.....the one with the two > vstring(500) fields... > > Given the database is growing all by itself (I have not altered the > data today)....I fear that if I adapt the remaining table to use > VarChars instead of vstring... I may fall off of my chair. Hmm. Then question is: What size of EMPTY db? If after switch to VarChar it have become 3.2 MB, I think this is okay, because when you will add MANY records, you will see win comparing to String[N]. Again, Let you make String[500] and VarChar[500] LEt you store there 1000 strings of 20 bytes. String[500] in fact eat 1000 * 500 = 500,000 bytes Varchar[500] eat (20 + 8) * 1000 = 28,000 > It took some difficulty in creating new fields...before removing the > old one's...I could not edit to my specs inside Studio (2.0Vb16) ie: > change the length of the varchar fields using the property editor (they > were set to 20).... Yes I have see this problem in Vstudio. Must be fixed. > I created new fields...wrote the old information to > them...after using studio to remove the old fields...I then I used > first studio and then my app to defragment and compact before I checked > the sizes. Hopefully this is not the reason that it grew. Did not you note if compact really have reduce size ? May be try do XML dump and load. Then you will get really clean and fresh db > So, any thoughts on whether I should continue to adapt my vstrings to > varchars? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From rwc1717 at shaw.ca Sun Nov 6 17:42:11 2005 From: rwc1717 at shaw.ca (Charles Cohen) Date: Sun Nov 6 23:40:28 2005 Subject: Ruslan's Rules In-Reply-To: References: Message-ID: On 6-Nov-05, at 12:30 AM, Ruslan Zasukhin wrote: >>Look. For SHORT strings, 2-20 bytes, Vstring[N] is the best. >>For strings as URLs, when can be 200 chars or 10 chars, better to use varChar. New & empty created with VStrings = 1.4 megs New & empty created with VVarChar = 1.7 megs Finished adapting to use the most efficient fields...tweaking can be rewarding.... Full created with VStrings = 2.7 megs Full created with VVarChar = 2.0 megs Needed to do an xml dump and load to accomplish this.... By the time I had done all tha adapting...the db had grown to 4.0 megs...defragmenting/compacting at this time corrupted the database....it was unusable. I will watch and after a month or I can compact and report back with the results....the defragmenting/compacting doesn't ruin the newly restored db. Thanks for Ruslan's rules.... Rik, Harry, Wendy, & Chuck From chuck at mediamacros.com Mon Nov 7 08:18:07 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 7 07:14:42 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436BE1D2.2080405@derlichtblick.at> References: <436BD8A2.1040504@mediamacros.com> <436BE1D2.2080405@derlichtblick.at> Message-ID: <436F540F.4000403@mediamacros.com> Is that the correct type of "hex"? If I run "400" on this I get 1024, which is not the correct value. I think this is a binary hex to integer. Is the code different for this? -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer thomas doppler wrote: > http://www.mcli.dist.maricopa.edu/director/tips/lingo/hex2dec.html > > gr th > > Chuck Neal wrote: > >> > But Lingo can show numbers in HEX ? >> >> Not directly. I can probably use an Xtra to convert but its not built >> in. I'll search for a script to help convert... >> >> >> -Chuck >> -------------------------- >> Chuck Neal >> CEO, MediaMacros, Inc. >> chuck@mediamacros.com >> http://www.mediamacros.com >> -------------------------- >> Check out the Developers Mall >> Your one stop shop for all your Director Xtra Needs >> http://www.mediamacros.net/customer >> >> >> _______________________________________________ >> Valentina mailing list >> Valentina@lists.macserve.net >> http://lists.macserve.net/mailman/listinfo/valentina > > From sunshine at public.kherson.ua Mon Nov 7 17:43:44 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 7 09:44:19 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: <436F540F.4000403@mediamacros.com> Message-ID: On 11/7/05 3:18 PM, "Chuck Neal" wrote: > Is that the correct type of "hex"? If I run "400" on this I get 1024, > which is not the correct value. I think this is a binary hex to > integer. Is the code different for this? It looks that you do 400 hex into 1024 decimal Must be 400 decimal into HEX (190) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Mon Nov 7 12:19:35 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 7 11:16:30 2005 Subject: V4MD 2.0.5 getDatabaseVersion crashes In-Reply-To: References: Message-ID: <436F8CA7.1000109@mediamacros.com> Ah. Now I feel silly. Thanks. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/7/05 3:18 PM, "Chuck Neal" wrote: > > >>Is that the correct type of "hex"? If I run "400" on this I get 1024, >>which is not the correct value. I think this is a binary hex to >>integer. Is the code different for this? > > > It looks that you do > > 400 hex into 1024 decimal > > > Must be > > 400 decimal into HEX (190) > > > From doppler at derlichtblick.at Mon Nov 7 21:34:58 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Mon Nov 7 14:35:32 2005 Subject: V4MD 2.0.5 crashes In-Reply-To: References: Message-ID: <436FBA72.4050100@derlichtblick.at> hi ruslan, for some days i told valentina 205 crashs but 2.1b2 works. this is wrong i worked for the last days on win, today i try my works on osx and its the same shit. 205 and 2.1b2 crashs, now without a bugreport of the system. everytime i want to initialize the xtra it crashs. this is very bad becouse 203 (the last version i have cannot store german characters like ?,?,?...) gr th -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From softil at onlinehome.de Tue Nov 8 00:03:36 2005 From: softil at onlinehome.de (SoftIl) Date: Mon Nov 7 17:04:12 2005 Subject: where-statement for a date column Message-ID: <000601c5e3ef$7dc57a70$16b2a8c0@ipsbasis> Hi, how is the rigth syntax for a where-statement for an date column? I tried this: select * from myTable where dateCol like '\A04.10.2004' but it dosen't work. I get no records return. If I select the records without the where-statement I get all records return. In the column "dateCol" are this values for example: 04.10.2004 00:00:00 15.05.2004 00:00:00 Thanks for your help, Carsten From sunshine at public.kherson.ua Tue Nov 8 07:51:23 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 7 23:52:02 2005 Subject: V4MD 2.0.5 crashes In-Reply-To: <436FBA72.4050100@derlichtblick.at> Message-ID: On 11/7/05 10:34 PM, "thomas doppler" wrote: > hi ruslan, > > for some days i told valentina 205 crashs but 2.1b2 works. > this is wrong > i worked for the last days on win, today i try my works on osx and its > the same shit. > 205 and 2.1b2 crashs, now without a bugreport of the system. > everytime i want to initialize the xtra it crashs. > > this is very bad becouse 203 (the last version i have cannot store > german characters like ?,?,?...) Hi Thomas, Please send me any crash report you have -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 8 07:53:07 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 7 23:53:43 2005 Subject: where-statement for a date column In-Reply-To: <000601c5e3ef$7dc57a70$16b2a8c0@ipsbasis> Message-ID: On 11/8/05 1:03 AM, "SoftIl" wrote: Hi Paolo, > Hi, > how is the rigth syntax for a where-statement for an date column? > > I tried this: > > select * from myTable where dateCol like '\A04.10.2004' It looks you do REGEX search, but use LIKE. > but it dosen't work. I get no records return. > > If I select the records without the where-statement I get all records return. > In the column "dateCol" are this values for example: > > 04.10.2004 00:00:00 > 15.05.2004 00:00:00 -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From softil at onlinehome.de Tue Nov 8 08:16:16 2005 From: softil at onlinehome.de (SoftIl) Date: Tue Nov 8 01:16:53 2005 Subject: where-statement for a date column References: Message-ID: <001a01c5e434$509dac30$16b2a8c0@ipsbasis> Hi, > It looks you do REGEX search, but use LIKE. Why do you think I use REGEX for search? This syntax (like '\A04.10.2004') is from the SQL-Reference Document: How looks a right where-statement for search an value in an column from type date? Best regards, Carsten From doppler at derlichtblick.at Tue Nov 8 09:01:07 2005 From: doppler at derlichtblick.at (thomas doppler) Date: Tue Nov 8 02:01:45 2005 Subject: V4MD 2.0.5 crashes In-Reply-To: References: Message-ID: <43705B43.6030303@derlichtblick.at> hi i attached the report of the 205-crash, 2.1b2 quitts witrhout any message thanks th Ruslan Zasukhin wrote: >On 11/7/05 10:34 PM, "thomas doppler" wrote: > > > >>hi ruslan, >> >>for some days i told valentina 205 crashs but 2.1b2 works. >>this is wrong >>i worked for the last days on win, today i try my works on osx and its >>the same shit. >>205 and 2.1b2 crashs, now without a bugreport of the system. >>everytime i want to initialize the xtra it crashs. >> >>this is very bad becouse 203 (the last version i have cannot store >>german characters like ?,?,?...) >> >> > >Hi Thomas, > >Please send me any crash report you have > > > -- LichtBlick-L?sungen aus einer Hand ------------------------------------------ http://derlichtblick.at meillergasse 5/3 a-1130 wien ------------------------------------------ tel 00 43 1 802 50 32 mobil 00 43 664 243 96 51 mail doppler@derlichtblick.at From sunshine at public.kherson.ua Tue Nov 8 10:21:45 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 8 02:22:23 2005 Subject: V4MD 2.0.5 crashes In-Reply-To: <43705B43.6030303@derlichtblick.at> Message-ID: On 11/8/05 10:01 AM, "thomas doppler" wrote: > hi > > i attached the report of the 205-crash, 2.1b2 quitts witrhout any message You cannot sent attachments to list Send them to me directly -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 8 10:22:29 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 8 02:23:07 2005 Subject: where-statement for a date column In-Reply-To: <001a01c5e434$509dac30$16b2a8c0@ipsbasis> Message-ID: On 11/8/05 9:16 AM, "SoftIl" wrote: > Hi, >> It looks you do REGEX search, but use LIKE. > Why do you think I use REGEX for search? > This syntax (like '\A04.10.2004') is from the SQL-Reference Document: > > How looks a right where-statement for search an value in an column from type > date? \A this is REGEX anchor For LIKE you should use WHERE f LIKE '%04.10.2004' -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 8 10:35:12 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 8 02:35:50 2005 Subject: crashreport In-Reply-To: <437060BE.60809@derlichtblick.at> Message-ID: On 11/8/05 10:24 AM, "thomas doppler" wrote: Hi Thomas, > thanks You use SO X 10.1 ? I believe that Valentina expect 10.2 at least. Do not use 10.1. Many people agree that 10.1 was in fact like an alpha. Please try MAC with 10.2 or better. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From softil at onlinehome.de Tue Nov 8 09:53:28 2005 From: softil at onlinehome.de (SoftIl) Date: Tue Nov 8 02:54:05 2005 Subject: where-statement for a date column References: Message-ID: <003b01c5e441$e56b7880$16b2a8c0@ipsbasis> > For LIKE you should use > > WHERE f LIKE '%04.10.2004' Ok, thanks a lot. Best regards, Carsten From daisychain at iinet.net.au Thu Nov 10 11:47:33 2005 From: daisychain at iinet.net.au (Damon Pillinger) Date: Wed Nov 9 18:48:45 2005 Subject: No class with this name Message-ID: <001a01c5e590$567b4e20$6500a8c0@daisycha13caa1> Hi Guys, I am getting an error "No class with this name" for the declaration "valdb as VRBDataBase" It happened when I installed 2.1b1 WIN but I have reinstalled 2.05 final win and still get the same error. Has the name of VRBDatabase changed? Thanks Best regards Damon L. Pillinger Director Daisychain EDi Suite 3 4 Winston Court Moorabbin 3189 Phone : +61 3 9532 1220 Fax : +61 3 9532 1280 From sunshine at public.kherson.ua Thu Nov 10 08:42:07 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 10 00:42:46 2005 Subject: No class with this name In-Reply-To: <001a01c5e590$567b4e20$6500a8c0@daisycha13caa1> Message-ID: On 11/10/05 2:47 AM, "Damon Pillinger" wrote: Hi Damon, > I am getting an error "No class with this name" for the declaration > "valdb as VRBDataBase" > > It happened when I installed 2.1b1 WIN but I have reinstalled 2.05 final > win and still get the same error. > > Has the name of VRBDatabase changed? No, let me check code. All looks correct. All other classes of Valentina work? Do you use RB Standard or PRO ? Only RB PRO can use that class -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From daisychain at iinet.net.au Fri Nov 11 08:01:48 2005 From: daisychain at iinet.net.au (Damon Pillinger) Date: Thu Nov 10 15:02:30 2005 Subject: No class with this name In-Reply-To: Message-ID: <001501c5e639$f7d89da0$6500a8c0@daisycha13caa1> Hi Ruslan, I have 5.5.5 Pro and after your email I checked and none of the Valentina classes work even a simple .init and shutdown. So rebooted again and same problem. So went home and shut down computer, come in this morning, re booted and all is working. Very weird??? Must be gremlins in the Computer. Thanks anyway as always. Best regards Damon L. Pillinger Director Daisychain EDi Suite 3 4 Winston Court Moorabbin 3189 Phone : +61 3 9532 1220 Fax : +61 3 9532 1280 -----Original Message----- From: valentina-bounces@lists.macserve.net [mailto:valentina-bounces@lists.macserve.net] On Behalf Of Ruslan Zasukhin Sent: Thursday, 10 November 2005 5:42 PM To: valentina@lists.macserve.net Subject: Re: No class with this name On 11/10/05 2:47 AM, "Damon Pillinger" wrote: Hi Damon, > I am getting an error "No class with this name" for the declaration > "valdb as VRBDataBase" > > It happened when I installed 2.1b1 WIN but I have reinstalled 2.05 final > win and still get the same error. > > Has the name of VRBDatabase changed? No, let me check code. All looks correct. All other classes of Valentina work? Do you use RB Standard or PRO ? Only RB PRO can use that class -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] _______________________________________________ Valentina mailing list Valentina@lists.macserve.net http://lists.macserve.net/mailman/listinfo/valentina From macsforever2000 at goodeast.com Fri Nov 11 13:10:42 2005 From: macsforever2000 at goodeast.com (Frank Schima) Date: Fri Nov 11 14:11:31 2005 Subject: Test Message-ID: <063B4045-3889-4EA2-90B2-50DDF4F99749@goodeast.com> Testing. -Frank Schima From ruslan at paradigmasoft.com Sat Nov 12 08:15:24 2005 From: ruslan at paradigmasoft.com (Ruslan Zasukhin) Date: Sat Nov 12 00:17:42 2005 Subject: [V4REV + VServer] Valentina - problems In-Reply-To: <2B34593F-E2B0-452A-8583-3654274F125F@bigred.unl.edu> Message-ID: On 11/12/05 1:33 AM, "Suhasini Kotcherlakota" wrote: Hi Suhasini, > Ruslan: > > Thanks for your quick reply. I see the serial numbers and other files. I > downloaded Embedded valentina server as well as valentina for revolution. > I'm unsuccessful in connecting, creating database etc., > > From the Zip File when we purchased I have got the following folders: > ? > Additional 10 Connection > ADK serial.txt > VServer Embedded > Embedded Serial.txt > license_emb_...numbers > VServer Office right > --- > My file structure looks like this: > Library > StartupItems > VServer_Embedded > StartupParameters.plist > VServer_Embedded > VServer_Embedded > Databases > licenses > license_emb_....numbers I have correct intents. In your letter they was broken. Yes looks right > (I replaced this with the emb license file you gave us > when we purchased present in the Vserver Embedded folder. There is also > Embedded Serial.txt which I don't know where I have to put?) This can be a problem !!! IF you have start to use license_emb, then you MUST specify embedded serial, which is key of your application to connect to server. You should specify that big hex string from file Embedded Serial.txt In the parameter Options of Vdatabase constructor Vdatabase_New( host, user, passw, port, timeout, Options ) ^^^^^^^^^^ Having this you should be able connect. ------- BTW, During development you can use not license_emb, But license_mac_xxxx file. In this case server will work without Option parameter. Keep in mind that license_mac and license_win should be used only in your development, and not distributed to your customers. Why this is better? Because then you will be able run and check our examples, which of course DO NOT have parameter Option. > VServer_Embedded > VServer_Embedded.ini ( What should the SystmCatalog contain? Where and how > should I put the pathname, hostname,username and password? ) > VServer_Embedded.log usually SystmCatalog = "Databases" Host, User, Password you should specify in the constructor of DB. Vdatabase_New( host, user, passw, port, timeout, Options ) ---------------------------------------------- Please do next steps: * at first you can try run V4REV examples without server. make sure that radio button is LOCAL. works ? * then drop license_mac into Vserver. * using Terminal (top) or ActivityMonitor check that Vserver is not running. btw, please read Vserver.pdf how to run, stop Vserver. * in the .ini file set VerboseLevel = 3 * Now you should learn how to run Vserver from terminal. Then you will see in live mode all messages of server. * when you have server running in terminal, then you can try V4REV examples to work with server. For this just check radio button CLIENT and do examples. Works ? And only after this you can start make own project. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Claudius at sailer-online.de Sat Nov 12 21:58:40 2005 From: Claudius at sailer-online.de (Claudius Sailer) Date: Sat Nov 12 14:59:28 2005 Subject: FieldType Message-ID: <8826FD6D-6404-4697-97F2-3A28580B341E@sailer-online.de> Hi, with V4RB1 I read FieldType with following code Dim FieldType as string FieldType=tmpDB.Table(i).Field(j).Type and have a function which tells me format select case type case 2 return "Boolean" case 3 return "Byte" case 4 return "Short" case 5 return "UShort" case 6 return "Long" case 7 return "ULong" case 8 return "Float" case 9 return "Double" case 10 return "String" case 11 return "Unknown" case 12 return "Date" case 13 return "Time" case 14 return "DateTime" case 15 return "ObjectPtr" case 16 return "Unknown" case 17 return "Text" case 18 return "Unknown" case 19 return "Picture" case 20 return "Unknown" case 21 return "BLOB" case 22 return "Medium" case 23 return "UMedium" case 24 return "FixedBinary" case 25 return "VarBinary" case 26 return "VarChar" case 27 return "LLong" case 28 return "ULLong" end select but with V4RB2 it seems this is not longer correct. My datefields are shown as unknown and my VarChars are shown as nothing. Where can I find new correct solution? thanks Claudius -- G4/733 QS / MacOS X 10.4de / RB 2005r4/ Valentina 1.10.0 & 2.0.4 Homepage http://www.ClaSai.de iChat ryhoruk RealBasic ListBoxes: [ I feel the need...the need for speed!!! ] From Claudius at sailer-online.de Sat Nov 12 22:12:47 2005 From: Claudius at sailer-online.de (Claudius Sailer) Date: Sat Nov 12 15:13:34 2005 Subject: FieldType In-Reply-To: <8826FD6D-6404-4697-97F2-3A28580B341E@sailer-online.de> References: <8826FD6D-6404-4697-97F2-3A28580B341E@sailer-online.de> Message-ID: <0A9FB4CE-5882-42B0-8C98-BE6EC6502E5F@sailer-online.de> Hi, okay I found it. it changed :-(( Am 12. Nov 2005 um 21:58 Uhr schrieb Claudius Sailer: > Hi, > > with V4RB1 I read FieldType with following code > > Dim FieldType as string > > FieldType=tmpDB.Table(i).Field(j).Type > > but with V4RB2 it seems this is not longer correct. My datefields > are shown as unknown and my VarChars are shown as nothing. Where > can I find new correct solution? bye Claudius -- G4/733 QS / MacOS X 10.4de / RB 2005r4/ Valentina 1.10.0 & 2.0.4 Homepage http://www.ClaSai.de iChat ryhoruk RealBasic ListBoxes: [ I feel the need...the need for speed!!! ] From sunshine at public.kherson.ua Sat Nov 12 23:24:42 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 12 15:25:27 2005 Subject: FieldType In-Reply-To: <0A9FB4CE-5882-42B0-8C98-BE6EC6502E5F@sailer-online.de> Message-ID: On 11/12/05 11:12 PM, "Claudius Sailer" wrote: > Hi, > > okay I found it. it changed :-(( I am sorry, but I do not see what was changed. Valentina 2.0 do have: Vfield.Type as Integer And new property Vfield.TypeString as String Aha, I see. Type of property changed to Integer. > Am 12. Nov 2005 um 21:58 Uhr schrieb Claudius Sailer: > >> Hi, >> >> with V4RB1 I read FieldType with following code >> >> Dim FieldType as string >> >> FieldType=tmpDB.Table(i).Field(j).Type >> >> but with V4RB2 it seems this is not longer correct. My datefields >> are shown as unknown and my VarChars are shown as nothing. Where >> can I find new correct solution? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Claudius at sailer-online.de Sat Nov 12 22:38:57 2005 From: Claudius at sailer-online.de (Claudius Sailer) Date: Sat Nov 12 15:39:43 2005 Subject: FieldType In-Reply-To: References: Message-ID: <2A4D8FE4-0EEC-44EA-B570-5C86B2D0FF6A@sailer-online.de> Am 12. Nov 2005 um 22:24 Uhr schrieb Ruslan Zasukhin: > On 11/12/05 11:12 PM, "Claudius Sailer" > wrote: > >> Hi, >> >> okay I found it. it changed :-(( > > I am sorry, but I do not see what was changed. > > Valentina 2.0 do have: > > Vfield.Type as Integer > > And new property > > Vfield.TypeString as String > > Aha, I see. Type of property changed to Integer. this was not the problem. Type=15 was ObjectPtr in V4RB1 in V4RB2 it is 30 ;-)) bye Claudius -- G4/733 QS / MacOS X 10.4de / RB 2005r4/ Valentina 1.10.0 & 2.0.4 Homepage http://www.ClaSai.de iChat ryhoruk RealBasic ListBoxes: [ I feel the need...the need for speed!!! ] From sunshine at public.kherson.ua Sun Nov 13 10:11:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 13 02:11:55 2005 Subject: FieldType In-Reply-To: <2A4D8FE4-0EEC-44EA-B570-5C86B2D0FF6A@sailer-online.de> Message-ID: On 11/12/05 11:38 PM, "Claudius Sailer" wrote: > > Am 12. Nov 2005 um 22:24 Uhr schrieb Ruslan Zasukhin: > >> On 11/12/05 11:12 PM, "Claudius Sailer" >> wrote: >> >>> Hi, >>> >>> okay I found it. it changed :-(( >> >> I am sorry, but I do not see what was changed. >> >> Valentina 2.0 do have: >> >> Vfield.Type as Integer >> >> And new property >> >> Vfield.TypeString as String >> >> Aha, I see. Type of property changed to Integer. > > this was not the problem. > > Type=15 was ObjectPtr in V4RB1 in V4RB2 it is 30 ;-)) Aha I see. The correct way is to use not numbers! But constants: if fld.Type = EVFieldType.kObjectPtr Then you have safe sources -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 14 10:30:35 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 14 02:31:22 2005 Subject: IndexbyWords In-Reply-To: Message-ID: On 11/14/05 12:20 AM, "Ed Kleban" wrote: Hi Ed, > Ruslan, > > I found in the mail archives where you noted: > > How Valentina break string on words: > again it try do this naturally > spaces, punctuations -- all these are breakers. > > Can you be more specific? In fact can you be absolutely precise please? Well in Valentina 2.0 we use IBM ICU and its algorithms. ICU break to words using rules of specified language /locale. We even do not control this > For example is "_" part of words, or a separator of words? How about > digit-only numbers, do these get indexed as words? Even not remember :-) > how about a word > containing alphabetic characters but beginning with a numeric digit? How > about the punctuation that separates words... are there any special cases > where these too are indexed? Well, it is possible to check all this using V4RB exmaple Common/SplitToWords > I see no clues to any of these questions in any of the manuals. > > Thanks! > --Ed > > -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 14 10:46:39 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 14 02:47:27 2005 Subject: Hash for varChar fields. In-Reply-To: Message-ID: On 11/14/05 12:51 AM, "Ed Kleban" wrote: Hi Ed, > Hi Ruslan, > > I'm working with a unique varChar(504) field that will have lots of strings > of average length of say 30 chars. > Normally, if I were not using a database, I'd create us a hash table, such > as the Einhugur ObjectDictionary class to store and rapidly lookup strings. > I'm wondering if it would be worth while to create a hash of the string, say > using the RB variant.hash method, and store this in a parallel field of the > table containing the unique VarChar field, as a means of accelerating > lookups. Binary Searches for the LongInt hash field would presumbably be > much faster than binary searches on the varChar field. Does this sound > like a reasonable approach, or am I mising something? Yes this is good idea, just note, with hash you can do searches only as WHERE fld = xxxx But not WHERE fld > xxx > I'd still have to handle hash collisions of course if the text didn't > match... Which raises an interesting question. If I knew which entry of the > index table for the hashfield was associated with the FindFirst call, then I > could just check varchar records identified by successive rows in the index > table for all matching hash values. Is there a way to access the the index > table and use it in this manner? Valentina 2.0 have for this method Vfield.ExistsValue( ) also Vfield.FindValue( ) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 14 11:55:34 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 14 03:56:20 2005 Subject: Child vs parent records In-Reply-To: Message-ID: On 11/14/05 7:15 AM, "Ed Kleban" wrote: Hi Ed, * this should be used only for SELF-recursive links * so image Table "Person". And Let it have link "Born To" of kind M : M (actually 2 : M) * let you have record with RecID = 100 and it is person "John Johnson". this record have linked records. but look! This linked records are of 2 kinds: Parents and Childs. So when you do found = FindLink( 100, tblPerson, tblPerson, ??? ) Who you want to find? Parents or Childs? Last parameter allow you specify this. P.S. Hmm... May be in fact we can make MORE functions to make things more clean... FindLinked( recid, T1, T2 ) FindLinkedRecursive( recid, T, inRecursionDirection ) > Hi Ruslan, > > I've read the description below about the difference between > kFromParentToChild vs "from child to parent" at least a dozen times and > just can't figure it out. Can you explain this in some other terms than > parent/child or define what constitutes a parent vs a child in this context? > I'm especially confused by the "(s)" in "record(s)" in that I can understand > a parent record having multiple children by virtue of the the children > having an ObjectPtr field that identifies the RecID of the Parent. But I > can't figure out what could possibly be meant by the implication that a > child record might have more than one parent. > > Thanks! > --Ed > > > >> FindLinked( >> inRecID as Integer, >> inTableA as VTable. >> inTableB as VTable >> inRecursionDirection as EVRecursionDirection = kFromParentToChild ) >> as VArraySet >> Returns the records from inTableB linked to inRecID from inTableA. If zero >> records are >> found then returns NIL. >> For a recursive link you should specify the parameter inRecursionDirection. >> If >> you specify >> kFromParentToChild then the function will use child records of the inRecID >> record. Other- >> wise it will use parent record(s) of the inRecID record. > -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 14 23:38:39 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 14 15:39:33 2005 Subject: Child vs parent records In-Reply-To: Message-ID: On 11/14/05 10:49 PM, "Ed Kleban" wrote: Hi Ed, >> * this should be used only for SELF-recursive links > > Yes, I understand. That's exactly what I want to use it for. > >> * so image Table "Person". >> And Let it have link "Born To" of kind M : M (actually 2 : M) >> >> * let you have record with RecID = 100 and it is person "John Johnson". >> this record have linked records. >> but look! This linked records are of 2 kinds: Parents and Childs. >> >> So when you do >> >> found = FindLink( 100, tblPerson, tblPerson, ??? ) >> >> Who you want to find? Parents or Childs? >> Last parameter allow you specify this. > > Yes, I understand all of this also. > However I still can't tell from the description in the document which record > "child" refers to and which record "parent" refers to. Aha, I see. Look again on method: found = FindLink( 100, tblPerson, tblPerson, ??? ) ^^^^^^^ ^^^^^^^^ parent child. > The text states: " If you specify kFromParentToChild then the function will > use child records of the inRecID record. Otherwise it will use parent > record(s) of the inRecID record." > > What do you mean by "child records of the inRecID record"? > What do you mean by "parent record(s) of the inRecID record? Okay, let me try to explain. We have recursive link. Recursive link can in fact be written as: Table1 --- Link --- Table1 So it similar to Table1 --- Link --- Table2 Right? We often write 1 : M link between 2 tables as: ParentTable -- link -- ChildTable Right? So if you in brain unroll recursive link as above, you can think that LEFT table is PARENT table, ad RIGHT table is CHILD table. If take example with "Person" table and link "ParentOf" Person ----- ParentOf ------ Person ^^^^^^ ^^^^^ parent child > Is child vs parent determined by which is specified as TableA vs TableB? For 2 different tables YES! For recursive link, they are the same, THIS IS WHY we need this last parameter: to specify direction of hierarchy. > Are parents always the records in tableA that have a recID value that is > equal to the ObjectPtr field value in Table B; And therefore children are > the records that have an ObjectPtr field value in Table B matching the recID > of TableA? > The terms "parent" and "child" simply have no meaning here without > definition. There must be something that you understand implicitly about > what a parent or child is that simply isn't documented here. > > Or perhaps there's something explicit here that I'm simply not getting. I hope now it is more clear. Yes we need write a lots more text in docs about this . >> P.S. Hmm... May be in fact we can make MORE functions to make things more >> clean... >> >> FindLinked( recid, T1, T2 ) >> FindLinkedRecursive( recid, T, inRecursionDirection ) > > An interesting idea, but sheds on light on the question I have. Hmm, or even better idea! It looks more natural as set = TableA.FindLinkedTo( ThisRecID, ByThisLink, InTableB ); // and for recursive links: TableA.FindChildsOf( ThisRcID, ByThisLink ) TableA.FindParentsOf( ThisRcID, ByThisLink ) Problem is that somebody can use this for non-recursive ... May be its possible to prevent this ... It needs to think about this . -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 15 09:41:46 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 01:42:40 2005 Subject: Child vs parent records In-Reply-To: Message-ID: On 11/15/05 12:22 AM, "Ed Kleban" wrote: Hi Ed, >> Okay, let me try to explain. >> >> We have recursive link. Recursive link can in fact be written as: >> >> Table1 --- Link --- Table1 >> >> So it similar to >> >> Table1 --- Link --- Table2 >> >> Right? >> >> We often write 1 : M link between 2 tables as: >> >> ParentTable -- link -- ChildTable >> >> Right? >> >> So if you in brain unroll recursive link as above, you can think that LEFT >> table is PARENT table, ad RIGHT table is CHILD table. >> >> If take example with "Person" table and link "ParentOf" >> >> Person ----- ParentOf ------ Person >> ^^^^^^ ^^^^^ >> parent child >> > > Ok, now we're getting somewhere, and somewhere that makes some good > intuitive sense. > > Based on the above, would the following statements be true? > > S1) Creating a non-unique ObjectPtr field that contains recID values > referring to records in the same table that contains the ObjectPtr field > inherently defines a one to many relationship between a record indexed by a > specific recID value, and all of the records that contain that recID value > in their ObjectPtr field. YES. The same correct for 2 different tables. > S2) If the ObjectPtr field referring to recId values in the same table that > contains the field is also defined as having unique values, then the > relationship established is inherently one to one. YES. The same correct for 2 different tables. > S3) A one to many relationship can also be referred to as a "parent/child" > relationship in which any one record associated with a given recID value is > called the parent of all child records that contain that recID value in > their ObjectPtr field. YES > If these are true, then these are the "missing" implicit information that > experienced database users may take for granted, but which novices (or picky > readers like myself) would appreciate seeing explicitly stated in the > documentation. AGREE > This is still insufficient however, to explicitly explain what values of > inRecursionDirection actually do. Let me attempt to define those as well: Yes, I think it needs add some pictures also > S4) When calling aLink.FindLinked( inRecID, inTableA, inTableB, > inRecursionDirection ) with inTableA = inTableB, then a value of > inRecursionDirect = kFromParentToChild returns a VArraySet with the recID of > zero, one, or many children records having inRecID as the value of their > ObjectPtr field; whereas > S5) inRecursionDirect = kFromChildToParent returns a VArraySet with at most > a single parent record identified by the non-zero value in the ObjectPtr > field value of the child record indexed by recID. If the ObjectPtr field > contains zero then the returned VArraySet is empty. Right. Only not complete yet: * you talk only about ObjectPtr always. * Valentina 2 have also link - FOREIGN KEY. it is similar to ObjectPtr by nature. It is a field. * Valentina 2 also have new link -- BbinaryLink. This is not a field at all. Methods of Vlink class work for ALL link kinds. So it needs write text in general form. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 15 09:44:29 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 01:45:21 2005 Subject: Cache size for in-Memory database In-Reply-To: Message-ID: On 11/15/05 1:22 AM, "Ed Kleban" wrote: Hi Ed, > What is the affect of selecting various cache sizes when your database resizes > in RAM only? RAM DB do not use Cache at all. > If you're running on MacOS X with virtual memory is there any good reason to > or not to just create a very large cache size? For RAM database -- you can make it very small e.g. 1MB. Note, virtual RAM of OS is not very good for DB tasks. This is why each DBMS have own cache. It is more specific and effective -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 15 10:50:33 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 02:51:26 2005 Subject: IndexbyWords In-Reply-To: Message-ID: On 11/15/05 10:29 AM, "Ed Kleban" wrote: Hi Ed, > Yeah, I'm sure it probably can. But SQL REGEX is going to run a dynamic > search upon demand looking at all records, not offer me a pre-indexed > solution that will be able to provide an almost instant response. right > Is there a way to get a list of all the words that are in the index? No. Not in V4RB. Hey, Ed! you ask so advanced features!!! May be you need go down to C++ level ? In C++ we have IndexIterator which allow iterate unique indexed values. > If so I could play some games by searching for a partial match among > those candidates to greatly reduce the number of records that I'd have > to do a REGEX search on. > On 11/15/05 2:18 AM, "Ruslan Zasukhin" wrote: > >> On 11/15/05 1:29 AM, "Ed Kleban" wrote: >> >>> >>>> Well, it is possible to check all this using V4RB exmaple >>>> >>>> Common/SplitToWords >>>> >>> >>> Thanks. Very useful. Answer appears to be that words consist of >>> alphanumerics in any order. Is there any way to change this so that for >>> example IndexByWords includes "_" as a character, or to make SQL calls >>> directly to do an indexByWords on a field with the scan characteristics you >>> desire? >> >> ICU allow this, but its advanced techniques. We have not study this deeply, >> And than more did not provide this feature to plugins API yet. >> >> I think, SQL REGEX can do most thing you can wish -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 15 10:51:43 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 02:52:34 2005 Subject: Cache size for in-Memory database In-Reply-To: Message-ID: On 11/15/05 10:31 AM, "Ed Kleban" wrote: > Is there a way to tell how much RAM V4RB is using for the database at any > given time? Yes. Valentina.CacheSize P.S will be great if you will ask Valentina specific questions on Valentina list. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 15 11:02:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 03:03:15 2005 Subject: Child vs parent records In-Reply-To: Message-ID: On 11/15/05 10:38 AM, "Ed Kleban" wrote: > >> Only not complete yet: >> >> * you talk only about ObjectPtr always. >> >> * Valentina 2 have also link - FOREIGN KEY. >> it is similar to ObjectPtr by nature. It is a field. > > Yes, certainly. And in a full discussion it should be mentioned as well. > But of course it's the ObjectPtr aspect that offers the great performance > and appeal to me and which I intend to use, so it was on the top of my mind > in trying to understand this. I see. Although note, that BinaryLinks also are very cool. Actually we think now: who need ObjectPtr if exists BinaryLink ?! :-) >> * Valentina 2 also have new link -- BbinaryLink. This is not a field at all. >> >> Methods of Vlink class work for ALL link kinds. So it needs write text in >> general form. > > Yes. I understand and agree. First I had to understand what it did. I > could write something that would serve all cases now. > I very much like the concept of the Vlink; making the links be an instantiated > abstraction is very cool. Actually Link abstraction play a lots of roles and benefits. E.g. The same SQL do joins using Link abstraction. And so on. New Link Function (VSQL.pdf) can do cool things also. And new SQL syntax of fldPtrOfT1->fldOfT2 also do interesting things. > I think however that there may be a better way of presenting > the description of Vlinks to the reader. No doubts. Just I need catch time and write many new pages about this feature. > It's a little difficult for me to > rationalize them being a third kind of link rather than a separate > abstraction for referring to a link using a mechanism that is essentially > the same as the ObjectPtr approach if you look underneath the covers -- at > least I presume that's the case. It's just a matter of where you keep track > of that extra ObjectPtr column for book-keeping purposes. > > Thanks for the feedback. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Tue Nov 15 11:55:36 2005 From: Ed at Kleban.com (Ed Kleban) Date: Tue Nov 15 11:56:29 2005 Subject: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: > P.S will be great if you will ask Valentina specific questions on Valentina > list. > >Ruslan Zasukhin Yes, You're right of course. Here then, for the benefit of the list, is a summary of some important things we've discussed recently. I'll endeavor to use the list more. =========================== HOW MUCH RAM IS BEING USED =========================== >> Is there a way to tell how much RAM V4RB is using for the database at any >> given time? > > Yes. > > Valentina.CacheSize > >> What is the affect of selecting various cache sizes when your database >> resizes >> in RAM only? > > RAM DB do not use Cache at all. > >> If you're running on MacOS X with virtual memory is there any good reason to >> or not to just create a very large cache size? > > For RAM database -- you can make it very small e.g. 1MB. > > > Note, virtual RAM of OS is not very good for DB tasks. > This is why each DBMS have own cache. It is more specific and effective ========================================== PRE-HASHING OF UNIQUE VARCHAR FIELD VALUES ========================================== >> Hi Ruslan, >> >> I'm working with a unique varChar(504) field that will have lots of strings >> of average length of say 30 chars. >> Normally, if I were not using a database, I'd create us a hash table, such >> as the Einhugur ObjectDictionary class to store and rapidly lookup strings. >> I'm wondering if it would be worth while to create a hash of the string, say >> using the RB variant.hash method, and store this in a parallel field of the >> table containing the unique VarChar field, as a means of accelerating >> lookups. Binary Searches for the LongInt hash field would presumbably be >> much faster than binary searches on the varChar field. Does this sound >> like a reasonable approach, or am I mising something? > > Yes this is good idea, just note, with hash you can do searches only as > > WHERE fld = xxxx > > But not > > WHERE fld > xxx > > >> I'd still have to handle hash collisions of course if the text didn't >> match... Which raises an interesting question. If I knew which entry of the >> index table for the hashfield was associated with the FindFirst call, then I >> could just check varchar records identified by successive rows in the index >> table for all matching hash values. Is there a way to access the the index >> table and use it in this manner? > > Valentina 2.0 have for this method > > Vfield.ExistsValue( ) > > also > > Vfield.FindValue( ) > ======================================= WHAT IS MEANT BY "PARENT" AND "CHILD"? ======================================= >>> Hi Ruslan, >>> >>> I've read the description below about the difference between >>> kFromParentToChild vs "from child to parent" at least a dozen times and >>> just can't figure it out. Can you explain this in some other terms than >>> parent/child or define what constitutes a parent vs a child in this context? >>> I'm especially confused by the "(s)" in "record(s)" in that I can understand >>> a parent record having multiple children by virtue of the the children >>> having an ObjectPtr field that identifies the RecID of the Parent. But I >>> can't figure out what could possibly be meant by the implication that a >>> child record might have more than one parent. >>> >> Hi Ed, >> >> * this should be used only for SELF-recursive links >> >> * so image Table "Person". >> And Let it have link "Born To" of kind M : M (actually 2 : M) >> >> * let you have record with RecID = 100 and it is person "John Johnson". >> this record have linked records. >> but look! This linked records are of 2 kinds: Parents and Childs. >> >> So when you do >> >> found = FindLink( 100, tblPerson, tblPerson, ??? ) >> >> Who you want to find? Parents or Childs? >> Last parameter allow you specify this. >> >> P.S. Hmm... May be in fact we can make MORE functions to make things more >> clean... >> >> FindLinked( recid, T1, T2 ) >> >> FindLinkedRecursive( recid, T, inRecursionDirection ) >> >> > >>>> RUSLAN: >>>> Okay, let me try to explain. >>>> >>>> We have recursive link. Recursive link can in fact be written as: >>>> >>>> Table1 --- Link --- Table1 >>>> >>>> So it similar to >>>> >>>> Table1 --- Link --- Table2 >>>> >>>> Right? >>>> >>>> We often write 1 : M link between 2 tables as: >>>> >>>> ParentTable -- link -- ChildTable >>>> >>>> Right? >>>> >>>> So if you in brain unroll recursive link as above, you can think that LEFT >>>> table is PARENT table, ad RIGHT table is CHILD table. >>>> >>>> If take example with "Person" table and link "ParentOf" >>>> >>>> Person ----- ParentOf ------ Person >>>> ^^^^^^ ^^^^^ >>>> parent child >>>> >>> >>> Ok, now we're getting somewhere, and somewhere that makes some good >>> intuitive sense. >>> >>> Based on the above, would the following statements be true? >>> >>> S1) Creating a non-unique ObjectPtr field that contains recID values >>> referring to records in the same table that contains the ObjectPtr field >>> inherently defines a one to many relationship between a record indexed by a >>> specific recID value, and all of the records that contain that recID value >>> in their ObjectPtr field. >> >> YES. The same correct for 2 different tables. >> >>> S2) If the ObjectPtr field referring to recId values in the same table that >>> contains the field is also defined as having unique values, then the >>> relationship established is inherently one to one. >> >> YES. The same correct for 2 different tables. >> >>> S3) A one to many relationship can also be referred to as a "parent/child" >>> relationship in which any one record associated with a given recID value is >>> called the parent of all child records that contain that recID value in >>> their ObjectPtr field. >> >> YES >> >>> If these are true, then these are the "missing" implicit information that >>> experienced database users may take for granted, but which novices (or picky >>> readers like myself) would appreciate seeing explicitly stated in the >>> documentation. >> >> AGREE >> >>> This is still insufficient however, to explicitly explain what values of >>> inRecursionDirection actually do. Let me attempt to define those as well: >> >> Yes, I think it needs add some pictures also >> >>> S4) When calling aLink.FindLinked( inRecID, inTableA, inTableB, >>> inRecursionDirection ) with inTableA = inTableB, then a value of >>> inRecursionDirect = kFromParentToChild returns a VArraySet with the recID of >>> zero, one, or many children records having inRecID as the value of their >>> ObjectPtr field; whereas >> >>> S5) inRecursionDirect = kFromChildToParent returns a VArraySet with at most >>> a single parent record identified by the non-zero value in the ObjectPtr >>> field value of the child record indexed by recID. If the ObjectPtr field >>> contains zero then the returned VArraySet is empty. >> >> Right. >> > >Ok, great. Now I understand. :) > >> Only not complete yet: >> >> * you talk only about ObjectPtr always. >> >> * Valentina 2 have also link - FOREIGN KEY. >> it is similar to ObjectPtr by nature. It is a field. > >Yes, certainly. And in a full discussion it should be mentioned as well. But of course it's the ObjectPtr aspect that offers the great performance >and appeal to me and which I intend to use, so it was on the top of my mind in trying to understand this. > >> * Valentina 2 also have new link -- BbinaryLink. This is not a field at all. >> >> Methods of Vlink class work for ALL link kinds. So it needs write text in >> general form. >> > >Yes. I understand and agree. First I had to understand what it did. I could write something that would serve all cases now. I very much like the concept of the Vlink; making the links be an instantiated abstraction is very cool. I think however that there may be a better way of presenting the description of Vlinks to the reader. It's a little difficult for me to rationalize them being a third kind of link rather than a separate abstraction for referring to a link using a mechanism that is essentially the same as the ObjectPtr approach if you look underneath the covers -- at least I presume that's the case. It's just a matter of where you keep track of that extra ObjectPtr column for book-keeping purposes. > >Thanks for the feedback. > >--Ed ============================================ PROPOSED NEW SYNTAXES FOR PARENT-CHILD LINKS ============================================ >>>> RUSLAN: >>>> P.S. Hmm... May be in fact we can make MORE functions to make things more >>>> clean... >>>> >>>> FindLinked( recid, T1, T2 ) >>>> FindLinkedRecursive( recid, T, inRecursionDirection ) >>> >>> An interesting idea, but sheds [no] light on the question I have. >> >> Hmm, or even better idea! >> It looks more natural as >> >> set = TableA.FindLinkedTo( ThisRecID, ByThisLink, InTableB ); >> >> // and for recursive links: >> >> TableA.FindChildsOf( ThisRcID, ByThisLink ) >> TableA.FindParentsOf( ThisRcID, ByThisLink ) >> >> Problem is that somebody can use this for non-recursive ... >> May be its possible to prevent this ... It needs to think about this . > > Yes, I certainly like this form in which the ObjectPtr field is identified > explicitly as an argument much better, and this syntax does make it more > clear. I'll have to leave the Table validation question for you to consider. > > The correct grammar would be "FindChildrenOf" rather than "FindChildsOf". > > From Ed at Kleban.com Tue Nov 15 12:45:20 2005 From: Ed at Kleban.com (Ed Kleban) Date: Tue Nov 15 12:46:13 2005 Subject: test post - please ignore Message-ID: From sunshine at public.kherson.ua Tue Nov 15 21:49:27 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 13:50:17 2005 Subject: [V4NET] ... In-Reply-To: <02d001c5ea1c$11804510$6601a8c0@pantherdev> Message-ID: On 11/15/05 9:37 PM, "Panther Support" wrote: > Ruslan: > > Is there any way to load a Cursor from another Cursor? No. On many strong reasons > As an example, say I > load a Cursor with this: > > dim db as new ValentinaCOM2.VDataBase > dim dbreed as ValentinaCOM2.VCursor > dbreed = db.SqlSelect("select * from breeds") > > This will load the entrie table to information for me. But suppose now I > want to just focus on some of the records. Do I need to access the database > all the time to load another cursor? MAY BE you can make new table. And query it. But I think this will be slower Aha, if you sure that result is not big, you can make temporary RAM table > dim dbreed1 as ValentinaCOM2.VCursor > dbreed1= **** selected records of dbreed **** > > Any thoughts? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 07:36:44 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 15 23:37:36 2005 Subject: [V4RB] How to make own func with params EVLockType, ... Message-ID: Hi Dan, > Also could you help me with this. > > I don't seem to be able to pass parameters for CursorLocation, LockType > and CursorDirection. > > I can get it to work in a SqlSelect command just passing > ",EVCursorLocation.kServerSide, EVLockType.kReadWrite, > EVCursorDirection.kRandom" > But if i for instance make a method and want to pass parameters for a > VCursor, I can pass the Database, and The SqlString, but not the > CursorLocation, LockType and CursorDirection. > > If I for instance add a parameter " InCursorLocation As EVCursorLocation" > And try passing kServerSide I get "Parameters are not kompatible?? > > What should I pass? You should pass EVCursorLocation.kServerSide -- full form, This is like enums work in Java. And they will work in this form in RB, When REAL will add support of them. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 16 00:24:00 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 00:24:54 2005 Subject: Cache allocation for RAM-only databases. In-Reply-To: Message-ID: On 11/15/05 11:55 AM, "Ed Kleban" wrote: >>> What is the affect of selecting various cache sizes when your database >>> resizes >>> in RAM only? >> >> RAM DB do not use Cache at all. >> > >>> If you're running on MacOS X with virtual memory is there any good reason to >>> or not to just create a very large cache size? >> >> For RAM database -- you can make it very small e.g. 1MB. >> >> >> Note, virtual RAM of OS is not very good for DB tasks. >> This is why each DBMS have own cache. It is more specific and effective If a RAM DB does not use Cache at all, then what does it matter what you set the cache size to? Can you also make it very large, e.g. 100 MB, and if you do does that make any difference? From sunshine at public.kherson.ua Wed Nov 16 08:29:12 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 00:30:10 2005 Subject: Cache allocation for RAM-only databases. In-Reply-To: Message-ID: On 11/16/05 8:24 AM, "Ed Kleban" wrote: >>>> What is the affect of selecting various cache sizes when your database >>>> resizes in RAM only? >>>> >>> RAM DB do not use Cache at all. >>>> If you're running on MacOS X with virtual memory is there any good reason >>>> to or not to just create a very large cache size? >>>> >>> For RAM database -- you can make it very small e.g. 1MB. >>> >>> Note, virtual RAM of OS is not very good for DB tasks. >>> This is why each DBMS have own cache. It is more specific and effective > > If a RAM DB does not use Cache at all, then what does it matter what you set > the cache size to? No matter > Can you also make it very large, e.g. 100 MB, and if you > do does that make any difference? Just you waste your RAM. Sense ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 16 00:36:03 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 00:36:54 2005 Subject: Cache allocation for RAM-only databases. In-Reply-To: Message-ID: On 11/16/05 12:29 AM, "Ruslan Zasukhin" wrote: > On 11/16/05 8:24 AM, "Ed Kleban" wrote: > >>>>> What is the affect of selecting various cache sizes when your database >>>>> resizes in RAM only? >>>>> >>>> RAM DB do not use Cache at all. > >>>>> If you're running on MacOS X with virtual memory is there any good reason >>>>> to or not to just create a very large cache size? >>>>> >>>> For RAM database -- you can make it very small e.g. 1MB. >>>> >>>> Note, virtual RAM of OS is not very good for DB tasks. >>>> This is why each DBMS have own cache. It is more specific and effective >> >> If a RAM DB does not use Cache at all, then what does it matter what you set >> the cache size to? > > No matter > >> Can you also make it very large, e.g. 100 MB, and if you >> do does that make any difference? > > Just you waste your RAM. Sense ? > So it ends up allocating RAM that it simply doesn't use. If that's what you mean, then yes it makes sense. So why not allocate a cache size of 0 ? From Ed at Kleban.com Wed Nov 16 00:45:59 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 00:46:54 2005 Subject: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/15/05 11:55 AM, "Ed Kleban" wrote: > >> Ok, great. Now I understand. :) >> >>> Only not complete yet: >>> >>> * you talk only about ObjectPtr always. >>> >>> * Valentina 2 have also link - FOREIGN KEY. >>> it is similar to ObjectPtr by nature. It is a field. >> >> Yes, certainly. And in a full discussion it should be mentioned as well. >> But of course it's the ObjectPtr aspect that offers the great performance >> and appeal to me and which I intend to use, so it was on the top of my mind >> in trying to understand this. >> >>> * Valentina 2 also have new link -- BbinaryLink. This is not a field at all. >>> >>> Methods of Vlink class work for ALL link kinds. So it needs write text in >>> general form. >>> >> >> Yes. I understand and agree. First I had to understand what it did. I could >> write something that would serve all cases now. I very much like >> the concept of the Vlink; making the links be an instantiated abstraction >> is very cool. >> --Ed Well, I take that back. I clearly don't understand. Last night in bed I started wondering: "I wonder if there's a way to simply use a byte-wide field as you would use an ObjectPtr field if it is known in advance that the linked-to table will never have more that 255 records? " Then I thought, "Hmm. maybe that's what Binary Links can do if you create a link between a byte-size field in one table with the virtual RecID field in another table." But I just now read all the doc in both the Kernel and RB Reference about Binary Links and decided I don't have a clue about what they do or how to use 'em. Specifically: The constructor VBinaryLink takes as arguments a pair of tables, and some referece ("power") , deletion, and storage arguments -- but has no field arguments. So my conjecture that it could relate specific fields was clearly wrong, that's not what it does, nor is that what the documentation says it does. In fact as the doc clearly states it doesn't require field allocations. So I guess I'm stuck. I don't understand these critters. I understand how to create a binary link with VBinaryLink(...), but I don't know how to use the link once I have it. How do I tell the link, or how does the link know, which records in Table A correspond to which records in Table B, or vice versa? --Ed From sunshine at public.kherson.ua Wed Nov 16 08:51:43 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 00:52:34 2005 Subject: Cache allocation for RAM-only databases. In-Reply-To: Message-ID: On 11/16/05 8:36 AM, "Ed Kleban" wrote: >> Just you waste your RAM. Sense ? >> > > So it ends up allocating RAM that it simply doesn't use. If that's what you > mean, then yes it makes sense. right > So why not allocate a cache size of 0 ? I have never try this :-) May happens that Import/Export for example still will try use cache. It needs check code. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 08:57:14 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 00:58:06 2005 Subject: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 8:45 AM, "Ed Kleban" wrote: Hi Ed, >>>> * Valentina 2 also have new link -- BbinaryLink. This is not a field at >>>> all. >>>> >>>> Methods of Vlink class work for ALL link kinds. So it needs write text in >>>> general form. >>>> >>> >>> Yes. I understand and agree. First I had to understand what it did. I >>> could >>> write something that would serve all cases now. I very much like >>> the concept of the Vlink; making the links be an instantiated abstraction >>> is very cool. >>> --Ed > > Well, I take that back. I clearly don't understand. Last night in bed I > started wondering: :-) > "I wonder if there's a way to simply use a byte-wide field as you would use > an ObjectPtr field if it is known in advance that the linked-to table will > never have more that 255 records? " YES. Then you get old good FOREIGN KEY. And this still is VLink. > Then I thought, "Hmm. maybe that's what Binary Links can do if you create a > link between a byte-size field in one table with the virtual RecID field in > another table." NO. BinaryLink always use RecID values. > But I just now read all the doc in both the Kernel and RB Reference about > Binary Links and decided I don't have a clue about what they do or how to > use 'em. They do effectively the same as you do in Relational model with help of third Table to establish M:M link. What do this table? It just remember pairs (id1, id2) BinaryLink do the same, but use RecIDs and it is NOT a table. It is more effective structure. > Specifically: The constructor VBinaryLink takes as arguments a pair of > tables, and some referece ("power") , deletion, and storage arguments -- but > has no field arguments. Right. Because always RecID are used. ObjectPtr also do not use "field" of pointed table. Remember? Only Foreign KEY need field. > So my conjecture that it could relate specific > fields was clearly wrong, that's not what it does, nor is that what the > documentation says it does. In fact as the doc clearly states it doesn't > require field allocations. right > So I guess I'm stuck. I don't understand these critters. I understand how > to create a binary link with VBinaryLink(...), but I don't know how to use > the link once I have it. How do I tell the link, or how does the link know, > which records in Table A correspond to which records in Table B, or vice > versa? simple: link.Add( tblA.RecID, tblBRecID ) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 08:58:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 00:59:43 2005 Subject: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 8:57 AM, "Ruslan Zasukhin" wrote: >> So I guess I'm stuck. I don't understand these critters. I understand how >> to create a binary link with VBinaryLink(...), but I don't know how to use >> the link once I have it. How do I tell the link, or how does the link know, >> which records in Table A correspond to which records in Table B, or vice >> versa? > > simple: > > link.Add( tblA.RecID, tblBRecID ) And note, you can link NOT only current records! You can send into link.Add() any RecIDs, the main is that records exists. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 16 01:05:54 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 01:06:47 2005 Subject: How to link records. Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 12:45 AM, "Ed Kleban" wrote: > > > > On 11/15/05 11:55 AM, "Ed Kleban" wrote: > >> >>> Ok, great. Now I understand. :) >>> >>>> Only not complete yet: >>>> >>>> * you talk only about ObjectPtr always. >>>> >>>> * Valentina 2 have also link - FOREIGN KEY. >>>> it is similar to ObjectPtr by nature. It is a field. >>> >>> Yes, certainly. And in a full discussion it should be mentioned as well. >>> But of course it's the ObjectPtr aspect that offers the great performance >>> and appeal to me and which I intend to use, so it was on the top of my mind >>> in trying to understand this. >>> >>>> * Valentina 2 also have new link -- BbinaryLink. This is not a field at >>>> all. >>>> >>>> Methods of Vlink class work for ALL link kinds. So it needs write text in >>>> general form. >>>> >>> >>> Yes. I understand and agree. First I had to understand what it did. I >>> could >>> write something that would serve all cases now. I very much like >>> the concept of the Vlink; making the links be an instantiated abstraction >>> is very cool. >>> --Ed > > Well, I take that back. I clearly don't understand. Last night in bed I > started wondering: > > "I wonder if there's a way to simply use a byte-wide field as you would use > an ObjectPtr field if it is known in advance that the linked-to table will > never have more that 255 records? " > > Then I thought, "Hmm. maybe that's what Binary Links can do if you create a > link between a byte-size field in one table with the virtual RecID field in > another table." > > But I just now read all the doc in both the Kernel and RB Reference about > Binary Links and decided I don't have a clue about what they do or how to > use 'em. > > Specifically: The constructor VBinaryLink takes as arguments a pair of > tables, and some referece ("power") , deletion, and storage arguments -- but > has no field arguments. So my conjecture that it could relate specific > fields was clearly wrong, that's not what it does, nor is that what the > documentation says it does. In fact as the doc clearly states it doesn't > require field allocations. > > So I guess I'm stuck. I don't understand these critters. I understand how > to create a binary link with VBinaryLink(...), but I don't know how to use > the link once I have it. How do I tell the link, or how does the link know, > which records in Table A correspond to which records in Table B, or vice > versa? > Ah, I found it. It's the LinkRecords and UnlinkRecords commands. Hmm. I'll have to think about this. I guess you get a performance advantage from not having to actually put data into the table in order to create a link. Therefore you don't have to visit all the records, but rather just stuff the relation info into the Binary Link table. I guess that makes sense. And I suppose that if the base tables are on disk, but you only need a temporary binary link in RAM, you get a big boost as well. But if both are retained on disk, does this end up taking less storage or more? I guess if you're only storing a list of which records have relations, then you need a Ulong index for both each link to designate one record in each table, but you don't have to have space allocated in baseTable records that aren't linked. Aha! That's the big speed gain. When you need to check relations you don't have to visit every record in one of the tables to see if there's a link, you only need to scan the list of established links in the binary Vlink. From Ed at Kleban.com Wed Nov 16 01:16:10 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 01:17:04 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: >> But I just now read all the doc in both the Kernel and RB Reference about >> Binary Links and decided I don't have a clue about what they do or how to >> use 'em. > > They do effectively the same as you do in Relational model with help of > third Table to establish M:M link. > > What do this table? It just remember pairs (id1, id2) > > BinaryLink do the same, but use RecIDs and it is NOT a table. > It is more effective structure. Hmm, more effective than a table. Meaning not a linear list of pairs, or meaning not a list that's the same length as one of the two related tables. The former I guess. Hmm... Well, you could certainly make it more efficient by keeping two separate indexed lists so you could do a binary search in either direction. And you could make it more efficient with an indexed list that pointed to a "bucket" with a block of corresponding related entries so that when doing a binary search you didn't have the overhead of duplicates, and once you did the seach you had a linear list of corresponding records. Yeah, that's probably how I'd do it... if I could avoid the temptation of hashing the indexes to avoid the log2N cost of the binary search for large sets of links. From Ed at Kleban.com Wed Nov 16 09:52:58 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 09:53:55 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 1:16 AM, "Ed Kleban" wrote: >>> But I just now read all the doc in both the Kernel and RB Reference about >>> Binary Links and decided I don't have a clue about what they do or how to >>> use 'em. >> >> They do effectively the same as you do in Relational model with help of >> third Table to establish M:M link. >> >> What do this table? It just remember pairs (id1, id2) >> >> BinaryLink do the same, but use RecIDs and it is NOT a table. >> It is more effective structure. > > Hmm, more effective than a table. Meaning not a linear list of pairs, or > meaning not a list that's the same length as one of the two related tables. > The former I guess. Hmm... Well, you could certainly make it more efficient > by keeping two separate indexed lists so you could do a binary search in > either direction. And you could make it more efficient with an indexed list > that pointed to a "bucket" with a block of corresponding related entries so > that when doing a binary search you didn't have the overhead of duplicates, > and once you did the seach you had a linear list of corresponding records. > Yeah, that's probably how I'd do it... if I could avoid the temptation of > hashing the indexes to avoid the log2N cost of the binary search for large > sets of links. Ok, I've had some time to sleep on this, and am getting excited! And of course I have more questions. :) Let's run through an example. Say I have a table of people that has a recursive relationship for "MotherOf" as a M:1 relationship. If I use an ObjectPtr field for the MotherOf relationship, then finding the mother for any given record is EXCEEDINGLY fast. I simply go to the MotherOf field, and there's the record ID I need for the mother. And if I intend to record a mother for every person in the table then the representation is also exceedingly efficient. I could make it more space efficient, and possibly less speed efficient by using a UShort field instead of an ObjectPtr if I knew, say I was only going to have, say 20,000 records maximum. Question 1: Would use of a UShort field here for 16-bit encoding of RecID actually be slower than use of an ObjectPtr? In the opposite direction, if I wanted to find all the children of a given mother, it would take much more time since the DB would have to scan every record to look for a matching value in the MotherOf field for a given mother record RecId. For N records the order of performance of the linear search would be o(N). Now lets consider Binary Links. And since you've not yet commented on my supposition above, let's assume that the implementation I postulated is indeed close to what Valentina implements; Namely for a M:1 relationship it keeps separate "info" for the "M->1" and the "1->M" directions, and that for the "1->M" direction in fact the execution requires a single binary search of an indexed list of, in this case known mother records, and results in obtaining a list of all the associated children. In this direction the performance is much better then because: The DB does a single binary search on a small list of records known to be mothers and ends up immediately with the same result as would require scanning every record of the table if an ObjectPtr field was used. Question 2: Am I pretty close so far? In the reverse direction, find the mother of a given child, I can't conceive of anything special the binary link could do that would be any faster than the ObjectPtr case. So for the M->1 "info" retained by the link I'm guessing it simply does that. I.e. it just keeps a mapping for every record that a link is defined for, which in the example above whever every record has a valid MotherOf would essentially end up taking the same amount of space as, and operating in the same amount of time as the ObjectPtr case. Question 3: Am I still on target with this supposition? From Ed at Kleban.com Wed Nov 16 11:30:57 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 11:31:51 2005 Subject: IndexbyWords In-Reply-To: Message-ID: On 11/15/05 2:18 AM, "Ruslan Zasukhin" wrote: > >>> Well, it is possible to check all this using V4RB exmaple >>> >>> Common/SplitToWords >>> Great. I see that this app works by using the method Vstring.SplitToWords( text as string) Which does not appear to be documented in any of the manuals. Is this a supported method we can rely on and use? I also note in the field creation the following comment: // To be able use function SplitToWords() the field must be indexed. fld = tbl.CreateStringField( "fldString", 40, _ EVFlag.fIndexed + EVFlag.fIndexByWords ) But I thought that fIndexed was meaningless when fIndexedByWords was used. Does that mean that fIndexed is not needed here? Or is SplitToWords somehow special? Unfortunately it turns out that "words" as defined by SplitToString include "." and exclude "_". The later is unfortunate for my application. The former seems to be unfortunate for most every application I can think of that might want to otherwise use IndexedByWords. It's not good for prose, verse, or code since "foo" and "foo." and ".foo" would all parse out to be different search terms. That is unless a search for "foo" would return entries that included all three of these (and other) combinations. I'll have to go back and read the fine print on query for IndexedByWords fields. >> Is there any way to change this so that for >> example IndexByWords includes "_" as a character, or to make SQL calls >> directly to do an indexByWords on a field with the scan characteristics you >> desire? > > ICU allow this, but its advanced techniques. We have not study this deeply, > And than more did not provide this feature to plugins API yet. > So... I would like to request as a future enhancement that you offer some control through the API to control how Valentina uses ICU. Perhaps allow an argument with a list of separators or the REGEX that gets used with ICU to determine what is a word. > I think, SQL REGEX can do most thing you can wish > >> Yeah, I'm sure it probably can. But SQL REGEX is going to run a dynamic >> search upon demand looking at all records, not offer me a pre-indexed >> solution that will be able to provide an almost instant response. > > right > >> Is there a way to get a list of all the words that are in the index? > > No. Not in V4RB. > > Hey, Ed! you ask so advanced features!!! > May be you need go down to C++ level ? > No thank you. That's what we've got you for ;-) > In C++ we have IndexIterator which allow iterate unique indexed values. > Ah. This is actually what I was asking about the other day, but didn't convey to well. I was curious if there was a way to directly access the Index tables that V4RB creates. This would allow you to, for example, find out not only whether there was a record in a Hash table indexed by hash, but in the case of duplicate records find out what is the recID within the INDEX TABLE at which the index referring to the entry containing the hash was found. Thus in the case of a hash collision you could retrieve the record referred to by the index in the Next entry of the IndexTable rather than having to do a query that would return all records that shared the same hash. That make sense? Is the above possible in C++? Any chance you'll ever add this interface to the RB API? Thanks as always! --Ed From Ed at Kleban.com Wed Nov 16 12:02:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 12:03:47 2005 Subject: IndexbyWords In-Reply-To: Message-ID: > > Unfortunately it turns out that "words" as defined by SplitToString include > "." and exclude "_". The later is unfortunate for my application. The > former seems to be unfortunate for most every application I can think of > that might want to otherwise use IndexedByWords. It's not good for prose, > verse, or code since "foo" and "foo." and ".foo" would all parse out to be > different search terms. That is unless a search for "foo" would return > entries that included all three of these (and other) combinations. I'll > have to go back and read the fine print on query for IndexedByWords fields. > Well, the fine print is ambiguous at best. But reading between the lines, my guess is that it a search for "foo" is a search for an entry containing a word that BEGINS with "foo". Thus it would find "foo" and "foo." and "foobar." but not ".foo" --- which is probably pretty good for most real applications you'd want to write. From sunshine at public.kherson.ua Wed Nov 16 20:43:11 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 12:44:07 2005 Subject: How to link records. Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 9:05 AM, "Ed Kleban" wrote: > Ah, I found it. It's the LinkRecords and UnlinkRecords commands. > > Hmm. I'll have to think about this. I guess you get a performance advantage > from not having to actually put data into the table in order to create a > link. Therefore you don't have to visit all the records, but rather just > stuff the relation info into the Binary Link table. I guess that makes > sense. BinaryLink have many benefits. One of the most important is: you get Tables as Tables. Links as Links. Structure of Tables do not depend now on how many links around. > And I suppose that if the base tables are on disk, but you only need a > temporary binary link in RAM, you get a big boost as well. I have not think about this, but probably yes. AGAIN. With such tmp link, you DO NOT change tables. > But if both are retained on disk, does this end up taking less storage or > more? YES. 2 times less than usual M M table > I guess if you're only storing a list of which records have > relations, then you need a Ulong index for both each link to designate one > record in each table, but you don't have to have space allocated in > baseTable records that aren't linked. > > Aha! That's the big speed gain. When you need to check relations you don't > have to visit every record in one of the tables to see if there's a link, > you only need to scan the list of established links in the binary Vlink. And this is also. As well as other -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 20:45:05 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 12:46:08 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 9:16 AM, "Ed Kleban" wrote: >>> But I just now read all the doc in both the Kernel and RB Reference about >>> Binary Links and decided I don't have a clue about what they do or how to >>> use 'em. >> >> They do effectively the same as you do in Relational model with help of >> third Table to establish M:M link. >> >> What do this table? It just remember pairs (id1, id2) >> >> BinaryLink do the same, but use RecIDs and it is NOT a table. >> It is more effective structure. > > Hmm, more effective than a table. Meaning not a linear list of pairs, or > meaning not a list that's the same length as one of the two related tables. > The former I guess. Hmm... Well, you could certainly make it more efficient > by keeping two separate indexed lists so you could do a binary search in > either direction. right > And you could make it more efficient with an indexed list > that pointed to a "bucket" with a block of corresponding related entries so > that when doing a binary search you didn't have the overhead of duplicates, > and once you did the seach you had a linear list of corresponding records. > Yeah, that's probably how I'd do it... if I could avoid the temptation of > hashing the indexes to avoid the log2N cost of the binary search for large > sets of links. Valentina have about 5 different kinds of indexes it seems. And non of them is Btree or Hash. All is something else :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 16 13:04:23 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 13:05:18 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 12:45 PM, "Ruslan Zasukhin" wrote: > On 11/16/05 9:16 AM, "Ed Kleban" wrote: > >>>> But I just now read all the doc in both the Kernel and RB Reference about >>>> Binary Links and decided I don't have a clue about what they do or how to >>>> use 'em. >>> >>> They do effectively the same as you do in Relational model with help of >>> third Table to establish M:M link. >>> >>> What do this table? It just remember pairs (id1, id2) >>> >>> BinaryLink do the same, but use RecIDs and it is NOT a table. >>> It is more effective structure. >> >> Hmm, more effective than a table. Meaning not a linear list of pairs, or >> meaning not a list that's the same length as one of the two related tables. >> The former I guess. Hmm... Well, you could certainly make it more efficient >> by keeping two separate indexed lists so you could do a binary search in >> either direction. > > right > >> And you could make it more efficient with an indexed list >> that pointed to a "bucket" with a block of corresponding related entries so >> that when doing a binary search you didn't have the overhead of duplicates, >> and once you did the seach you had a linear list of corresponding records. >> Yeah, that's probably how I'd do it... if I could avoid the temptation of >> hashing the indexes to avoid the log2N cost of the binary search for large >> sets of links. > > Valentina have about 5 different kinds of indexes it seems. > > And non of them is Btree or Hash. > All is something else Yeah, well, you're making a convert out of me on that approach pretty quickly. I've just finished a first pass at re-diagramming the database I'm designing using Binary Links instead of ObjectPtr fields. It's amazing! One question that has come up, is that if I want to associate some type of a typeCode that is a small byte constant that identifies the type of an item, and I need to access that typeCode all the time for items to do select case statements and such, and I also want to keep an anciallary table with additional information about that typeCode, then should I: 1) Both create a Binary link between item and their typeInfo records, where the recID is inherently a representation for the typeCd, and then additionally allocate a byte TypeCode field in the item records, under the theory that I can grab the typeCode very fast most of the time, and then do an indexed lookup by record ID when I occasionally need the extra type info... or 2) Punt the darn typeCode byte in the item record because binary links are so blazingly fast in the many to 1 direction that that having a typeCode byte would be no faster and just make for messy double book keeping. or 3) Right idea, but if you really want the blazingly fast connect from the typeCode in the item record to the typeInfo record then use a full Ulong for the typeCode as an OffsetPtr because Valentina will optimize accesses and be more efficient using the ObjectPtr than converting from a byte index. > :-) From sunshine at public.kherson.ua Wed Nov 16 20:59:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 13:08:24 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 5:52 PM, "Ed Kleban" wrote: >>> They do effectively the same as you do in Relational model with help of >>> third Table to establish M:M link. >>> >>> What do this table? It just remember pairs (id1, id2) >>> >>> BinaryLink do the same, but use RecIDs and it is NOT a table. >>> It is more effective structure. >> >> Hmm, more effective than a table. Meaning not a linear list of pairs, or >> meaning not a list that's the same length as one of the two related tables. >> The former I guess. Hmm... Well, you could certainly make it more efficient >> by keeping two separate indexed lists so you could do a binary search in >> either direction. And you could make it more efficient with an indexed list >> that pointed to a "bucket" with a block of corresponding related entries so >> that when doing a binary search you didn't have the overhead of duplicates, >> and once you did the seach you had a linear list of corresponding records. >> Yeah, that's probably how I'd do it... if I could avoid the temptation of >> hashing the indexes to avoid the log2N cost of the binary search for large >> sets of links. > > Ok, I've had some time to sleep on this, and am getting excited! > And of course I have more questions. :) Great! I like how you think! > Let's run through an example. Say I have a table of people that has a > recursive relationship for "MotherOf" as a M:1 relationship. If I use an > ObjectPtr field for the MotherOf relationship, then finding the mother for > any given record is EXCEEDINGLY fast. I simply go to the MotherOf field, > and there's the record ID I need for the mother. And if I intend to record > a mother for every person in the table then the representation is also > exceedingly efficient. > > I could make it more space efficient, and possibly less speed efficient by > using a UShort field instead of an ObjectPtr if I knew, say I was only going > to have, say 20,000 records maximum. This will speed up also, because HDD -- main break in db. > Question 1: Would use of a UShort field here for 16-bit encoding > of RecID actually be slower than use of an ObjectPtr? NO. And we have think about giving to developer such ability to specify type of ObjectPtr. This bring few problems: - mistake of developer - type of ObjectPtr different, then how right correct code? And honestly was no time to go so deeply. But yes in Valentina 2 you can use FOREIGN KEY now, it works as link,and it have constraints. So in general you loose nothing here. ------------------------ > In the opposite direction, if I wanted to find all the children of a given > mother, it would take much more time since the DB would have to scan every > record to look for a matching value in the MotherOf field for a given mother > record RecId. For N records the order of performance of the linear search > would be o(N). But usually you have INDEX for such fields. And ObjectPtr is always indexed and FK fields also. ------------ > Now lets consider Binary Links. And since you've not yet commented on my > supposition above, let's assume that the implementation I postulated is > indeed close to what Valentina implements; Namely for a M:1 relationship it > keeps separate "info" for the "M->1" and the "1->M" directions, and that for > the "1->M" direction in fact the execution requires a single binary search > of an indexed list of, in this case known mother records, and results in > obtaining a list of all the associated children. In this direction the > performance is much better then because: The DB does a single binary search > on a small list of records known to be mothers and ends up immediately with > the same result as would require scanning every record of the table if an > ObjectPtr field was used. > > Question 2: Am I pretty close so far? Yes, only remember that ObjectPtr is indexed. So search will be the same - binary by index. ObjectPtr and Binary Link here are equal by speed. > In the reverse direction, find the mother of a given child, I can't conceive > of anything special the binary link could do that would be any faster than > the ObjectPtr case. Right, and now it is ieven slow, because we have implement for both direction only the general M M way. We still can improve things to get jump in director ToOne as fast and by ObjectPtr. > So for the M->1 "info" retained by the link I'm > guessing it simply does that. In ideal yes. > I.e. it just keeps a mapping for every record > that a link is defined for, which in the example above whever every record > has a valid MotherOf would essentially end up taking the same amount of > space as, and operating in the same amount of time as the ObjectPtr case. > > Question 3: Am I still on target with this supposition? Absolutely correct Ed! And now COMPARE: ObjectPtr and BinaryLink 1 : M 1 : M SPEED THE SAME SPACE on disk THE SAME But !!!!! 1) Binary link do not infect Structure of Table. 2) Binary Link can be easy converted tomorrow into M:M if your customer or you change mind. 3) IT IS possible for binary link add parameter, that DEVELOPER swear todo searches only in one direction, then we can even more win in space AND speed of modification. We need add all this docs asap :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 21:03:22 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 13:09:09 2005 Subject: IndexbyWords // Vstring.SplitToWords( text as string) In-Reply-To: Message-ID: On 11/16/05 7:30 PM, "Ed Kleban" wrote: >>>> Well, it is possible to check all this using V4RB exmaple >>>> >>>> Common/SplitToWords >>>> > > Great. I see that this app works by using the method > Vstring.SplitToWords( text as string) > Which does not appear to be documented in any of the manuals. > Is this a supported method we can rely on and use? Yes we have decide that it is useful for all. > I also note in the field creation the following comment: > > // To be able use function SplitToWords() the field must be indexed. > fld = tbl.CreateStringField( "fldString", 40, _ > EVFlag.fIndexed + EVFlag.fIndexByWords ) > > But I thought that fIndexed was meaningless when fIndexedByWords > was used. Does that mean that fIndexed is not needed here? > Or is SplitToWords somehow special? In Valentina 2.0 it is NOT enough say fIndexByWords !!!!!!!!! This will set only modifier index flag (fUnique the same) But you still can set ON/OFF fIndexed flag. ------- Also Valentina 2 DO NOT index unindexed field automatically ON SEARCH As Valentina 1 did. Valentina 2 still mark field as indexed if you do SORTING > Unfortunately it turns out that "words" as defined by SplitToString include > "." and exclude "_". I believe they follow natural language rules. > The later is unfortunate for my application. The > former seems to be unfortunate for most every application I can think of > that might want to otherwise use IndexedByWords. right > It's not good for prose, > verse, or code since "foo" and "foo." and ".foo" would all parse out to be > different search terms. That is unless a search for "foo" would return > entries that included all three of these (and other) combinations. I'll > have to go back and read the fine print on query for IndexedByWords fields. Add feature request :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 20:59:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 13:09:27 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 5:52 PM, "Ed Kleban" wrote: >>> They do effectively the same as you do in Relational model with help of >>> third Table to establish M:M link. >>> >>> What do this table? It just remember pairs (id1, id2) >>> >>> BinaryLink do the same, but use RecIDs and it is NOT a table. >>> It is more effective structure. >> >> Hmm, more effective than a table. Meaning not a linear list of pairs, or >> meaning not a list that's the same length as one of the two related tables. >> The former I guess. Hmm... Well, you could certainly make it more efficient >> by keeping two separate indexed lists so you could do a binary search in >> either direction. And you could make it more efficient with an indexed list >> that pointed to a "bucket" with a block of corresponding related entries so >> that when doing a binary search you didn't have the overhead of duplicates, >> and once you did the seach you had a linear list of corresponding records. >> Yeah, that's probably how I'd do it... if I could avoid the temptation of >> hashing the indexes to avoid the log2N cost of the binary search for large >> sets of links. > > Ok, I've had some time to sleep on this, and am getting excited! > And of course I have more questions. :) Great! I like how you think! > Let's run through an example. Say I have a table of people that has a > recursive relationship for "MotherOf" as a M:1 relationship. If I use an > ObjectPtr field for the MotherOf relationship, then finding the mother for > any given record is EXCEEDINGLY fast. I simply go to the MotherOf field, > and there's the record ID I need for the mother. And if I intend to record > a mother for every person in the table then the representation is also > exceedingly efficient. > > I could make it more space efficient, and possibly less speed efficient by > using a UShort field instead of an ObjectPtr if I knew, say I was only going > to have, say 20,000 records maximum. This will speed up also, because HDD -- main break in db. > Question 1: Would use of a UShort field here for 16-bit encoding > of RecID actually be slower than use of an ObjectPtr? NO. And we have think about giving to developer such ability to specify type of ObjectPtr. This bring few problems: - mistake of developer - type of ObjectPtr different, then how right correct code? And honestly was no time to go so deeply. But yes in Valentina 2 you can use FOREIGN KEY now, it works as link,and it have constraints. So in general you loose nothing here. ------------------------ > In the opposite direction, if I wanted to find all the children of a given > mother, it would take much more time since the DB would have to scan every > record to look for a matching value in the MotherOf field for a given mother > record RecId. For N records the order of performance of the linear search > would be o(N). But usually you have INDEX for such fields. And ObjectPtr is always indexed and FK fields also. ------------ > Now lets consider Binary Links. And since you've not yet commented on my > supposition above, let's assume that the implementation I postulated is > indeed close to what Valentina implements; Namely for a M:1 relationship it > keeps separate "info" for the "M->1" and the "1->M" directions, and that for > the "1->M" direction in fact the execution requires a single binary search > of an indexed list of, in this case known mother records, and results in > obtaining a list of all the associated children. In this direction the > performance is much better then because: The DB does a single binary search > on a small list of records known to be mothers and ends up immediately with > the same result as would require scanning every record of the table if an > ObjectPtr field was used. > > Question 2: Am I pretty close so far? Yes, only remember that ObjectPtr is indexed. So search will be the same - binary by index. ObjectPtr and Binary Link here are equal by speed. > In the reverse direction, find the mother of a given child, I can't conceive > of anything special the binary link could do that would be any faster than > the ObjectPtr case. Right, and now it is ieven slow, because we have implement for both direction only the general M M way. We still can improve things to get jump in director ToOne as fast and by ObjectPtr. > So for the M->1 "info" retained by the link I'm > guessing it simply does that. In ideal yes. > I.e. it just keeps a mapping for every record > that a link is defined for, which in the example above whever every record > has a valid MotherOf would essentially end up taking the same amount of > space as, and operating in the same amount of time as the ObjectPtr case. > > Question 3: Am I still on target with this supposition? Absolutely correct Ed! And now COMPARE: ObjectPtr and BinaryLink 1 : M 1 : M SPEED THE SAME SPACE on disk THE SAME But !!!!! 1) Binary link do not infect Structure of Table. 2) Binary Link can be easy converted tomorrow into M:M if your customer or you change mind. 3) IT IS possible for binary link add parameter, that DEVELOPER swear todo searches only in one direction, then we can even more win in space AND speed of modification. We need add all this docs asap :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 21:20:08 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 13:30:35 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 9:04 PM, "Ed Kleban" wrote: > Yeah, well, you're making a convert out of me on that approach pretty > quickly. I've just finished a first pass at re-diagramming the database I'm > designing using Binary Links instead of ObjectPtr fields. It's amazing! > > One question that has come up, is that if I want to associate some type of a > typeCode that is a small byte constant that identifies the type of an item, > and I need to access that typeCode all the time for items to do select case > statements and such, and I also want to keep an anciallary table with > additional information about that typeCode, then should I: That is the problem. BinaryLink cannot have associated information. For this it needs to use old good MM tables. I still cannot find way how to add this association...effectively. > 1) Both create a Binary link between item and their typeInfo records, where > the recID is inherently a representation for the typeCd, and then > additionally allocate a byte TypeCode field in the item records, under the > theory that I can grab the typeCode very fast most of the time, and then do > an indexed lookup by record ID when I occasionally need the extra type > info... > > or > > 2) Punt the darn typeCode byte in the item record because binary links are > so blazingly fast in the many to 1 direction that that having a typeCode > byte would be no faster and just make for messy double book keeping. If you can put it into ONE table then I think good And keep in mind, right now M to One is not ideal speed. > or > > 3) Right idea, but if you really want the blazingly fast connect from the > typeCode in the item record to the typeInfo record then use a full Ulong for > the typeCode as an OffsetPtr because Valentina will optimize accesses and be > more efficient using the ObjectPtr than converting from a byte index. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 16 14:22:04 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 14:22:59 2005 Subject: Index Styles and such In-Reply-To: Message-ID: Do index styles work with fields that are indexedByWords so that I can specify a list of words to be excluded from indexing? How much performance loss does Valentina incur for processing char fields stored on disk as UTF-8 since it has to convert them to UTF-16 every time it does an search or comparison? If the string "aaaa bbbb aaaa bbbb " is indexedByWords, then presumably there is only a single index entry for "aaaa", not two? Thus its necessary to parse lines found on a match to scan for multiple occurrences of the search string? From Ed at Kleban.com Wed Nov 16 15:25:39 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 15:26:32 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: > >> Let's run through an example. Say I have a table of people that has a >> recursive relationship for "MotherOf" as a M:1 relationship. If I use an >> ObjectPtr field for the MotherOf relationship, then finding the mother for >> any given record is EXCEEDINGLY fast. I simply go to the MotherOf field, >> and there's the record ID I need for the mother. And if I intend to record >> a mother for every person in the table then the representation is also >> exceedingly efficient. >> >> I could make it more space efficient, and possibly less speed efficient by >> using a UShort field instead of an ObjectPtr if I knew, say I was only going >> to have, say 20,000 records maximum. > > This will speed up also, because HDD -- main break in db. > >> Question 1: Would use of a UShort field here for 16-bit encoding >> of RecID actually be slower than use of an ObjectPtr? > > NO. > > And we have think about giving to developer such ability to specify type of > ObjectPtr. This bring few problems: > - mistake of developer > - type of ObjectPtr different, then how right correct code? > > And honestly was no time to go so deeply. Well, I suppose you could have 3 types of ObjectPtr: LongPtr, ShortPtr, BytePtr. Furthermore, you could put a "MaxRecords" constraint on tables that you intended index by ShortPtr or BytePtr. You already have constraint checking to support record additions in the case of a Unique field for example. A "too many records" check would be very fast. Similarly, if the user did indeed blow it, worst case is that the indexes would wrap modulo word or byte boundaries and refer to the wrong record. Which is unfortunate, but not particularly dangerous in that it could point to a non-existant record that has not been allocated. In fact, this would be no worse of an error than would result if a user claimed that a binary link was 1->M or M->1 or 1->1 when it was in fact M->M. If they supply the wrong assertions, they'll get the wrong results as the system endeavors to optimize search on their behalf. > But yes in Valentina 2 you can use FOREIGN KEY now, it works as link,and it > have constraints. So in general you loose nothing here. You say "lose nothing here", and in terms of functionality I agree. What I have not seen you state clearly yet however is that "Because of the way links are implemented in Valentina, if you use say a single-byte or short-word key field and use as a relation link to a recId, then the performance is just as fast as if you had used and ObjectPtr." Question: Is that the case? Or is there a performance penalty for using foreign key fields in this manner? > ------------------------ >> In the opposite direction, if I wanted to find all the children of a given >> mother, it would take much more time since the DB would have to scan every >> record to look for a matching value in the MotherOf field for a given mother >> record RecId. For N records the order of performance of the linear search >> would be o(N). > > But usually you have INDEX for such fields. And ObjectPtr is always indexed > and FK fields also. I don't see how that makes any difference or helps. If I want to find all the children of a mother record and I'm using a an ObjectPtr or KeyField and not using a binary Link, then doesn't V4RB have to examine every single record to find out if it is a child of a given mother? Ah, No it does not! As I finally figured out and discussed below. > ------------ >> Now lets consider Binary Links. And since you've not yet commented on my >> supposition above, let's assume that the implementation I postulated is >> indeed close to what Valentina implements; Namely for a M:1 relationship it >> keeps separate "info" for the "M->1" and the "1->M" directions, and that for >> the "1->M" direction in fact the execution requires a single binary search >> of an indexed list of, in this case known mother records, and results in >> obtaining a list of all the associated children. In this direction the >> performance is much better then because: The DB does a single binary search >> on a small list of records known to be mothers and ends up immediately with >> the same result as would require scanning every record of the table if an >> ObjectPtr field was used. >> >> Question 2: Am I pretty close so far? > > Yes, only remember that ObjectPtr is indexed. So search will be the same - > binary by index. > > ObjectPtr and Binary Link here are equal by speed. No... I'm making some assumptions that imply that Binary links should be faster than ObjectPtr links for this case. I'm doing that so that you can tell me that yes my assumptions are true, or that no my assumption is false. My assumptions are that... Ah, never mind. I get it. (I think by writing, so I had to write a couple paragraphs that I just erased). In essence, the reason that a BinaryLink is equal to ObjectPtr in performance is because doing a binary search on an indexed objectPtr field containing recId of parent records IS an implementation of the exact approach I proposed for doing a single binary lookup leading to a table of children. I get it. For the record, however there may be a tiny difference between the use of an ObjectPtr field and a BinaryLlink in that the number of records you're having to do a binary search on is potentially smaller in the case of a BinaryLink, whereas in the ObjectPtr link case even though you're doing a binary search it's still over a span of length equal to the total number of records. So in retrospect, I've been making this more complicated than it really is. A binary link is simply a table of ordered pairs of recIds where both sides are indexed. Thus it is equally efficient to find all the children of a parent or ... what may in fact be multiple parents of a child. That's why even though I keep using the terms "1->M" and "M->1" you keep typing in reply "M->M". The binary link is inherently a "M->M" relationship and use of it as "M->1", "1->M", or "1->" are simply special usage side cases. The reason you want the user to supply whether a given side is "one" or 'Many" is so that you know whether you can stop your search or processing of an index after you find the first occurrence, or whether you need to look for more. > >> In the reverse direction, find the mother of a given child, I can't conceive >> of anything special the binary link could do that would be any faster than >> the ObjectPtr case. > > Right, and now it is ieven slow, because we have implement for both > direction only the general M M way. We still can improve things to get jump > in director ToOne as fast and by ObjectPtr. Ah! There it is in black and white. Just what I said. >> So for the M->1 "info" retained by the link I'm >> guessing it simply does that. > > In ideal yes. > >> I.e. it just keeps a mapping for every record >> that a link is defined for, which in the example above whever every record >> has a valid MotherOf would essentially end up taking the same amount of >> space as, and operating in the same amount of time as the ObjectPtr case. >> >> Question 3: Am I still on target with this supposition? > > Absolutely correct Ed! > > And now COMPARE: > > ObjectPtr and BinaryLink > > 1 : M 1 : M > > SPEED THE SAME > > SPACE on disk THE SAME > > But !!!!! > > 1) Binary link do not infect Structure of Table. > > 2) Binary Link can be easy converted tomorrow into M:M if your customer or > you change mind. > > 3) IT IS possible for binary link add parameter, that DEVELOPER swear > todo searches only in one direction, then we can even more win > in space AND speed of modification. > You'd save in space because you'd only need to create an index for one half of the linkTable, not both sides. You'd save in speed because... hmm. ...because you could eliminate the check for finding additional records after you've found the first one on a "->1" link. > We need add all this docs asap :-) > From Ed at Kleban.com Wed Nov 16 15:37:20 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 16 15:38:16 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 1:20 PM, "Ruslan Zasukhin" wrote: > On 11/16/05 9:04 PM, "Ed Kleban" wrote: > >> Yeah, well, you're making a convert out of me on that approach pretty >> quickly. I've just finished a first pass at re-diagramming the database I'm >> designing using Binary Links instead of ObjectPtr fields. It's amazing! >> >> One question that has come up, is that if I want to associate some type of a >> typeCode that is a small byte constant that identifies the type of an item, >> and I need to access that typeCode all the time for items to do select case >> statements and such, and I also want to keep an anciallary table with >> additional information about that typeCode, then should I: > > That is the problem. BinaryLink cannot have associated information. > For this it needs to use old good MM tables. > > I still cannot find way how to add this association...effectively. > > >> 1) Both create a Binary link between item and their typeInfo records, where >> the recID is inherently a representation for the typeCd, and then >> additionally allocate a byte TypeCode field in the item records, under the >> theory that I can grab the typeCode very fast most of the time, and then do >> an indexed lookup by record ID when I occasionally need the extra type >> info... >> >> or >> >> 2) Punt the darn typeCode byte in the item record because binary links are >> so blazingly fast in the many to 1 direction that that having a typeCode >> byte would be no faster and just make for messy double book keeping. > > If you can put it into ONE table then I think good > > And keep in mind, right now M to One is not ideal speed. So you've said. But you haven't really explained why or what speed is to be gained by optimizing. > >> or >> >> 3) Right idea, but if you really want the blazingly fast connect from the >> typeCode in the item record to the typeInfo record then use a full Ulong for >> the typeCode as an OffsetPtr because Valentina will optimize accesses and be >> more efficient using the ObjectPtr than converting from a byte index. Now that I understand, and understand why, that performance of a BinaryLink is the same as an ObjectPtr I'd probably go this 3rd route. It would require more space for the ObjectPtr than the Byte field, but I'd end up with optimal performance for all possible operations. And when you implement BytePtr in Valentina 3 I'll get the wasted space back :) From sunshine at public.kherson.ua Wed Nov 16 23:36:27 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:38:26 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:25 PM, "Ed Kleban" wrote: >>> Question 1: Would use of a UShort field here for 16-bit encoding >>> of RecID actually be slower than use of an ObjectPtr? >> >> NO. >> >> And we have think about giving to developer such ability to specify type of >> ObjectPtr. This bring few problems: >> - mistake of developer >> - type of ObjectPtr different, then how right correct code? >> >> And honestly was no time to go so deeply. > > Well, I suppose you could have 3 types of ObjectPtr: LongPtr, ShortPtr, > BytePtr. Furthermore, you could put a "MaxRecords" constraint on tables > that you intended index by ShortPtr or BytePtr. You already have constraint > checking to support record additions in the case of a Unique field for > example. A "too many records" check would be very fast. GOOD IDEA! > Similarly, if the > user did indeed blow it, worst case is that the indexes would wrap modulo > word or byte boundaries and refer to the wrong record. Which is > unfortunate, but not particularly dangerous in that it could point to a > non-existant record that has not been allocated. > > In fact, this would be no worse of an error than would result if a user > claimed that a binary link was 1->M or M->1 or 1->1 when it was in fact > M->M. If they supply the wrong assertions, they'll get the wrong results as > the system endeavors to optimize search on their behalf. No no. if you have specify for Link 1 : M for example, Then Valentina will watch for integrity and refuse wrong operations. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:42:28 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:43:45 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:25 PM, "Ed Kleban" wrote: >> But yes in Valentina 2 you can use FOREIGN KEY now, it works as link,and it >> have constraints. So in general you loose nothing here. > > You say "lose nothing here", and in terms of functionality I agree. What I > have not seen you state clearly yet however is that "Because of the way > links are implemented in Valentina, if you use say a single-byte or > short-word key field and use as a relation link to a recId, then the > performance is just as fast as if you had used and ObjectPtr." > > Question: Is that the case? Or is there a performance penalty for > using foreign key fields in this manner? I remember that join on FK is slower than on ObjectPtr. Because it needs jump into Key field. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:44:02 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:45:01 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:25 PM, "Ed Kleban" wrote: >> And now COMPARE: >> >> ObjectPtr and BinaryLink >> >> 1 : M 1 : M >> >> SPEED THE SAME >> >> SPACE on disk THE SAME >> >> But !!!!! >> >> 1) Binary link do not infect Structure of Table. >> >> 2) Binary Link can be easy converted tomorrow into M:M if your customer or >> you change mind. >> >> 3) IT IS possible for binary link add parameter, that DEVELOPER swear >> todo searches only in one direction, then we can even more win >> in space AND speed of modification. >> > > You'd save in space because you'd only need to create an index for one half > of the linkTable, not both sides. > > You'd save in speed because... hmm. ...because you could eliminate the > check for finding additional records after you've found the first one on a > "->1" link. Speed of MODIFICATION. Add/Delete Again because only half todo -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:51:21 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:52:29 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:25 PM, "Ed Kleban" wrote: > So in retrospect, I've been making this more complicated than it really is. > A binary link is simply a table of ordered pairs of recIds where both sides > are indexed. Thus it is equally efficient to find all the children of a > parent or ... what may in fact be multiple parents of a child. That's why > even though I keep using the terms "1->M" and "M->1" you keep typing in > reply "M->M". The binary link is inherently a "M->M" relationship and use > of it as "M->1", "1->M", or "1->" are simply special usage side cases. The > reason you want the user to supply whether a given side is "one" or 'Many" > is so that you know whether you can stop your search or processing of an > index after you find the first occurrence, or whether you need to look for > more. Right, additional info is additional constraints additional possible improvements -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:32:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:55:41 2005 Subject: Index Styles and such In-Reply-To: Message-ID: On 11/16/05 10:22 PM, "Ed Kleban" wrote: > Do index styles work with fields that are indexedByWords so that I can > specify a list of words to be excluded from indexing? YES. Read docs. Index Stlyes only and can do this -- have black list :-) > How much performance loss does Valentina incur for processing char fields > stored on disk as UTF-8 since it has to convert them to UTF-16 every time it > does an search or comparison? Stop. UTF8 encoding for disk is not solid. Do not use it. CPU conversion of strings is only 1% But reduce of db size in 2 times can give you 50% of win, But NOT works now. > If the string "aaaa bbbb aaaa bbbb " is indexedByWords, then presumably > there is only a single index entry for "aaaa", not two? Right > Thus its necessary > to parse lines found on a match to scan for multiple occurrences of the > search string? If you need this -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:57:59 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 15:58:56 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:37 PM, "Ed Kleban" wrote: >> If you can put it into ONE table then I think good >> >> And keep in mind, right now M to One is not ideal speed. > > So you've said. But you haven't really explained why or what speed is to be > gained by optimizing. For direction M to one, we should use not current index-like method, But mapping like do ObjectPtr. You have describe this in your text in fact. :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 16 23:59:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 16 16:01:23 2005 Subject: Binary links Re: Question Backlog for Valentina mailing list. In-Reply-To: Message-ID: On 11/16/05 11:37 PM, "Ed Kleban" wrote: >>> 3) Right idea, but if you really want the blazingly fast connect from the >>> typeCode in the item record to the typeInfo record then use a full Ulong for >>> the typeCode as an OffsetPtr because Valentina will optimize accesses and be >>> more efficient using the ObjectPtr than converting from a byte index. > > Now that I understand, and understand why, that performance of a BinaryLink > is the same as an ObjectPtr I'd probably go this 3rd route. It would > require more space for the ObjectPtr than the Byte field, but I'd end up > with optimal performance for all possible operations. And when you > implement BytePtr in Valentina 3 I'll get the wasted space back :) So it sounds like you going to have Table(s) with less than 255 records ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Wed Nov 16 17:26:18 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Wed Nov 16 16:27:11 2005 Subject: [V4Rb] VDatabase.LoadDump problem Message-ID: I'm finding that VDatabase.LoadDump causes my app to crash. I'm dumping a database to an XML file and loading it into a new database. Are there known problems with LoadDump? -------------- Charles Yeomans From gaseous1 at mac.com Wed Nov 16 16:50:01 2005 From: gaseous1 at mac.com (Eric Werner, MD) Date: Wed Nov 16 16:51:16 2005 Subject: v4RB2, DB create/open and subclasses In-Reply-To: Message-ID: Ruslan, Using v4RB2 v2.05 and REALbasic 2005, I?m trying to create or open a Valentina database using the ?classes? technique. The wrinkle is that I?m using a subclass of Vdatabase, to which I have added a variety of properties and methods (this class is called vDB). I create another object, whose super is vDB, named vbScheduler, which creates the database tables. Yet another object, whose super is vTable, creates the field entries. This is all coded as per the tutorial (and a similar project worked flawlessly in VB v1.x) When I run this project, an incomplete (db with tables, but without fields) database is created on disk. What do I need to do differently to get this to work. I have a simple file which demonstrates my problem, if you?d like to see it. -- Regards, Eric From Ed at Kleban.com Thu Nov 17 01:28:31 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 17 01:29:29 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: I see that there are casting functions for the VCursor class, which I understand are for moving values between a Cursor and a Table's Vfields. But what I don't find are any functions to assist with getting 8-byte values into and out of RB variables. For example, How do I get the high vs low four bytes of an LLong or ULong field? The only mechanism I see for this is GetString an SetString, which is a bit perplexing. I use Valentina to do a super fast lookup of my data to get it into a current Table record or into a Cursor, but then I have to convert a binary value to a string which I then have to chop up with string operations and convert back to a binary value to use? Yuck! ... or am I missing something here? How about some numeric accessor methods for VFields in general, or at least the 8-byte fields specifically: rbIntVar = aVULongField.highLong rbIntVar = aVULongField.lowLong rbIntVar = aVLongField.hightLong rbIntVar = aVLongField.lowLong Admittedly, most or all of these will be moving an unsigned 32-bit value into what RB would treat as a signed integer, but at least we could rapidly read and then "do the math" on binary values without having to go through strings. And who knows... considering all the database announcements coming out of REALsoftware, someday they might eventually implement size and representation specific variables such as Int8, Int16, Int32, UInt8, UInt16, and UInt32, and Byte; then you wouldn't even have to do messy math. Heck, they may even implement Int64 and UInt64 someday in which case you could read, use, compare, and write the whole sucker directly. Also, for dates and times: rbDateVar = aVDateTimeField.rbDateValue aVDateTimeField.setFromRbDateValue( rbDateValue ) By the way... If it should happen that in some distant year in the future that the REALfolk did indeed implement variables such as Int8, Int16, Int32, UInt8, UInt16, and UInt32, Byte, Int64 and UInt64 in the language, what other advantages could Valentina make of such an addition? From sunshine at public.kherson.ua Thu Nov 17 10:26:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 02:27:20 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 9:28 AM, "Ed Kleban" wrote: Hi Ed, > I see that there are casting functions for the VCursor class, which I > understand are for moving values between a Cursor and a Table's Vfields. Mmmmm actually this is to write one line of code instead of 2 fldLong = curs.Longfield( I ) vs fld = curs.field( I ) fldLong = Vlong( fld ) > But what I don't find are any functions to assist with getting 8-byte values > into and out of RB variables. Moment. FYI. REALbasic send/get to plugin always int = 4 bytes IF you put this into Valentina's Vbyte field, then on disk will be use ONE byte only > For example, How do I get the high vs low four bytes of an LLong or ULong > field? The only mechanism I see for this is GetString an SetString, which > is a bit perplexing. I use Valentina to do a super fast lookup of my data > to get it into a current Table record or into a Cursor, but then I have to > convert a binary value to a string which I then have to chop up with string > operations and convert back to a binary value to use? > > Yuck! ... or am I missing something here? I have not catch yet what you want > How about some numeric accessor methods for VFields in general, or at least > the 8-byte fields specifically: > rbIntVar = aVULongField.highLong > rbIntVar = aVULongField.lowLong > rbIntVar = aVLongField.hightLong > rbIntVar = aVLongField.lowLong Not agree. We give you 8 bytes, which we have extract from Table. You self can get any bit as you want using YOUR language. If REALbasic is not able do this, I think exists plugins which can. IT is not Valentina task split to bits > Admittedly, most or all of these will be moving an unsigned 32-bit value > into what RB would treat as a signed integer, but at least we could rapidly > read and then "do the math" on binary values without having to go through > strings. > > And who knows... considering all the database announcements coming out of > REALsoftware, someday they might eventually implement size and > representation specific variables such as Int8, Int16, Int32, UInt8, UInt16, > and UInt32, and Byte; then you wouldn't even have to do messy math. > > Heck, they may even implement Int64 and UInt64 someday in which case you > could read, use, compare, and write the whole sucker directly. > > Also, for dates and times: > > rbDateVar = aVDateTimeField.rbDateValue > aVDateTimeField.setFromRbDateValue( rbDateValue ) Yes we going add support of RB Date > By the way... If it should happen that in some distant year in the future > that the REALfolk did indeed implement variables such as Int8, Int16, Int32, > UInt8, UInt16, and UInt32, Byte, Int64 and UInt64 in the language, what > other advantages could Valentina make of such an addition? Just we will return you this types :-) Although you right now can use UNSIGNED fields. Problem is that YOU in RB cannot reach top bit to get correct range -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 17 10:33:34 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 02:34:30 2005 Subject: [V4Rb] VDatabase.LoadDump problem In-Reply-To: Message-ID: On 11/17/05 12:26 AM, "Charles Yeomans" wrote: Hi Charles, > I'm finding that VDatabase.LoadDump causes my app to crash. I'm > dumping a database to an XML file and loading it into a new database. > Are there known problems with LoadDump? To crash, It seems no. * Do you have crash log ? * can you send us XML ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 17 10:41:12 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 02:42:06 2005 Subject: v4RB2, DB create/open and subclasses In-Reply-To: Message-ID: On 11/17/05 12:50 AM, "Eric Werner, MD" wrote: Hi Eric, > Using v4RB2 v2.05 and REALbasic 2005, I?m trying to create or open a > Valentina database using the ?classes? technique. > The wrinkle is that I?m using a subclass of Vdatabase, to which I have added > a variety of properties and methods (this class is called vDB). This is okay! > I create another object, whose super is vDB, named vbScheduler, which > creates the database tables. Here is important to call in constructor parent's constructor like do our exmaples > Yet another object, whose super is vTable, > creates the field entries. This is all coded as per the tutorial (and a > similar project worked flawlessly in VB v1.x) > When I run this project, an incomplete (db with tables, but without fields) > database is created on disk. > > What do I need to do differently to get this to work. I have a simple file > which demonstrates my problem, if you?d like to see it. Check db and table constructors Valentina 2 require from you to call Vdatabase and Vtable's one -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Thu Nov 17 03:17:22 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 17 03:18:16 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 2:26 AM, "Ruslan Zasukhin" wrote: > On 11/17/05 9:28 AM, "Ed Kleban" wrote: > > Hi Ed, > >> I see that there are casting functions for the VCursor class, which I >> understand are for moving values between a Cursor and a Table's Vfields. > > Mmmmm actually this is to write one line of code instead of 2 > > fldLong = curs.Longfield( I ) > > vs > > fld = curs.field( I ) > fldLong = Vlong( fld ) > Ah. Makes sense. I hadn't thought it through but presumed it was for something like that. This would be a good statement to add to the manual. >> But what I don't find are any functions to assist with getting 8-byte values >> into and out of RB variables. > > Moment. FYI. > > REALbasic send/get to plugin always int = 4 bytes > > IF you put this into Valentina's Vbyte field, then on disk will be use ONE > byte only > Yes, exactly what I expected. That's fine. >> How about some numeric accessor methods for VFields in general, or at least >> the 8-byte fields specifically: > >> rbIntVar = aVULongField.highLong >> rbIntVar = aVULongField.lowLong >> rbIntVar = aVLongField.hightLong >> rbIntVar = aVLongField.lowLong > > Not agree. > > We give you 8 bytes, which we have extract from Table. > > You self can get any bit as you want using YOUR language. > If REALbasic is not able do this, I think exists plugins which can. > > IT is not Valentina task split to bits > Fair enough, especially since I expect there's some simple way to do this that's not apparent to me. Possibly by mapping a Memory block pointer to the VField or something. I'll run this by Thomas Templeton or the NUG list on RB. Or I see Charles Yeoman is on this list. He'll likely know. >> Admittedly, most or all of these will be moving an unsigned 32-bit value >> into what RB would treat as a signed integer, but at least we could rapidly >> read and then "do the math" on binary values without having to go through >> strings. >> >> And who knows... considering all the database announcements coming out of >> REALsoftware, someday they might eventually implement size and >> representation specific variables such as Int8, Int16, Int32, UInt8, UInt16, >> and UInt32, and Byte; then you wouldn't even have to do messy math. >> >> Heck, they may even implement Int64 and UInt64 someday in which case you >> could read, use, compare, and write the whole sucker directly. >> >> Also, for dates and times: >> >> rbDateVar = aVDateTimeField.rbDateValue >> aVDateTimeField.setFromRbDateValue( rbDateValue ) > > Yes we going add support of RB Date > >> By the way... If it should happen that in some distant year in the future >> that the REALfolk did indeed implement variables such as Int8, Int16, Int32, >> UInt8, UInt16, and UInt32, Byte, Int64 and UInt64 in the language, what >> other advantages could Valentina make of such an addition? > > Just we will return you this types :-) > > Although you right now can use UNSIGNED fields. > Problem is that YOU in RB cannot reach top bit to get correct range Uh, sure you can. The top bit is there, but since RB treats all ints as signed you have to be cleaver about interpreting what it is. i.e. if the number is negative, the top bit is set and you can do some 1's compliment arithmetic if needed. That's what I meant above by "do the math". But provision of the typed Integers would solve that problem in theory someday. From yeomans at desuetude.com Thu Nov 17 11:52:27 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 17 10:53:26 2005 Subject: [V4Rb] VDatabase.LoadDump problem In-Reply-To: References: Message-ID: On Nov 17, 2005, at 3:33 AM, Ruslan Zasukhin wrote: > On 11/17/05 12:26 AM, "Charles Yeomans" wrote: > > Hi Charles, > >> I'm finding that VDatabase.LoadDump causes my app to crash. I'm >> dumping a database to an XML file and loading it into a new database. >> Are there known problems with LoadDump? > > To crash, It seems no. > > * Do you have crash log ? No crash log was generated. > > * can you send us XML ? Oops; I forgot to examine the XML. I'll send it along. -------------- Charles Yeomans From yeomans at desuetude.com Thu Nov 17 11:55:17 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 17 10:56:11 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: References: Message-ID: <7464d3e7ee21944967b0dd31181f64d1@desuetude.com> On Nov 17, 2005, at 2:28 AM, Ed Kleban wrote: > > I see that there are casting functions for the VCursor class, which I > understand are for moving values between a Cursor and a Table's > Vfields. > > But what I don't find are any functions to assist with getting 8-byte > values > into and out of RB variables. > > For example, How do I get the high vs low four bytes of an LLong or > ULong > field? The only mechanism I see for this is GetString an SetString, > which > is a bit perplexing. I use Valentina to do a super fast lookup of my > data > to get it into a current Table record or into a Cursor, but then I > have to > convert a binary value to a string which I then have to chop up with > string > operations and convert back to a binary value to use? > > Yuck! ... or am I missing something here? I think you're working too hard. Valentina is extremely fast, so perhaps you might first write simpler code and Valentina supply the speed. -------------- Charles Yeomans From yeomans at desuetude.com Thu Nov 17 12:20:52 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 17 11:21:46 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: References: Message-ID: On Nov 17, 2005, at 4:17 AM, Ed Kleban wrote: > > > > On 11/17/05 2:26 AM, "Ruslan Zasukhin" > wrote: > >> On 11/17/05 9:28 AM, "Ed Kleban" wrote: >> >> Hi Ed, >> >>> I see that there are casting functions for the VCursor class, which I >>> understand are for moving values between a Cursor and a Table's >>> Vfields. >> >> Mmmmm actually this is to write one line of code instead of 2 >> >> fldLong = curs.Longfield( I ) >> >> vs >> >> fld = curs.field( I ) >> fldLong = Vlong( fld ) >> > > Ah. Makes sense. I hadn't thought it through but presumed it was for > something like that. This would be a good statement to add to the > manual. > >>> But what I don't find are any functions to assist with getting >>> 8-byte values >>> into and out of RB variables. >> >> Moment. FYI. >> >> REALbasic send/get to plugin always int = 4 bytes >> >> IF you put this into Valentina's Vbyte field, then on disk will be >> use ONE >> byte only >> > > Yes, exactly what I expected. That's fine. > >>> How about some numeric accessor methods for VFields in general, or >>> at least >>> the 8-byte fields specifically: >> >>> rbIntVar = aVULongField.highLong >>> rbIntVar = aVULongField.lowLong >>> rbIntVar = aVLongField.hightLong >>> rbIntVar = aVLongField.lowLong >> >> Not agree. >> >> We give you 8 bytes, which we have extract from Table. >> >> You self can get any bit as you want using YOUR language. >> If REALbasic is not able do this, I think exists plugins which can. >> >> IT is not Valentina task split to bits >> > > Fair enough, especially since I expect there's some simple way to do > this > that's not apparent to me. Possibly by mapping a Memory block pointer > to > the VField or something. I'll run this by Thomas Templeton or the NUG > list > on RB. Or I see Charles Yeoman is on this list. He'll likely know. It's not hard to get the four-byte pieces. Take the string returned from a LLong or ULLong field, stick it into an eight-byte MemoryBlock, then read out the pieces using MemoryBlock.Long. For MacOS, it would be simple enough to write U64 and S64 classes that use the OS support for 64-bit integers, and make them work with Valentina. I have some classes half-written already. This wouldn't be a fast solution, but it would work. -------------- Charles Yeomans From Ed at Kleban.com Thu Nov 17 11:53:34 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 17 11:54:33 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 11:20 AM, "Charles Yeomans" wrote: > >> Yuck! ... or am I missing something here? > > I think you're working too hard. Valentina is extremely fast, so > perhaps you might first write simpler code and Valentina supply the > speed. No, the problem is I'm not working at all. Or rather I'm not writing the code that does this yet, I'm merely thinking about it during the design stage while reading and learning about Valentina. The concerns expressed in my email lack the benefit of grounding in real application usage. I appreciate the words of wisdom coming from your "voice of experience" suggesting that my concern is basically a non-issue. That's a good thing! >> >> Fair enough, especially since I expect there's some simple way to do >> this >> that's not apparent to me. Possibly by mapping a Memory block pointer >> to >> the VField or something. I'll run this by Thomas Templeton or the NUG >> list >> on RB. Or I see Charles Yeoman is on this list. He'll likely know. > > It's not hard to get the four-byte pieces. Take the string returned > from a LLong or ULLong field, stick it into an eight-byte MemoryBlock, > then read out the pieces using MemoryBlock.Long. Ah! So the string returned by GetString is a binary string. Doh! Of course it is. I was thinking it would be a printable numeric for some stupid reason. That's the problem with having at least 5 overloaded uses for the term "string". Ok, this is the clue I was missing, and the exact MemoryBlock usage details I knew had to exist but wasn't too clueful about. Thanks Charles! > For MacOS, it would > be simple enough to write U64 and S64 classes that use the OS support > for 64-bit integers, and make them work with Valentina. I have some > classes half-written already. This wouldn't be a fast solution, but it > would work. > Naw, only thing I care about in this particular case is speed. Also, I predict it won't be long before the REALfolk implement Int64 and UInt64 which will offer another solution that hopefully is fast. Presumably a cast of Integer = UInt64 would truncate the high-order word, and shifting the bits right 32 position via a divide for example for a UInt64 would provide the high-order word. But the MemoryBlock solution is all I need for now. Thanks! BTW, I use and love your ArraySorter. > -------------- > Charles Yeomans > From maxprog at mac.com Thu Nov 17 18:56:15 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 17 11:57:13 2005 Subject: [V4RB] VComponents path Message-ID: <0470CDA1-3B76-451E-BC6D-6AF53413C7A7@mac.com> Hi, How can I set the VComponent path? I don't feel dropping a lot of junk to my application product folder is to even consider right now. Is there something I don't know? TIA, ~/Stan From Claudius at sailer-online.de Thu Nov 17 19:59:30 2005 From: Claudius at sailer-online.de (Claudius Sailer) Date: Thu Nov 17 13:00:28 2005 Subject: [V4RB] VComponents path In-Reply-To: <0470CDA1-3B76-451E-BC6D-6AF53413C7A7@mac.com> References: <0470CDA1-3B76-451E-BC6D-6AF53413C7A7@mac.com> Message-ID: <43915BBE-15CA-4E7F-AAB5-53C6EEEEB802@sailer-online.de> Hi, Am 17. Nov 2005 um 18:56 Uhr schrieb Stan Busk: > Hi, > > How can I set the VComponent path? I don't feel dropping a lot of > junk to my application product folder is to even consider right > now. Is there something I don't know? I am also waiting for a better solution. Ruslan said that they will realize that VComponents could be inside of an MacOS X package, when your application is a package. bye Claudius -- G4/733 QS / MacOS X 10.4de / RB 2005r4/ Valentina 1.10.0 & 2.0.4 Homepage http://www.ClaSai.de iChat ryhoruk RealBasic ListBoxes: [ I feel the need...the need for speed!!! ] From maxprog at mac.com Thu Nov 17 21:01:56 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 17 14:02:55 2005 Subject: [V4RB] VComponents path In-Reply-To: <43915BBE-15CA-4E7F-AAB5-53C6EEEEB802@sailer-online.de> References: <0470CDA1-3B76-451E-BC6D-6AF53413C7A7@mac.com> <43915BBE-15CA-4E7F-AAB5-53C6EEEEB802@sailer-online.de> Message-ID: <413C1412-2B7B-4C9E-8FB3-D3CE602F3E81@mac.com> I personally think Mac 'installations' should be *CLEAN*. The application folder *SHOULD NEVER* include a big wad of libraries as I see it is suggested again and again in that list. I personally think is it unbelievable and astounding somebody can suggest or even consider asking us to do that! How could Valentina developers suggest that? I develop on Windows as well, no problem there, you create an installer and place the files anywhere you want, the end-user doesn't mind. Mac users have a direct access to your product folder. Most Mac developer are very minimalists, you open the app folder and you see 2 or 3 files. Many developers have the help built-in just to be as minimalist and straightforward as possible. Valentina developers say that all of a sudden our applications have to look like PC folders full of junk... I am so annoyed. I can't create bundles because I think it is too soon right now, I am still PEFed. Bundles for RB developers will maybe be usual in 6 months or so, with macho and those universal binaries. Really awful... Some days I wonder I did not make a mistake choosing Valentina 3 years ago. Not only my apps are now a lot bigger, now they also look crappy. Next step will be to use an alias instead of the application to hide Valentina junk. I fill personally molested when they ask me to do things like if I were using a PC. A PC is a PC, great but please a little respect about Mac people want the things done. I haven't purchased the last G5 Quad to use it as a PC box, really! Please heed and flag this request with maximum priority, so many Mac developers will be really happy then! ~/Stan > Hi, > > > Am 17. Nov 2005 um 18:56 Uhr schrieb Stan Busk: > >> Hi, >> >> How can I set the VComponent path? I don't feel dropping a lot of >> junk to my application product folder is to even consider right >> now. Is there something I don't know? > > I am also waiting for a better solution. Ruslan said that they will > realize that VComponents could be inside of an MacOS X package, > when your application is a package. > > > bye > > Claudius > > -- > G4/733 QS / MacOS X 10.4de / RB 2005r4/ Valentina 1.10.0 & 2.0.4 > Homepage http://www.ClaSai.de > iChat ryhoruk > RealBasic ListBoxes: [ I feel the need...the need for speed!!! ] > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Thu Nov 17 23:08:06 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 15:09:00 2005 Subject: [V4RB] VComponents path In-Reply-To: <413C1412-2B7B-4C9E-8FB3-D3CE602F3E81@mac.com> Message-ID: On 11/17/05 10:01 PM, "Stan Busk" wrote: Hi Stan, > I personally think Mac 'installations' should be *CLEAN*. The > application folder *SHOULD NEVER* include a big wad of libraries as I > see it is suggested again and again in that list. I personally think > is it unbelievable and astounding somebody can suggest or even > consider asking us to do that! > How could Valentina developers suggest that? Easy. Because MAC OS force you put dlls only in 3 locations: 1) App folder 2) Library/CFMSupport 3) Home/SFMSupport IF you will put dlls into subfolder in this locations. Problems. MacOS do not find them and do not load. > I develop on Windows as well, no problem there, you create an > installer and place the files anywhere you want, the end-user doesn't > mind. Dlls ??? You put them again 1) in app folder 2) into ProgramFiles/MyCompany/MyProduct In last case you ALSO must set PATH in that computer to MyProduct folder, right ? > Mac users have a direct access to your product folder. Most Mac > developer are very minimalists, you open the app folder and you see 2 > or 3 files. > Many developers have the help built-in just to be as > minimalist and straightforward as possible. > Valentina developers say > that all of a sudden our applications have to look like PC folders > full of junk... I am so annoyed. Can I ask you: about what kind of MAC application you talk now? - old style executable (Carbon) - or about new Apple style Package ? > I can't create bundles because I > think it is too soon right now, I am still PEFed. Aha, so you talk about OLD SYTLE application. > Bundles for RB developers will maybe be usual in 6 months or so, with macho > and those universal binaries. Well, Apple use packages - bundles for years already. Right ? > Really awful... Some days I wonder > I did not make a mistake choosing Valentina 3 years ago. > Not only my apps are now a lot bigger, now they also look crappy. > Next step will be to use an alias instead of the application to hide > Valentina junk. I fill personally molested when they ask me to do > things like if I were using a PC. A PC is a PC, great but please a > little respect about Mac people want the things done. I haven't > purchased the last G5 Quad to use it as a PC box, really! Please heed > and flag this request with maximum priority, so many Mac developers > will be really happy then! Stan, if you want get nice MODERN Application, then you should use Packages. I already 5 times have told the same on this list. It is time move to FAQ probably. Open any application from Apple, which is as package. It contains hundreds of files. Or have you see Valentina Studio? Looks nice? Just one application file. AND, you still can put VComponents into CFMSupport, in your isntaller. Then this is similar to Windows way. Right? Actually as Claudius says, I need at last of end force to work VComponents folder from inside of Macho package that build RB. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 17 23:12:15 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 15:13:09 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 7:53 PM, "Ed Kleban" wrote: >>> Fair enough, especially since I expect there's some simple way to do >>> this >>> that's not apparent to me. Possibly by mapping a Memory block pointer >>> to >>> the VField or something. I'll run this by Thomas Templeton or the NUG >>> list >>> on RB. Or I see Charles Yeoman is on this list. He'll likely know. >> >> It's not hard to get the four-byte pieces. Take the string returned >> from a LLong or ULLong field, stick it into an eight-byte MemoryBlock, >> then read out the pieces using MemoryBlock.Long. > > Ah! So the string returned by GetString is a binary string. Doh! Of course > it is. I was thinking it would be a printable numeric for some stupid > reason. That's the problem with having at least 5 overloaded uses for the > term "string". Wait guys. longFld.GetString() Return normal string. Not binary > Ok, this is the clue I was missing, and the exact MemoryBlock usage details > I knew had to exist but wasn't too clueful about. Thanks Charles! > -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Thu Nov 17 16:18:39 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 17 15:19:33 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: References: Message-ID: On Nov 17, 2005, at 4:12 PM, Ruslan Zasukhin wrote: > On 11/17/05 7:53 PM, "Ed Kleban" wrote: > >>>> Fair enough, especially since I expect there's some simple way to do >>>> this >>>> that's not apparent to me. Possibly by mapping a Memory block >>>> pointer >>>> to >>>> the VField or something. I'll run this by Thomas Templeton or the >>>> NUG >>>> list >>>> on RB. Or I see Charles Yeoman is on this list. He'll likely know. >>> >>> It's not hard to get the four-byte pieces. Take the string returned >>> from a LLong or ULLong field, stick it into an eight-byte >>> MemoryBlock, >>> then read out the pieces using MemoryBlock.Long. >> >> Ah! So the string returned by GetString is a binary string. Doh! Of >> course >> it is. I was thinking it would be a printable numeric for some stupid >> reason. That's the problem with having at least 5 overloaded uses for >> the >> term "string". > > Wait guys. > > longFld.GetString() > > Return normal string. Not binary Could you be more precise? What, exactly, does LLongFld.GetString return? -------------- Charles Yeomans From Ed at Kleban.com Thu Nov 17 15:59:29 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 17 16:00:26 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 3:18 PM, "Charles Yeomans" wrote: > > On Nov 17, 2005, at 4:12 PM, Ruslan Zasukhin wrote: > >> On 11/17/05 7:53 PM, "Ed Kleban" wrote: >> >>>>> Fair enough, especially since I expect there's some simple way to do >>>>> this >>>>> that's not apparent to me. Possibly by mapping a Memory block >>>>> pointer >>>>> to >>>>> the VField or something. I'll run this by Thomas Templeton or the >>>>> NUG >>>>> list >>>>> on RB. Or I see Charles Yeoman is on this list. He'll likely know. >>>> >>>> It's not hard to get the four-byte pieces. Take the string returned >>>> from a LLong or ULLong field, stick it into an eight-byte >>>> MemoryBlock, >>>> then read out the pieces using MemoryBlock.Long. >>> >>> Ah! So the string returned by GetString is a binary string. Doh! Of >>> course >>> it is. I was thinking it would be a printable numeric for some stupid >>> reason. That's the problem with having at least 5 overloaded uses for >>> the >>> term "string". >> >> Wait guys. >> >> longFld.GetString() >> >> Return normal string. Not binary > Thankyou for making my point. Clearly you read a different meaning into the word "String" than I ended up concluding was really meant. Depending upon precisely what you mean, I'm sure you can read in the RB documentation places where it will tell you that a string can contain binary. Consider for example the difference between the definitions in the language reference for len(aString) and lenB(aString). The only way to resolve this is to answer the question Charles asks below. You have to give us a concrete example using quoted string characters "Like this" and Hex like this: 0h12F34285 -- without that there's just know way to have a good clue of either what longFld.GetSting() does or what you mean to say -- short of looking at one of your fine code examples which is another way of accomplishing the same thing. > Could you be more precise? What, exactly, does LLongFld.GetString > return? > -------------- > Charles Yeomans > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From cindy at kowhaiprogramming.com Fri Nov 18 12:01:23 2005 From: cindy at kowhaiprogramming.com (Cindy Brown) Date: Thu Nov 17 17:03:04 2005 Subject: Valentina and RB2005 Message-ID: Hi All, I am trying to compile a program that heavily uses Valentina in RB 2005. It keeps coming up with error messages related to _vtable. One of my smaller projects does run but brings up the error message first up whenever I try to compile it. I am using Valentina 1.12 as I have not yet moved up to version 2. Could someone please tell me if it possible to use Valentina 1 with RB 2005? Thanks. Cindy -- Cindy Brown Programmer SchoolMaster Kowhai Programming Systems PO Box 198, Invercargill 81 Marama Avenue South, 9RD Invercargill Phone (03) 213 1243 Fax (03) 213 1248 Mobile (021) 354 930 From sunshine at public.kherson.ua Fri Nov 18 01:10:49 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 17:11:45 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 11:18 PM, "Charles Yeomans" wrote: >> Wait guys. >> >> longFld.GetString() >> >> Return normal string. Not binary > > Could you be more precise? What, exactly, does LLongFld.GetString > return? If you have in Long field number 12345 Then GetString() returns string "12345". 6 bytes including ZERO end GetString() in V4RB1 did return binary values for BLOB only -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 01:12:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 17:13:25 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 11:59 PM, "Ed Kleban" wrote: >>> Wait guys. >>> >>> longFld.GetString() >>> >>> Return normal string. Not binary >> > > > Thankyou for making my point. Clearly you read a different meaning into the > word "String" than I ended up concluding was really meant. Depending upon > precisely what you mean, I'm sure you can read in the RB documentation > places where it will tell you that a string can contain binary. Right, I know this. But fld.GetString() returns normal string which you can show in edit field for example. > Consider > for example the difference between the definitions in the language reference > for len(aString) and lenB(aString). > > The only way to resolve this is to answer the question Charles asks below. > You have to give us a concrete example using quoted string characters "Like > this" and Hex like this: 0h12F34285 -- without that there's just know > way to have a good clue of either what longFld.GetSting() does or what you > mean to say -- short of looking at one of your fine code examples which is > another way of accomplishing the same thing. In terms of C++ GetString for integer executes itoa() -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Thu Nov 17 17:37:27 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 17 17:38:26 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 5:12 PM, "Ruslan Zasukhin" wrote: > > In terms of C++ > > GetString for integer executes itoa() > That is most unfortunate, at least for my interests. So then, we're back to where we started, name the point where I stood up on a soapbox and proclaimed: "I use Valentina to do a super fast lookup of my data to get it into a current Table record or into a Cursor, but then I have to convert a binary value to a string which I then have to chop up with string operations and convert back to a binary value to use? Yuck! ... or am I missing something here?" So if I understand this correctly to get read an integer value from a Vshort I have to write code that looks like: dim anInt as Integer dim valueToStore as Integer = 5423 anInt = val( aVShortField.getString( )) aVShortField.setString( str( valueToStore ) ) I got that right? All I can say is... Yuck! Oh well... at least I know how to do it. Ah, but wait! There may yet be hope. I see that there is a Value property. Thus I can presumably say: anInt = aVShortField.value aVShortField.value = valueToStore This was what I was missing. I was looking for a function instead of a property. So presumably I can also use: anInt = aVULongFile.value and end up with the low 32-bit word of the ULong field, although as Ruslan ponits out, if the high-order bit is set the value of anInt will be negative and require some bit twiddling; probably just a single subtraction or boolean operation. And if I want the high order word I can indeed use GetString() or wait until the REALfolk provide UInt64 and friends. Ok, I'm back on target and happy. I guess Ruslan wasn't brain dead after all when designing this part of the interface. We'll have to look elsewhere I guess if we want to try and find someplace where he was ;-) Thanks for putting up with my wild goose chase. I need to start coding and stop emailing to get some experience. Later! --Ed From sunshine at public.kherson.ua Fri Nov 18 01:46:43 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 17:47:38 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: On 11/18/05 1:01 AM, "Cindy Brown" wrote: > Hi All, > > I am trying to compile a program that heavily uses Valentina in RB 2005. It > keeps coming up with error messages related to _vtable. One of my smaller > projects does run but brings up the error message first up whenever I try to > compile it. I am using Valentina 1.12 as I have not yet moved up to version > 2. Could someone please tell me if it possible to use Valentina 1 with RB > 2005? I believe many developers still use it. It seems Keith Delong for example. Note, now it needs mention Rb2005 what version? r1? r2? r3? r4? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 01:47:40 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 17:48:34 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/18/05 1:37 AM, "Ed Kleban" wrote: >> In terms of C++ >> >> GetString for integer executes itoa() >> > > That is most unfortunate, at least for my interests. So then, we're back to > where we started, name the point where I stood up on a soapbox and > proclaimed: Hmm. But dim v as integer = fldLong.Value Give you binary value. What can be more ?! -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 01:48:11 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 17 17:49:04 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/18/05 1:37 AM, "Ed Kleban" wrote: > > I got that right? > All I can say is... Yuck! > > Oh well... at least I know how to do it. > > Ah, but wait! There may yet be hope. > > I see that there is a Value property. Thus I can presumably say: > > anInt = aVShortField.value > aVShortField.value = valueToStore RIGHT ! :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From delong at redcort.com Thu Nov 17 16:16:05 2005 From: delong at redcort.com (Keith DeLong) Date: Thu Nov 17 18:17:00 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: > On 11/18/05 1:01 AM, "Cindy Brown" wrote: > >> Hi All, >> >> I am trying to compile a program that heavily uses Valentina in RB 2005. It >> keeps coming up with error messages related to _vtable. One of my smaller >> projects does run but brings up the error message first up whenever I try to >> compile it. I am using Valentina 1.12 as I have not yet moved up to version >> 2. Could someone please tell me if it possible to use Valentina 1 with RB >> 2005? > > Ruslan responded: > I believe many developers still use it. > It seems Keith Delong for example. > > Note, now it needs mention Rb2005 what version? r1? r2? r3? r4? We did just release a major update that was developed and tested using RB2005 r4 and V4RB 1.11. I didn't discover any Valentina issues going from RB 5.5 to 2005. Cindy mentioned using 1.12... I must have missed 1.12 as it's not on my hard drive. I could not find it on your web site Ruslan. Do you have a link so I can see what's different? Thanks, Keith DeLong From cindy at kowhaiprogramming.com Fri Nov 18 13:23:25 2005 From: cindy at kowhaiprogramming.com (Cindy Brown) Date: Thu Nov 17 18:25:03 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: Hi Keith, Thanks for that. I did say 1.12 but I think that was just a small fix that Ruslan did for Windows. It doesn't have the release number on the plugin for me to check but it was compiled on the 21st May, 2004 if that's any help. All the best. Cindy > We did just release a major update that was developed and tested using > RB2005 r4 and V4RB 1.11. I didn't discover any Valentina issues going from > RB 5.5 to 2005. > > Cindy mentioned using 1.12... I must have missed 1.12 as it's not on my hard > drive. I could not find it on your web site Ruslan. Do you have a link so I > can see what's different? From delong at redcort.com Thu Nov 17 16:47:47 2005 From: delong at redcort.com (Keith DeLong) Date: Thu Nov 17 18:48:40 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: Wonderful... My 1.11 V4RB folder says 'special build December 2004 '. The build date on the plugin is December 19, 2004. It sounds like I may have the latest v1 build for Macs. Ruslan? > Hi Keith, > > Thanks for that. I did say 1.12 but I think that was just a small fix that > Ruslan did for Windows. It doesn't have the release number on the plugin for > me to check but it was compiled on the 21st May, 2004 if that's any help. > > All the best. > > Cindy > >> We did just release a major update that was developed and tested using >> RB2005 r4 and V4RB 1.11. I didn't discover any Valentina issues going from >> RB 5.5 to 2005. >> >> Cindy mentioned using 1.12... I must have missed 1.12 as it's not on my hard >> drive. I could not find it on your web site Ruslan. Do you have a link so I >> can see what's different? From yeomans at desuetude.com Thu Nov 17 19:58:14 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Thu Nov 17 18:59:11 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: References: Message-ID: <3fa420a9cc50abb42c3a8216841d243b@desuetude.com> On Nov 17, 2005, at 6:47 PM, Ruslan Zasukhin wrote: > On 11/18/05 1:37 AM, "Ed Kleban" wrote: > >>> In terms of C++ >>> >>> GetString for integer executes itoa() >>> >> >> That is most unfortunate, at least for my interests. So then, we're >> back to >> where we started, name the point where I stood up on a soapbox and >> proclaimed: > > Hmm. > > But > > dim v as integer = fldLong.Value > > Give you binary value. What can be more ?! But what about LLong fields? Recall that REALbasic lacks eight-byte integer types. -------------- Charles Yeomans From bkeeney at everestkc.net Thu Nov 17 21:01:57 2005 From: bkeeney at everestkc.net (Bob Keeney) Date: Thu Nov 17 21:02:52 2005 Subject: Val Studio Error Message Question In-Reply-To: <20051117231147.048843CDF98@edison.macserve.net> References: <20051117231147.048843CDF98@edison.macserve.net> Message-ID: <78718840-8574-4D22-90B4-51704D18EE72@everestkc.net> What does the following error mean? 20:59:56: Kernel error: 0x59504 20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is already open. 20:59:56: Database could not be opened! Is there a way to recover this database? Thanks, Bob K. From sunshine at public.kherson.ua Fri Nov 18 08:26:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 00:27:05 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: On 11/18/05 2:16 AM, "Keith DeLong" wrote: >>> Hi All, >>> >>> I am trying to compile a program that heavily uses Valentina in RB 2005. It >>> keeps coming up with error messages related to _vtable. One of my smaller >>> projects does run but brings up the error message first up whenever I try to >>> compile it. I am using Valentina 1.12 as I have not yet moved up to version >>> 2. Could someone please tell me if it possible to use Valentina 1 with RB >>> 2005? >> >> Ruslan responded: >> I believe many developers still use it. >> It seems Keith Delong for example. >> >> Note, now it needs mention Rb2005 what version? r1? r2? r3? r4? > > We did just release a major update that was developed and tested using > RB2005 r4 and V4RB 1.11. I didn't discover any Valentina issues going from > RB 5.5 to 2005. > > Cindy mentioned using 1.12... I must have missed 1.12 as it's not on my hard > drive. I could not find it on your web site Ruslan. Do you have a link so I > can see what's different? I think Cindy have made typo -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 08:28:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 00:29:28 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: On 11/18/05 2:47 AM, "Keith DeLong" wrote: > Wonderful... My 1.11 V4RB folder says 'special build December 2004 '. The > build date on the plugin is December 19, 2004. > > It sounds like I may have the latest v1 build for Macs. Ruslan? I afraid no. Just I have fix some show stopper bug for Windows for Cindy. Hmm, not remember which one ... Cindy? That was only for Cindy 1.11 + that bug build >> Hi Keith, >> >> Thanks for that. I did say 1.12 but I think that was just a small fix that >> Ruslan did for Windows. It doesn't have the release number on the plugin for >> me to check but it was compiled on the 21st May, 2004 if that's any help. >> >> All the best. >> >> Cindy >> >>> We did just release a major update that was developed and tested using >>> RB2005 r4 and V4RB 1.11. I didn't discover any Valentina issues going from >>> RB 5.5 to 2005. >>> >>> Cindy mentioned using 1.12... I must have missed 1.12 as it's not on my hard >>> drive. I could not find it on your web site Ruslan. Do you have a link so I >>> can see what's different? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 08:29:45 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 00:30:42 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: <3fa420a9cc50abb42c3a8216841d243b@desuetude.com> Message-ID: On 11/18/05 2:58 AM, "Charles Yeomans" wrote: >>>> In terms of C++ >>>> >>>> GetString for integer executes itoa() >>>> >>> >>> That is most unfortunate, at least for my interests. So then, we're >>> back to >>> where we started, name the point where I stood up on a soapbox and >>> proclaimed: >> >> Hmm. >> >> But >> >> dim v as integer = fldLong.Value >> >> Give you binary value. What can be more ?! > > But what about LLong fields? Recall that REALbasic lacks eight-byte > integer types. Sorry, this is question to REAL :-) right now V4RB do NOT support int64. you can assign value only via Strings. As only REAL will implement int64, we will add .Value for LLong -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 08:30:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 00:31:47 2005 Subject: Val Studio Error Message Question In-Reply-To: <78718840-8574-4D22-90B4-51704D18EE72@everestkc.net> Message-ID: On 11/18/05 5:01 AM, "Bob Keeney" wrote: > What does the following error mean? > > 20:59:56: Kernel error: 0x59504 > 20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is > already open. > 20:59:56: Database could not be opened! Yes, this is corruption. > Is there a way to recover this database? Try new way first of all: db.Clone() You can even use V4RB example Api/Db_Clone as utility todo this. Another way -- XML dump - Load -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Fri Nov 18 00:41:51 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 00:42:48 2005 Subject: Fine Tuning In-Reply-To: Message-ID: Howdy Ruslan, I've been fine tuning my schema and think I may have found a performance issue I hadn't considered before. Since every column is stored as a separate table, I started trying to combine some of the fields in parallel side tables into my main one. Messier on the page, but has the benefit of direct access indexing rather than the overhead of a binary search when you need access to those fields. Then I realized, "Doh!" Every time I do... "aTable.recID = aRecordNumberOfInterest" ...Valentina is going to lookup and load the value of every one of those columns for me to access, whether I need them or not. Right? Double Doh! So then I went the opposite direction and pulled out every single column I didn't expect to need upon virtually every access into a separate parallel table for occasional access at the cost of a binary search. Better. But now I perceive another opportunity that we never discussed. Yet another efficiency tradeoff that may well be the decisive argument in your favor for binary links. If the relation is in a binary link rather than a table field, I don't incur the overhead of pre-loading that link value every time I make a record current whether or not I need to use that link. If the info is in a binary link instead, it doesn't even get looked at unless I really need it. That correct? From j.peters at valentina-db.de Fri Nov 18 07:42:48 2005 From: j.peters at valentina-db.de (Jochen Peters) Date: Fri Nov 18 00:43:46 2005 Subject: Val Studio Error Message Question (Bob Keeney) In-Reply-To: <20051118062716.E31B43CE546@edison.macserve.net> References: <20051118062716.E31B43CE546@edison.macserve.net> Message-ID: <46401.62.154.199.179.1132296168.squirrel@webmail3.server-einstellung.de> Hi Bob, >What does the following error mean? > >20:59:56: Kernel error: 0x59504 >20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is >already open. >20:59:56: Database could not be opened! > >Is there a way to recover this database? Hhmm - this error comes directly from the kernel, so, maybe Ruslan can tell us the technical reson for tis error code? Bob - is this reproducable? Have you tried to close Studio and reopen the database? Maybe the database was not closed correctly... Try to describe steps to reproduce this behaviour. -- Best regards, Jochen Peters PIIT GmbH -------------------------- http://www.valentina-db.de From sunshine at public.kherson.ua Fri Nov 18 08:53:15 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 00:54:16 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 8:41 AM, "Ed Kleban" wrote: > Howdy Ruslan, > > I've been fine tuning my schema and think I may have found a performance > issue I hadn't considered before. > > Since every column is stored as a separate table, I started trying to > combine some of the fields in parallel side tables into my main one. > Messier on the page, but has the benefit of direct access indexing rather > than the overhead of a binary search when you need access to those fields. > > Then I realized, "Doh!" Every time I do... > > "aTable.recID = aRecordNumberOfInterest" > > ...Valentina is going to lookup and load the value of every one of those > columns for me to access, whether I need them or not. Right? Right. Cache can compensate a lots this. And we going to improve this using lazy reading. > Double Doh! So then I went the opposite direction and pulled out every > single column I didn't expect to need upon virtually every access into a > separate parallel table for occasional access at the cost of a binary > search. > > Better. In fact, if you use SQL and cursors, then you can SELECT e.g. 3 fields from 20. Then cursor will iterate 20/3 = 7 times faster. > But now I perceive another opportunity that we never discussed. Yet > another efficiency tradeoff that may well be the decisive argument in your > favor for binary links. If the relation is in a binary link rather than a > table field, I don't incur the overhead of pre-loading that link value every > time I make a record current whether or not I need to use that link. Yes good point. > If the info is in a binary link instead, it doesn't even get looked at unless > I really need it. > > That correct? 100%. At least until we get luck to add lazy reading. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Fri Nov 18 01:10:18 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 01:11:15 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 12:53 AM, "Ruslan Zasukhin" wrote: > On 11/18/05 8:41 AM, "Ed Kleban" wrote: > >> Howdy Ruslan, >> >> I've been fine tuning my schema and think I may have found a performance >> issue I hadn't considered before. >> >> Since every column is stored as a separate table, I started trying to >> combine some of the fields in parallel side tables into my main one. >> Messier on the page, but has the benefit of direct access indexing rather >> than the overhead of a binary search when you need access to those fields. >> >> Then I realized, "Doh!" Every time I do... >> >> "aTable.recID = aRecordNumberOfInterest" >> >> ...Valentina is going to lookup and load the value of every one of those >> columns for me to access, whether I need them or not. Right? > > Right. Cache can compensate a lots this. > And we going to improve this using lazy reading. What is this? Ah, read on need. I get it. Yes that would be a great thing to have. > >> Double Doh! So then I went the opposite direction and pulled out every >> single column I didn't expect to need upon virtually every access into a >> separate parallel table for occasional access at the cost of a binary >> search. >> >> Better. > > In fact, if you use SQL and cursors, then you can SELECT e.g. 3 fields from > 20. Then cursor will iterate 20/3 = 7 times faster. Ah, very good point. I hadn't thought about that. I was intentionally avoiding SQL accesses in favor of the native API based on presumed superior efficiency. But yes, I see now. SQL has the expressive power to limit the work done if your SQL parser is smart enough to code it well -- which you've just told me Valentina's is. Makes sense. You'd want to optimize the heck out of this since that's how the majority of the world would access the tables. Ok, Delving into SQL will be next on my study list. I believe in a previous email I saw you comment that you cant do a search on cursor results, but rather you can build a new table if need be. My intended approach had been to do Find extracts to VArraySets; but if if SQL provides me either with that or the ability to make interim temporary tables, either of those should be just dandy. > >> But now I perceive another opportunity that we never discussed. Yet >> another efficiency tradeoff that may well be the decisive argument in your >> favor for binary links. If the relation is in a binary link rather than a >> table field, I don't incur the overhead of pre-loading that link value every >> time I make a record current whether or not I need to use that link. > > Yes good point. > Not just a good point, it may in my particular case be the killer justification for using binary links occasionally, because I've seen essentially zero benefit so far for my needs. >> If the info is in a binary link instead, it doesn't even get looked at unless >> I really need it. >> >> That correct? > > 100%. > > At least until we get luck to add lazy reading. At which point I'll have no more use for binary links again, but should realize an increase in both space and performance efficiency. Great! One more thing that's come out of my study. The Find function that I'd really like to see is one that works like FindRange, but instead of taking a low and high value it takes a bit mask which is anded against the value and would conditionally include the record if: A) All bits in mask are also set in value... or B) Any bit in mask is also set in value.. or C) All bits in mask are clear in value.. I'm not sure if SQL can do this or not, but that will be one of the things I check for. I believe I looked before and saw boolean logic XOR and such on conditions but not on bits. But I'll look again :) Thanks! --Ed From Ed at Kleban.com Fri Nov 18 01:12:36 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 01:13:34 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 12:53 AM, "Ruslan Zasukhin" wrote: > On 11/18/05 8:41 AM, "Ed Kleban" wrote: >> ...Valentina is going to lookup and load the value of every one of those >> columns for me to access, whether I need them or not. Right? > > Right. Cache can compensate a lots this. > And we going to improve this using lazy reading. > Cache is something that I totally ignore in doing a performance analysis like this. Essentially I assume that the entire database is memory for this level of study. Once it ends up on disk of course then I have to cross my fingers and hope that you've implemented are really good cache :) From sunshine at public.kherson.ua Fri Nov 18 09:56:47 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 01:57:48 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 9:10 AM, "Ed Kleban" wrote: >> Right. Cache can compensate a lots this. >> And we going to improve this using lazy reading. > > What is this? Ah, read on need. I get it. Yes that would be a great thing > to have. You have it right now -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 09:59:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 02:00:48 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 9:10 AM, "Ed Kleban" wrote: >> In fact, if you use SQL and cursors, then you can SELECT e.g. 3 fields from >> 20. Then cursor will iterate 20/3 = 7 times faster. > > Ah, very good point. I hadn't thought about that. I was intentionally > avoiding SQL accesses in favor of the native API based on presumed superior > efficiency. But yes, I see now. That is the point. We have realize too late that API Vtable do not have mechanism as Vcursor have. So we have come to idea of lazy reading. > SQL has the expressive power to limit the > work done if your SQL parser is smart enough to code it well -- which you've > just told me Valentina's is. Makes sense. You'd want to optimize the heck > out of this since that's how the majority of the world would access the > tables. > > Ok, Delving into SQL will be next on my study list. I believe in a previous > email I saw you comment that you cant do a search on cursor results, but > rather you can build a new table if need be. My intended approach had been > to do Find extracts to VArraySets; but if if SQL provides me either with > that or the ability to make interim temporary tables, either of those should > be just dandy. You can MIX. Some task do in API, some in SQL FYI: In c++ developers have much more tools to work with. Just no sense open that tools e.g. V4RB. Or it is hard technically. For example, in C++ you can iterate single column applying some action. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Fri Nov 18 02:10:59 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 02:12:03 2005 Subject: Fine Tuning In-Reply-To: Message-ID: No, I meant that lazy reading would be a great thing to have. I know we have a cache right now. Valentina would be slow as a dead turtle without one. On 11/18/05 1:56 AM, "Ruslan Zasukhin" wrote: > On 11/18/05 9:10 AM, "Ed Kleban" wrote: > >>> Right. Cache can compensate a lots this. >>> And we going to improve this using lazy reading. >> >> What is this? Ah, read on need. I get it. Yes that would be a great thing >> to have. > > You have it right now From Ed at Kleban.com Fri Nov 18 02:14:57 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 02:15:54 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 1:59 AM, "Ruslan Zasukhin" wrote: > On 11/18/05 9:10 AM, "Ed Kleban" wrote: > >>> In fact, if you use SQL and cursors, then you can SELECT e.g. 3 fields from >>> 20. Then cursor will iterate 20/3 = 7 times faster. >> >> Ah, very good point. I hadn't thought about that. I was intentionally >> avoiding SQL accesses in favor of the native API based on presumed superior >> efficiency. But yes, I see now. > > That is the point. We have realize too late that API Vtable do not have > mechanism as Vcursor have. So we have come to idea of lazy reading. > Good idea. We likely to see it within a year? > >> SQL has the expressive power to limit the >> work done if your SQL parser is smart enough to code it well -- which you've >> just told me Valentina's is. Makes sense. You'd want to optimize the heck >> out of this since that's how the majority of the world would access the >> tables. >> >> Ok, Delving into SQL will be next on my study list. I believe in a previous >> email I saw you comment that you cant do a search on cursor results, but >> rather you can build a new table if need be. My intended approach had been >> to do Find extracts to VArraySets; but if if SQL provides me either with >> that or the ability to make interim temporary tables, either of those should >> be just dandy. > > You can MIX. > Some task do in API, some in SQL I intend to. > > FYI: In c++ developers have much more tools to work with. > Just no sense open that tools e.g. V4RB. Or it is hard technically. > > For example, in C++ you can iterate single column applying some action. > Good to know. My approach is to go as far as I can in my environment of choice where I'm productive, RB in this case, and then look for ways to optimize outside the environment later... such as writing a plugin in C++ that could provide additional performance. -- A common appropach. When I looked on the Paradigmasoft.com site in the document section the last couple times I never found the C++ API documentation. What's the document name? From Ed at Kleban.com Fri Nov 18 02:21:40 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 02:22:36 2005 Subject: Compact? In-Reply-To: Message-ID: What does the SQL compact command really do? From cindy at kowhaiprogramming.com Fri Nov 18 21:30:24 2005 From: cindy at kowhaiprogramming.com (Cindy Brown) Date: Fri Nov 18 02:32:07 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: I think I just remembered what it was. It was a but on windows which wasn't saving varchar fields correctly when a number of records were being updating very quickly. It has been working fine ever since then. Cindy > I afraid no. Just I have fix some show stopper bug for Windows for Cindy. > > Hmm, not remember which one ... Cindy? > > That was only for Cindy 1.11 + that bug build -- Cindy Brown Programmer SchoolMaster Kowhai Programming Systems PO Box 198, Invercargill 81 Marama Avenue South, 9RD Invercargill Phone (03) 213 1243 Fax (03) 213 1248 Mobile (021) 354 930 From sunshine at public.kherson.ua Fri Nov 18 11:07:37 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 03:08:32 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 10:14 AM, "Ed Kleban" wrote: >> That is the point. We have realize too late that API Vtable do not have >> mechanism as Vcursor have. So we have come to idea of lazy reading. >> > > Good idea. We likely to see it within a year? Within 12 moths? Lets hope -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 11:08:48 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 03:09:42 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 10:14 AM, "Ed Kleban" wrote: > Good to know. My approach is to go as far as I can in my environment of > choice where I'm productive, RB in this case, and then look for ways to > optimize outside the environment later... such as writing a plugin in C++ > that could provide additional performance. -- A common appropach. > > When I looked on the Paradigmasoft.com site in the document section the last > couple times I never found the C++ API documentation. What's the document > name? On site now present only VSDK 1.x You can just download MAC C++ SDK 2.1 b3 And check our headers -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 11:10:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 03:11:23 2005 Subject: Compact? In-Reply-To: Message-ID: On 11/18/05 10:21 AM, "Ed Kleban" wrote: > What does the SQL compact command really do? Right now it just remove big segments which you get for example on Drop field. Future we can add compact of BLOB files. It is possible also add option to compact as much as possible VarChars and indexes. This can be good for developers who want put db on media as CD/DVD -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Fri Nov 18 09:07:34 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 09:08:32 2005 Subject: link BOB values AS IS ? Message-ID: In the valentina SQL manual description for "Parameter Binding", the text states: Some advantages of linkage usage: [...] 4) you can link BLOB values AS IS. What does this mean? From Ed at Kleban.com Fri Nov 18 09:48:07 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 09:49:05 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/17/05 11:20 AM, "Charles Yeomans" wrote: > >>>> I see that there are casting functions for the VCursor class, which I >>>> understand are for moving values between a Cursor and a Table's >>>> Vfields. >>> >>> Mmmmm actually this is to write one line of code instead of 2 >>> >>> fldLong = curs.Longfield( I ) >>> >>> vs >>> >>> fld = curs.field( I ) >>> fldLong = Vlong( fld ) >>> >> >> Ah. Makes sense. I hadn't thought it through but presumed it was for >> something like that. This would be a good statement to add to the >> manual. >> Actually, it is already stated in the manual, quite well and very detailed. That's fast work Ruslan. Thanks! --Ed From Ed at Kleban.com Fri Nov 18 10:01:02 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 10:02:04 2005 Subject: Fine Tuning In-Reply-To: Message-ID: >>> Ok, Delving into SQL will be next on my study list. I believe in a previous >>> email I saw you comment that you cant do a search on cursor results, but >>> rather you can build a new table if need be. My intended approach had been >>> to do Find extracts to VArraySets; but if if SQL provides me either with >>> that or the ability to make interim temporary tables, either of those should >>> be just dandy. >> >> You can MIX. >> Some task do in API, some in SQL > > I intend to. > Ok, I've read the SQL manual and so now know enough to be dangerous I guess. I've also gone back and stared the Reference manual a good bit. I think I understand the abilities and benefits of using SQL vs native API. What I don't have a good understanding of is how to mix the two when it comes to manipulation of subsets via VCursors and VSets. Specifically, If I do an SQL query to determine a subset of records of interest, and obtain a VCursor with say, only a RecID field, then is there a way I can create a VSet for the corresponding table that only includes those RecIDs made accessible via the cursor? Conversely, if I have an ArraySet containing a subset of table records of interest, is there are way to perform an SQL query on only that subset of records? I'm guessing that the solution to both of these probably involves the creation of an temporary, intermediate table in memory or disk, that contains only the recID fields of the records of interest. What I'm not sure about is the best way to create such a table from either a cursor resulting from an SQL query, or an existing Vset. Do I just use the Vtable constructor commands to create the table manual and then iterate through either the Cursor or the Set manually in a loop to populate it? Or is there a better, more automatic way that I'm missing? Thanks! --Ed From delong at redcort.com Fri Nov 18 08:15:40 2005 From: delong at redcort.com (Keith DeLong) Date: Fri Nov 18 10:16:36 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: > I think I just remembered what it was. It was a but on windows which wasn't > saving varchar fields correctly when a number of records were being updating > very quickly. It has been working fine ever since then. > > Cindy > >> I afraid no. Just I have fix some show stopper bug for Windows for Cindy. >> >> Hmm, not remember which one ... Cindy? >> >> That was only for Cindy 1.11 + that bug build Is the latest 1.11 release available for download from your web site Ruslan? Keith From Ed at Kleban.com Fri Nov 18 12:13:35 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 12:14:33 2005 Subject: A plan emerges Re: Fine Tuning In-Reply-To: Message-ID: >>> >>> You can MIX. >>> Some task do in API, some in SQL >> >> I intend to. >> Ok, here's a first pass at a plan for more optimal access. I'm thinking this through as I write it here, so bear with me please. First of all, the nature of my processing is going to be as follows: 1) I'm going to start with a big hairy parse tree derived from some data source such as an XML file, and then render its contents into a V4RB database for fast access. I'll likely not parse it all at once, but incrementally over time in the background so the user essentially sees instantaneous response in the GUI at all times. But sooner or later, population of the DB from the parse is going to be completed -- and in all probability much sooner than the user can get around to interacting in a manner that will require a serious query. Worst case, the user has to wait a few more seconds for the population to complete before their first deep query gets processed. But navigation at high levels should be available immediately regardless. 2) Once the population effort is completed, the tables that result -- or perhaps more accurately, the content of the records in the range of rows used by the populated data in the various tables -- will never be modified, and none of the populated records will ever be deleted. Thus this essentially becomes a static database. 3) Any modifications that are made by the user will be represented either in separate tables, or in newly added rows to the existing tables. And the same static nature for these modifications is true as well. Once a modification is entered, it will never be deleted except perhaps by some eventual "cleanup" routine that gets trigged priore to long-term storage when the database tables are no longer in active use. After the database is populated I then plan to query the hell out of it. To facilitate that in a manner that would simulate the "lazy read" mechanism you propose I could create a collection of VCursors for the purpose by means of several "SELECT " SQL statements. If there were natural field groupings I intended to use together such as "HomePhone, WorkPhone, Fax", I could include multiple field names in . But in the extreme case, or if I was clueless in advance of what my desired access patterns were, I would essentially create a VCursor for each single field in this manner, each field spanning every record of the table. Let's call a Vcursor made in this manner and for this purpose a "VFieldRef". Since the table will never grow in a manner that will affect the range of records I care about, I never need to recreate or update these VFieldRef accessors. To work with these field accessors, I first get a list of records I care about either by using the API to come up with Vsets or by using SQL statements resulting in a cursor over a set of desired records with a single cursor field of RecId. I can then iterate over the Set or Cursor, pull out successive RecId values, and use them to access whichever particular field values I need to access via... aVFieldRef.position = recIdOfInterest anInt = aVFieldRef.ShortFied(1).value ...and thus rapidly access the sole field I need without paying the penalty for retrieving the the current value for every field of the record at recIdOfInterest. And to make it all look pretty we can create a new class for VFieldRef: class: VFieldRef subclassOf Object constructors: VFieldRef( inFieldID as Variant , inVCursor as Vcursor ) // Initialize VFieldRef for accessing the specified field // of the existing inVCursor. // The inFieldId may be a name String or index Integer VFieldRef( inFieldId as Variant, inTable as VTable ) // Initialize VFieldRef for accessing the specified field // of the existing inTable. // The inFieldId may be a name String or index Integer properties: myVCursor as Vcursor fieldIndex as Integer methods: Value( inRecID as Integer ) as Variant Value( inRecID as Integer, Assigns inValue as Variant ) ShortValue( inRecId as Integer ) as Integer ShortValue( inRecId as Integer, Assigns inValue as Integer ) StringValue( inRecId as Integer) as String StringValue( inRecId as Integer, Assigns inValue as String ) ... and many more for each of the various desired types. The net result is that instead of using the syntax shown above... aVFieldRef.position = recIdOfInterest anInt = aVFieldRef.ShortFied(1).value ...you could instead use the prettier syntax... anInt = aVFieldRef.ShortValue( recIdOfInterest ) ...as well as: aVFieldRef.ShortValue( recIdOfInterest ) = anInt There is a minor cost for calling a VFieldRef method of course to use this prettier syntax, but I suspect that the overhead is absolutely minimal compared to the overhead of retrieving the current value for every field of the record at recIdOfInterest when you only want to access one or two fields. This also has the benefit of completely eliminating usage of the cursor as an explicit access mechanism, delegating its usage to a hidden abstraction -- where I'd far rather have it be. So then, the big questions are: Does this make sense? Will it work as intended with the efficiency gains I infer? What did I miss? Assuming the above approach or something similar will work, then I'm now back to the conclusions: 1) There is once again no benefit, and indeed there is both a performance and storage penalty, for using Binary Links over ObjectPtr links for my particular usage needs where most all of the linked fields involve a 1:1 unique:unique or M:1 unique:unique mapping from my primary table records to the records of my various ancillary support tables. 2) I can be far less concerned about trying to predict in advance which are the most frequently accessed fields to include in my primary table records and use a layout that feels intuitive and comfortable rather than minimalist for (supposed) performance optimization -- which is the way it should be in my opinion. 3) I still have the option of making a wide or narrow primary record and shoveling off ancillary fields to other tables. Which is a good thing! Now I'm itching to start coding :) From sunshine at public.kherson.ua Fri Nov 18 22:20:40 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:21:37 2005 Subject: link BOB values AS IS ? In-Reply-To: Message-ID: On 11/18/05 5:07 PM, "Ed Kleban" wrote: > In the valentina SQL manual description for "Parameter Binding", the text > states: > > Some advantages of linkage usage: > [...] > 4) you can link BLOB values AS IS. > > What does this mean? For example mySQL do not have binding, So they put BLOB values into query string. And therefore they need do Escape of it. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 22:21:22 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:22:18 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: Message-ID: On 11/18/05 5:48 PM, "Ed Kleban" wrote: >>>> Mmmmm actually this is to write one line of code instead of 2 >>>> >>>> fldLong = curs.Longfield( I ) >>>> >>>> vs >>>> >>>> fld = curs.field( I ) >>>> fldLong = Vlong( fld ) >>>> >>> >>> Ah. Makes sense. I hadn't thought it through but presumed it was for >>> something like that. This would be a good statement to add to the >>> manual. >>> > > Actually, it is already stated in the manual, quite well and very detailed. > That's fast work Ruslan. Thanks! It was there long time. I have not change nothing %-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 22:26:24 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:27:20 2005 Subject: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 6:01 PM, "Ed Kleban" wrote: > Ok, I've read the SQL manual and so now know enough to be dangerous I guess. > I've also gone back and stared the Reference manual a good bit. I think I > understand the abilities and benefits of using SQL vs native API. What I > don't have a good understanding of is how to mix the two when it comes to > manipulation of subsets via VCursors and VSets. First of all. When we say about MIX, we mean not mix in the same function/algoroithm, but - in one function you an make SQL query - in other API searches All depend on task. E.g. JOINS better do in SQL. > Specifically, If I do an SQL query to determine a subset of records of > interest, and obtain a VCursor with say, only a RecID field, then is there a > way I can create a VSet for the corresponding table that only includes those > RecIDs made accessible via the cursor? No. this is wrong think Cursors separately, sets separately . > Conversely, if I have an ArraySet containing a subset of table records of > interest, is there are way to perform an SQL query on only that subset of > records? Again now. At least now . > I'm guessing that the solution to both of these probably involves the > creation of an temporary, intermediate table in memory or disk, that > contains only the recID fields of the records of interest. What I'm not > sure about is the best way to create such a table from either a cursor > resulting from an SQL query, or an existing Vset. > > Do I just use the Vtable constructor commands to create the table manual and > then iterate through either the Cursor or the Set manually in a loop to > populate it? Or is there a better, more automatic way that I'm missing? No you try to go in hard way. No need. Again, simples, single table searches can be made with API. Complex JOINs, distincts with SQL. But still, complex algorithms can be writted with help of Sets+Links for multi table searches and Navigations, which can be more effective than SQL joins. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 22:26:38 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:27:35 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: On 11/18/05 6:15 PM, "Keith DeLong" wrote: >> I think I just remembered what it was. It was a but on windows which wasn't >> saving varchar fields correctly when a number of records were being updating >> very quickly. It has been working fine ever since then. >> >> Cindy >> >>> I afraid no. Just I have fix some show stopper bug for Windows for Cindy. >>> >>> Hmm, not remember which one ... Cindy? >>> >>> That was only for Cindy 1.11 + that bug build > > > Is the latest 1.11 release available for download from your web site Ruslan? Yes, Keith. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From yeomans at desuetude.com Fri Nov 18 15:29:19 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Fri Nov 18 14:30:15 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: References: Message-ID: <8b14facb7502b06304695b16692ec4a6@desuetude.com> On Nov 18, 2005, at 1:29 AM, Ruslan Zasukhin wrote: > On 11/18/05 2:58 AM, "Charles Yeomans" wrote: > >>>>> In terms of C++ >>>>> >>>>> GetString for integer executes itoa() >>>>> >>>> >>>> That is most unfortunate, at least for my interests. So then, we're >>>> back to >>>> where we started, name the point where I stood up on a soapbox and >>>> proclaimed: >>> >>> Hmm. >>> >>> But >>> >>> dim v as integer = fldLong.Value >>> >>> Give you binary value. What can be more ?! >> >> But what about LLong fields? Recall that REALbasic lacks eight-byte >> integer types. > > Sorry, this is question to REAL :-) > > right now V4RB do NOT support int64. > you can assign value only via Strings. > > As only REAL will implement int64, we will add .Value for LLong No, this is question to Paradigma. What does fld.GetString return when fld is a LLong? Does it return an eight-byte string containing the integer data? Does it return a string containing the representation of the string in arabic numerals? I believe this is at least the third time I have asked the same question. If you just don't want to answer, please so inform me. -------------- Charles Yeomans From sunshine at public.kherson.ua Fri Nov 18 22:33:00 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:33:56 2005 Subject: A plan emerges Re: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 8:13 PM, "Ed Kleban" wrote: > So then, the big questions are: > > Does this make sense? > Will it work as intended with the efficiency gains I infer? > What did I miss? Ed, Only tests will show you truth. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 18 22:34:11 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 18 14:35:07 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: <8b14facb7502b06304695b16692ec4a6@desuetude.com> Message-ID: On 11/18/05 10:29 PM, "Charles Yeomans" wrote: >> Sorry, this is question to REAL :-) >> >> right now V4RB do NOT support int64. >> you can assign value only via Strings. >> >> As only REAL will implement int64, we will add .Value for LLong > > No, this is question to Paradigma. What does fld.GetString return when > fld is a LLong? It returns STRING which contains string representatino of value: e.g. 13405674684535 > Does it return an eight-byte string containing the > integer data? No. > Does it return a string containing the representation of > the string in arabic numerals? YES > I believe this is at least the third time I have asked the same > question. If you just don't want to answer, please so inform me. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From delong at redcort.com Fri Nov 18 12:53:42 2005 From: delong at redcort.com (Keith DeLong) Date: Fri Nov 18 14:54:37 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: >> >> Is the latest 1.11 release available for download from your web site Ruslan? > > Yes, Keith. Okay, I wasn't clear.... Where is the link for V4RB 1.x? ;-) I didn't find it at http://www.paradigmasoft.com/en/products/download. Keith DeLong From Ed at Kleban.com Fri Nov 18 15:07:43 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 15:08:43 2005 Subject: DateTimes, LLongs and other 8-byte fields. In-Reply-To: <8b14facb7502b06304695b16692ec4a6@desuetude.com> Message-ID: On 11/18/05 2:29 PM, "Charles Yeomans" wrote: >> Sorry, this is question to REAL :-) >> >> right now V4RB do NOT support int64. >> you can assign value only via Strings. >> >> As only REAL will implement int64, we will add .Value for LLong > > No, this is question to Paradigma. What does fld.GetString return when > fld is a LLong? Does it return an eight-byte string containing the > integer data? Does it return a string containing the representation of > the string in arabic numerals? > > I believe this is at least the third time I have asked the same > question. If you just don't want to answer, please so inform me. > Ruslan answered this question on the list here yesterday in terms I understood quite well: On 11/17/05 5:12 PM, "Ruslan Zasukhin" wrote: > > But fld.GetString() returns normal string which you can show in edit field > for example. > >> Consider >> for example the difference between the definitions in the language reference >> for len(aString) and lenB(aString). >> >> The only way to resolve this is to answer the question Charles asks below. > >> You have to give us a concrete example using quoted string characters "Like >> this" and Hex like this: 0h12F34285 -- without that there's just know >> way to have a good clue of either what longFld.GetSting() does or what you >> mean to say -- short of looking at one of your fine code examples which is >> another way of accomplishing the same thing. > > In terms of C++ > > GetString for integer executes itoa() > From Ed at Kleban.com Fri Nov 18 15:16:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 15:17:51 2005 Subject: A plan emerges Re: Fine Tuning In-Reply-To: Message-ID: On 11/18/05 2:33 PM, "Ruslan Zasukhin" wrote: > On 11/18/05 8:13 PM, "Ed Kleban" wrote: > >> So then, the big questions are: >> >> Does this make sense? >> Will it work as intended with the efficiency gains I infer? >> What did I miss? > > Ed, > > Only tests will show you truth. No problem. Someday I may do some detailed performance testing, but for right now I'm delighted with this approach and am coding away. Thanks for all your assistance. --Ed From chuck at mediamacros.com Fri Nov 18 17:16:08 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Fri Nov 18 16:13:33 2005 Subject: LoadDump making too many files Message-ID: <437E52A8.8090006@mediamacros.com> I am using Valentina 2 for Director. When I do a load dump it is creating 4 files instead of everything in one. The DB the dump was made from was a single file and the first part seems to correctly set the type to kDscDatBlbInd This is the start of my SQL dump... SET PROPERTY DatabaseMode, DatabaseSegment, SchemaVersion, DateFormat, DateSeparator, TimeSeparator, LocaleName, StorageEncoding, CollationFrenchCollation, CollationAlternateHandling, CollationCaseFirst, CollationCaseLevel, CollationNormalizationMode, CollationStrength, CollationHiraganaQuaternaryMode, CollationNumericCollation of DATABASE TO 'kDscDatBlbInd', 32768, 1, 'kMDY', '/', ':', 'en_US', 'UTF-16', 'kOff', 'kNonIgnorable', 'kOff', 'kOff', 'kOff', 'kTertiary', 'kOff', 'kOff'; -- -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer From Ed at Kleban.com Fri Nov 18 16:54:40 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 16:55:39 2005 Subject: Telling tables what their fields are Message-ID: All of the Classes_way examples declare properties of the appropriate VField subclass, and then have a constructor method that employs a constructor for each field using a form such as: mfBornDate = new VDate( "fldBornDate" ) As a result, you can access the VField methods for a table using the VTable subclass properties, but not by using any of the VTable methods, since at the VTable class level, the code is clues of what fields may have been defined. Thus inquries such as: mfBornDate.Table returns what? Nil I suppose. This can presumably be fixed by instead declaring the new VField instances using something like: mfBornDate = self.CreateDateField( "fldBornDate" ) which would then allow you to successfully use inquiries such as: mfBornDate.Table Is there anything wrong with this approach? Thanks! --Ed From Ed at Kleban.com Fri Nov 18 17:29:07 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 17:30:06 2005 Subject: Reassigning Cursor Position Message-ID: If you assign aVCursor.position = 23 when the cursor's position is already 23 will it refetch the current values for those fields or will it do nothing? From Ed at Kleban.com Fri Nov 18 22:47:42 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 18 22:48:40 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/18/05 4:54 PM, "Ed Kleban" wrote: > > All of the Classes_way examples declare properties of the appropriate VField > subclass, and then have a constructor method that employs a constructor for > each field using a form such as: > > mfBornDate = new VDate( "fldBornDate" ) > > As a result, you can access the VField methods for a table using the VTable > subclass properties, but not by using any of the VTable methods, since at > the VTable class level, the code is clues of what fields may have been > defined. > > Thus inquries such as: > > mfBornDate.Table returns what? Nil I suppose. > > > This can presumably be fixed by instead declaring the new VField instances > using something like: > > mfBornDate = self.CreateDateField( "fldBornDate" ) > > which would then allow you to successfully use inquiries such as: > > mfBornDate.Table > Actaully, this appears to have problems. It would work for a VDate perhaps, but not for a subclass of VDate. Unfortunately there doesn't appear to be any way to associate a created field with it's table in a manner that will be reflected by examining the Table through API calls unless the table is created in the API, that correct? There's a similar circumstance it appears for creating VTables as local properties in a VDatabase constructor using "new VTableSubclass". The tables don't get assigned either a name or a database in a manner visible through the Vtable.Name and Vtable.DataBase properties. At least the former is r/w but the latter is r/o. Doesn't the table need to know its database and/or vice versa for things to work consistently? -- or is it just for use in the case where you're using a tool like VS to open and examine the schema structure to permit viewing of the database with an external tool? If I'm interpreting this right, then if you choose to use the classes way as shown in the examples then your database structure isn't sufficiently whole for tools such as VS to view and navigate it. Is that correct? Is that desirable? It there a workaround? --Ed > > Is there anything wrong with this approach? > > Thanks! > --Ed > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Sat Nov 19 08:13:05 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 00:14:05 2005 Subject: Valentina and RB2005 In-Reply-To: Message-ID: On 11/18/05 10:53 PM, "Keith DeLong" wrote: >>> >>> Is the latest 1.11 release available for download from your web site Ruslan? >> >> Yes, Keith. > > Okay, I wasn't clear.... Where is the link for V4RB 1.x? ;-) > > I didn't find it at http://www.paradigmasoft.com/en/products/download. >From here go to V4RB, http://www.paradigmasoft.com/en/products/download/V4RB Here present Deprecated versions. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 19 08:15:14 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 00:16:12 2005 Subject: LoadDump making too many files In-Reply-To: <437E52A8.8090006@mediamacros.com> Message-ID: On 11/19/05 12:16 AM, "Chuck Neal" wrote: Hi Chuck, So you make SQL dump, then do SQL load. And getwrong number of files? Ivan, sounds like a bug. > I am using Valentina 2 for Director. When I do a load dump it is > creating 4 files instead of everything in one. The DB the dump was made > from was a single file and the first part seems to correctly set the > type to kDscDatBlbInd > > This is the start of my SQL dump... > > SET PROPERTY > DatabaseMode, > DatabaseSegment, > SchemaVersion, > DateFormat, > DateSeparator, > TimeSeparator, > LocaleName, > StorageEncoding, > CollationFrenchCollation, > CollationAlternateHandling, > CollationCaseFirst, > CollationCaseLevel, > CollationNormalizationMode, > CollationStrength, > CollationHiraganaQuaternaryMode, > CollationNumericCollation > of DATABASE TO > 'kDscDatBlbInd', > 32768, > 1, > 'kMDY', > '/', > ':', > 'en_US', > 'UTF-16', > 'kOff', > 'kNonIgnorable', > 'kOff', > 'kOff', > 'kOff', > 'kTertiary', > 'kOff', > 'kOff'; -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 19 08:20:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 00:21:39 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 12:54 AM, "Ed Kleban" wrote: Hi Ed, > All of the Classes_way examples declare properties of the appropriate VField > subclass, and then have a constructor method that employs a constructor for > each field using a form such as: > > mfBornDate = new VDate( "fldBornDate" ) right > As a result, you can access the VField methods for a table using the VTable > subclass properties, but not by using any of the VTable methods, since at > the VTable class level, the code is clues of what fields may have been > defined. Right, and THIS IS the main advantage of Class way * you avoid a lots lines in your code as Table.Field( "xxxx" ) so this affect performance. > Thus inquries such as: > > mfBornDate.Table returns what? Nil I suppose. No of course. :-) It returns Table where you have create fields. How it works? Constructor of Vtable work BEFORE start work constructors of fields, So in background it set one global pointer. (inside of plugin) > This can presumably be fixed by instead declaring the new VField instances > using something like: > > mfBornDate = self.CreateDateField( "fldBornDate" ) > > which would then allow you to successfully use inquiries such as: > > mfBornDate.Table > > Is there anything wrong with this approach? STOP STOP STOP. You guess or you really see that .Table returns nil ???? It should work correctly. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 19 08:23:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 00:24:39 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 6:47 AM, "Ed Kleban" wrote: >> All of the Classes_way examples declare properties of the appropriate VField >> subclass, and then have a constructor method that employs a constructor for >> each field using a form such as: >> >> mfBornDate = new VDate( "fldBornDate" ) >> >> As a result, you can access the VField methods for a table using the VTable >> subclass properties, but not by using any of the VTable methods, since at >> the VTable class level, the code is clues of what fields may have been >> defined. >> >> Thus inquries such as: >> >> mfBornDate.Table returns what? Nil I suppose. >> >> >> This can presumably be fixed by instead declaring the new VField instances >> using something like: >> >> mfBornDate = self.CreateDateField( "fldBornDate" ) >> >> which would then allow you to successfully use inquiries such as: >> >> mfBornDate.Table >> > > Actaully, this appears to have problems. It would work for a VDate perhaps, > but not for a subclass of VDate. Unfortunately there doesn't appear to be > any way to associate a created field with it's table in a manner that will > be reflected by examining the Table through API calls unless the table is > created in the API, that correct? NO. should work > There's a similar circumstance it appears for creating VTables as local > properties in a VDatabase constructor using "new VTableSubclass". The > tables don't get assigned either a name or a database in a manner visible > through the Vtable.Name and Vtable.DataBase properties. At least the former > is r/w but the latter is r/o. > > Doesn't the table need to know its database and/or vice versa for things to > work consistently? -- They DO KNOW > or is it just for use in the case where you're using a > tool like VS to open and examine the schema structure to permit viewing of > the database with an external tool? > > If I'm interpreting this right, then if you choose to use the classes way as > shown in the examples then your database structure isn't sufficiently whole > for tools such as VS to view and navigate it. Is that correct? Is that > desirable? It there a workaround? Again STOP. Tools as Valentina Studio and viSQL cannot use Class Way. NEVER. because they do not know structure of any db they going to open Class Way can be used only when you EXACTLY know future structure of your db. In YOUR app you will know this. You have draw diagram -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 19 08:24:58 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 00:25:57 2005 Subject: Reassigning Cursor Position In-Reply-To: Message-ID: On 11/19/05 1:29 AM, "Ed Kleban" wrote: > If you assign aVCursor.position = 23 when the cursor's position is already > 23 will it refetch the current values for those fields or will it do > nothing? You mean that you have change original values and do .Postion = current position number? Valentina will reload original values -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sat Nov 19 07:08:11 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 07:09:11 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 12:20 AM, "Ruslan Zasukhin" wrote: > On 11/19/05 12:54 AM, "Ed Kleban" wrote: > > Hi Ed, > >> All of the Classes_way examples declare properties of the appropriate VField >> subclass, and then have a constructor method that employs a constructor for >> each field using a form such as: >> >> mfBornDate = new VDate( "fldBornDate" ) > > right > >> As a result, you can access the VField methods for a table using the VTable >> subclass properties, but not by using any of the VTable methods, since at >> the VTable class level, the code is clues of what fields may have been >> defined. > > Right, and THIS IS the main advantage of Class way > > * you avoid a lots lines in your code as > Table.Field( "xxxx" ) > > so this affect performance. > Yes. Understood. That's a good thing! >> Thus inquries such as: >> >> mfBornDate.Table returns what? Nil I suppose. > > No of course. :-) > > It returns Table where you have create fields. > How it works? > > Constructor of Vtable work BEFORE start work constructors of fields, > So in background it set one global pointer. (inside of plugin) Eeek! So... hmm. So, presumably Tables are associated with a database in a similar manner with a different hidden global variable. Therefore: 1) All Tables created with "new VTableClass(...)" AFTER a database is created via "new VDatabaseClass(...)" are automatically associated with the last created database. 2) All Fields created with "new VFieldClass(...)" AFTER a table is created via "new VTableClass(...)" are automatically associated with the last created table. 3) The call pattern documented in the example where a Database constructor creates its tables and Table constructors create their fields, makes this a natural order of occurrence by virtue of defining and instantiating classes. If true, then yeah... I get it. It's very clever, indeed very elegant. For my personal tastes perhaps a bit too clever. I'm not sure I entirely like it. But I get it. I would prefer to see Table constructors that take an optional Database argument, and Field constructors that take an optional Table argument, and when no argument or nil was supplied defaults to the behavior that you've implemented but essentially have hidden. That would allow more complex definition strategies whereby you don't necessary have to define tables and fields in the order your methods mandate -- but still could. More importantly it would give the coder the option of explicitly including the arguments even if they were not needed by virtue of being identical to the defaults, so that: 1) The code would be more readable and 2) It would be absolutely explicit what was going on and that Tables were being linked with Databases and Fields with Tables. But I get it and can work with it now that I understand it. Thanks! You should DEFINITELY include this in the documentation. Please :) > >> This can presumably be fixed by instead declaring the new VField instances >> using something like: >> >> mfBornDate = self.CreateDateField( "fldBornDate" ) >> >> which would then allow you to successfully use inquiries such as: >> >> mfBornDate.Table >> >> Is there anything wrong with this approach? > > STOP STOP STOP. > > You guess or you really see that .Table returns nil ???? Haven't seen it, it was just a guess. Clearly a wrong guess. Thanks for the enlightenment. > It should work correctly. > > I believe you. Thanks! --Ed From Ed at Kleban.com Sat Nov 19 07:22:01 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 07:23:04 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 12:23 AM, "Ruslan Zasukhin" wrote: > On 11/19/05 6:47 AM, "Ed Kleban" wrote: > >> If I'm interpreting this right, then if you choose to use the classes way as >> shown in the examples then your database structure isn't sufficiently whole >> for tools such as VS to view and navigate it. Is that correct? Is that >> desirable? It there a workaround? > > Again STOP. > > Tools as Valentina Studio and viSQL cannot use Class Way. NEVER. > because they do not know structure of any db they going to open > Yes, but I want to "eat my cake and have it too" as we say here. I want to use classes way to define my databases, and then be able to inspect them on disk with VS :) So let's talk a bit about how we might fake that or what some of the inherent problems are... I accept what you say, namely that "[This can] NEVER [be done] because [VS and viSQL] do not know [the] structure of any db they are going to open." One of the options for saving out your database is to save the schema in a separate file from the rest of the data. Let's say I did that for my database created with the classes way, but then took the schema saved from another database defined the SQL way, but which was not populated with any data. Could I combine the Schema from the SQL way with the data files from the Classes way, and -- assuming they were correctly coded to match -- end up with a database that my RB code could reopen and use and which a VS or viSQL tool could also browse? If so, don't I have enough information within my RB environment to parse the structure of the daabase I create and essentailly generate the SQL code capable of creating this separate database? And if so, can't I then just feed that into SQL from within the application to create a separate database from which I can later move the schema file over and use? > Class Way can be used only when you EXACTLY know future structure of your > db. In YOUR app you will know this. You have draw diagram Yes. From sunshine at public.kherson.ua Sat Nov 19 16:41:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 08:42:10 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 3:08 PM, "Ed Kleban" wrote: >> Constructor of Vtable work BEFORE start work constructors of fields, >> So in background it set one global pointer. (inside of plugin) > > Eeek! So... hmm. So, presumably Tables are associated with a database in a > similar manner with a different hidden global variable. Therefore: > > 1) All Tables created with "new VTableClass(...)" AFTER a database is > created via "new VDatabaseClass(...)" are automatically associated with the > last created database. > > 2) All Fields created with "new VFieldClass(...)" AFTER a table is created > via "new VTableClass(...)" are automatically associated with the last > created table. > > 3) The call pattern documented in the example where a Database constructor > creates its tables and Table constructors create their fields, makes this a > natural order of occurrence by virtue of defining and instantiating classes. > > If true, then yeah... I get it. It's very clever, indeed very elegant. Yes right. > For my personal tastes perhaps a bit too clever. I'm not sure I entirely like > it. But I get it. Well, you can use just API way. But if you will start code with API way you may very fast note many repeated lines of code. If you will start to think how avoid them, you will come to above model. > I would prefer to see Table constructors that take an optional Database > argument, and Field constructors that take an optional Table argument, and > when no argument or nil was supplied defaults to the behavior that you've > implemented but essentially have hidden. > That would allow more complex > definition strategies whereby you don't necessary have to define tables and > fields in the order your methods mandate -- but still could. I will try explain below WHY such and only such order if offered. > More > importantly it would give the coder the option of explicitly including the > arguments even if they were not needed by virtue of being identical to the > defaults, so that: 1) The code would be more readable and 2) It would be > absolutely explicit what was going on and that Tables were being linked with > Databases and Fields with Tables. :-) Hey man! I have spend probably 10 long minutes inventing this mechanism of "hidden variables", exactly to avoid this parameter! > But I get it and can work with it now that I understand it. Thanks! > > You should DEFINITELY include this in the documentation. Please :) Now why order MUST BE such and only such. * in first I have use this technique in C++. later I have to see that Java, REALbasic, C# all have similar abilities to get the same technique. * lets looks on ANY class of REALbasic (for this example). we do not talk about V4RB and Valentina now. * Let we have one or few leaf classes (VFields) class LeafA Constructor() end class LeafB Constructor() end * Now let we have some parent class that must be supervisor of leafs class Tree properties: mi as integer ms as string mf1 as LeafA mf2 as LeafB sub constructor // By rules of OO design you MUST init data members of class // in the constructor. So we dothis mi = 0 ms = "aaaaa" mf1 = new LeafA() mf2 = new LeafB() end end So you should create tables in constructor of db And you should create fields in constructor of table. So you get correct hierarchy of objects by line db = new MyDB -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 19 16:43:05 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 08:44:04 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 3:22 PM, "Ed Kleban" wrote: >> Tools as Valentina Studio and viSQL cannot use Class Way. NEVER. >> because they do not know structure of any db they going to open >> > > Yes, but I want to "eat my cake and have it too" as we say here. I want to > use classes way to define my databases, and then be able to inspect them on > disk with VS :) You can this. No problems here. I think you have missunderstnad. you can develop YOUR application with Class way. viSQL was developed with API way > So let's talk a bit about how we might fake that or what some of the > inherent problems are... > > I accept what you say, namely that "[This can] NEVER [be done] because [VS > and viSQL] do not know [the] structure of any db they are going to open." > One of the options for saving out your database is to save the schema in a > separate file from the rest of the data. Let's say I did that for my > database created with the classes way, but then took the schema saved from > another database defined the SQL way, but which was not populated with any > data. Could I combine the Schema from the SQL way with the data files from > the Classes way, and -- assuming they were correctly coded to match -- end > up with a database that my RB code could reopen and use and which a VS or > viSQL tool could also browse? > > If so, don't I have enough information within my RB environment to parse the > structure of the daabase I create and essentailly generate the SQL code > capable of creating this separate database? > > And if so, can't I then just feed that into SQL from within the application > to create a separate database from which I can later move the schema file > over and use? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sat Nov 19 08:58:06 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 08:59:05 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 8:41 AM, "Ruslan Zasukhin" wrote: > Yes right. > >> For my personal tastes perhaps a bit too clever. I'm not sure I entirely >> like >> it. But I get it. > > Well, you can use just API way. But if you will start code with API way you > may very fast note many repeated lines of code. If you will start to think > how avoid them, you will come to above model. > Oh no, I fully intend to use the classes way. I like the elegance of it very much. It's the "hidden" aspect of it that I find unaesthetic. >> I would prefer to see Table constructors that take an optional Database >> argument, and Field constructors that take an optional Table argument, and >> when no argument or nil was supplied defaults to the behavior that you've >> implemented but essentially have hidden. > >> That would allow more complex >> definition strategies whereby you don't necessary have to define tables and >> fields in the order your methods mandate -- but still could. > > I will try explain below WHY such and only such order if offered. > >> More >> importantly it would give the coder the option of explicitly including the >> arguments even if they were not needed by virtue of being identical to the >> defaults, so that: 1) The code would be more readable and 2) It would be >> absolutely explicit what was going on and that Tables were being linked with >> Databases and Fields with Tables. > > :-) > > Hey man! I have spend probably 10 long minutes inventing this mechanism of > "hidden variables", exactly to avoid this parameter! > LOL. Different strokes for different folks! But making it an optional parameter that defaults to the behavior it has now would permit both modes of usage, and more importantly as I note above, make it explicit. >> But I get it and can work with it now that I understand it. Thanks! >> >> You should DEFINITELY include this in the documentation. Please :) > > Now why order MUST BE such and only such. > > * in first I have use this technique in C++. > later I have to see that Java, REALbasic, C# all have similar abilities > to get the same technique. > > * lets looks on ANY class of REALbasic (for this example). > we do not talk about V4RB and Valentina now. > > > * Let we have one or few leaf classes (VFields) > > class LeafA > > Constructor() > > end > > class LeafB > > Constructor() > > end > > > * Now let we have some parent class that must be supervisor of leafs > > class Tree > > properties: > mi as integer > ms as string > mf1 as LeafA > mf2 as LeafB > > sub constructor > // By rules of OO design you MUST init data members of class > // in the constructor. So we dothis > > mi = 0 > ms = "aaaaa" > > mf1 = new LeafA() > mf2 = new LeafB() > end > end > > So you should create tables in constructor of db > And you should create fields in constructor of table. > > So you get correct hierarchy of objects by line > > db = new MyDB > Yes, I understand how it works. As I say it's quite elegant, and I intend to use it just this way. You talk about the way "you should" do things, which reflects your opinions on how people should write code. And what you recommend is in keeping with object-oriented principles -- which is great. I'm simply expressing my opinions that: 1) I prefer a coding style that permits "you can" rather than "you should" or "you must". 2) That DOES NOT USE GLOBAL VARIABLES TO HIDE MECHANISMS, but rather makes it explicit, or allows the user to optionally make it explicit, what is happening. To do otherwise simply goes against my personal opinions of what "you should" do when writing code. Using global variables like this doesn't rank as high on my list of evil coding practices as prolific use of GOTO statements. But it is on my list. Don't sweat it Ruslan, it's just a difference in design philosophy and coding styles. Since you wrote the code, you get to pick the style :) That's fine. I'll use it and enjoy it. Just offering you a counter opinion of how I might have done it differently and why. From Ed at Kleban.com Sat Nov 19 09:00:47 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 09:01:46 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 8:43 AM, "Ruslan Zasukhin" wrote: > On 11/19/05 3:22 PM, "Ed Kleban" wrote: > >>> Tools as Valentina Studio and viSQL cannot use Class Way. NEVER. >>> because they do not know structure of any db they going to open >>> >> >> Yes, but I want to "eat my cake and have it too" as we say here. I want to >> use classes way to define my databases, and then be able to inspect them on >> disk with VS :) > > You can this. No problems here. > Ah great! Then I'm a happy camper. :) > I think you have missunderstnad. > > you can develop YOUR application with Class way. > viSQL was developed with API way > > Yes, I did misunderstand. Thanks for the clarification. I see what you meant now after re-reading your comment in a different context. Thanks! --Ed From sunshine at public.kherson.ua Sat Nov 19 17:10:01 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sat Nov 19 09:11:01 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 4:58 PM, "Ed Kleban" wrote: >> Hey man! I have spend probably 10 long minutes inventing this mechanism of >> "hidden variables", exactly to avoid this parameter! >> > > LOL. Different strokes for different folks! But making it an optional > parameter that defaults to the behavior it has now would permit both modes > of usage, and more importantly as I note above, make it explicit. If I will make parameter, then you get ability to bad things, e.g. In constructor of T1, you will write T2. So this is way to problems IMHO -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sat Nov 19 09:15:32 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 09:16:31 2005 Subject: Telling tables what their fields are In-Reply-To: Message-ID: On 11/19/05 9:10 AM, "Ruslan Zasukhin" wrote: > On 11/19/05 4:58 PM, "Ed Kleban" wrote: > >>> Hey man! I have spend probably 10 long minutes inventing this mechanism of >>> "hidden variables", exactly to avoid this parameter! >>> >> >> LOL. Different strokes for different folks! But making it an optional >> parameter that defaults to the behavior it has now would permit both modes >> of usage, and more importantly as I note above, make it explicit. > > If I will make parameter, then you get ability to bad things, > > e.g. In constructor of T1, you will write T2. > > So this is way to problems IMHO A valid point. From bkeeney at everestkc.net Sat Nov 19 14:48:17 2005 From: bkeeney at everestkc.net (Bob Keeney) Date: Sat Nov 19 14:49:20 2005 Subject: Val Studio Error Message Question In-Reply-To: <20051118075754.D5F633CE674@edison.macserve.net> References: <20051118075754.D5F633CE674@edison.macserve.net> Message-ID: <4BFBA129-D02A-4250-A6B3-86CFA50EC46F@everestkc.net> On Nov 18, 2005, at 1:57 AM, valentina-request@lists.macserve.net wrote: > >> What does the following error mean? >> >> 20:59:56: Kernel error: 0x59504 >> 20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is >> already open. >> 20:59:56: Database could not be opened! > > Yes, this is corruption. > >> Is there a way to recover this database? > > Try new way first of all: > > db.Clone() > > You can even use V4RB example Api/Db_Clone as utility todo this. > > Another way -- XML dump - Load Using the dbclone example, I get the same error message in RB. Same thing with the XML dump example. Bob K. From Ed at Kleban.com Sat Nov 19 21:35:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sat Nov 19 21:36:54 2005 Subject: Toggling EVFlag.fIndexed Message-ID: Is it faster to fill up a single-field table and then enable indexing than it is to fill up the table while indexing is enabled? I'd think so. From Ed at Kleban.com Sun Nov 20 00:43:20 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 00:44:24 2005 Subject: Setting Unique Message-ID: The Valentina Kernel manual states: " If the flag Unique is changed at runtime and a table is not empty, then the index will be automatically rebuilt. This occurs because Valentina uses different formats for indexing unique and non unique fields. The unique index has more compact format because it does not need to store a count of records for each value (for unique field it is always 1). " But what happens if you set the unique flag, and the table gets indexed, but it turns out the value really aren't unique? From sunshine at public.kherson.ua Sun Nov 20 08:50:47 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 00:51:49 2005 Subject: Val Studio Error Message Question In-Reply-To: <4BFBA129-D02A-4250-A6B3-86CFA50EC46F@everestkc.net> Message-ID: On 11/19/05 10:48 PM, "Bob Keeney" wrote: >>> What does the following error mean? >>> >>> 20:59:56: Kernel error: 0x59504 >>> 20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is >>> already open. >>> 20:59:56: Database could not be opened! >> >> Yes, this is corruption. >> >>> Is there a way to recover this database? >> >> Try new way first of all: >> >> db.Clone() >> >> You can even use V4RB example Api/Db_Clone as utility todo this. >> >> Another way -- XML dump - Load > > > Using the dbclone example, I get the same error message in RB. > > Same thing with the XML dump example. You mean that during Clone or XML dump you get >>> 20:59:56: File "Volume: My DB_2.dat: Embedded File: 0x300030000" is >>> already open. >>> 20:59:56: Database could not be opened! Ah, so you even cannot open db? If database is not big and you can send it to us ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 08:51:11 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 00:52:12 2005 Subject: Toggling EVFlag.fIndexed In-Reply-To: Message-ID: On 11/20/05 5:35 AM, "Ed Kleban" wrote: > Is it faster to fill up a single-field table and then enable indexing than > it is to fill up the table while indexing is enabled? > > I'd think so. yes -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 08:51:55 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 00:52:56 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 8:43 AM, "Ed Kleban" wrote: > The Valentina Kernel manual states: > > " If the flag Unique is changed at runtime and a table is not empty, then > the index will be automatically rebuilt. This occurs because Valentina uses > different formats for indexing unique and non unique fields. The unique > index has more compact format because it does not need to store a count of > records for each value (for unique field it is always 1). " > > But what happens if you set the unique flag, and the table gets indexed, but > it turns out the value really aren't unique? Valentina refuse add such record. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 00:59:01 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 01:00:03 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 12:51 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 8:43 AM, "Ed Kleban" wrote: > >> The Valentina Kernel manual states: >> >> " If the flag Unique is changed at runtime and a table is not empty, then >> the index will be automatically rebuilt. This occurs because Valentina uses >> different formats for indexing unique and non unique fields. The unique >> index has more compact format because it does not need to store a count of >> records for each value (for unique field it is always 1). " >> >> But what happens if you set the unique flag, and the table gets indexed, but >> it turns out the value really aren't unique? > > Valentina refuse add such record. I'm not sure what that means. If you have already added all the records and then enable the unique flag, what happens? Does Valentina refuse to set the unique flag? Or does the flag get set but the items are not unique? Or does Valentina throw away any non-unique records? From Ed at Kleban.com Sun Nov 20 01:01:16 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 01:02:17 2005 Subject: Indexing or sorting ordered arrays. Message-ID: What sorting algorithm does Valentina use? Is the algorithm similar to Quicksort that will perform poorly when sorting or indexing an array that is already correctly ordered? Or is it an IntroSort algorithm that degrades nicely? Or does it randomize a few elements prior to sorting to avoid the Quicksort performance problems? Or something else? From sunshine at public.kherson.ua Sun Nov 20 09:17:54 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 01:18:57 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 8:59 AM, "Ed Kleban" wrote: >>> The Valentina Kernel manual states: >>> >>> " If the flag Unique is changed at runtime and a table is not empty, then >>> the index will be automatically rebuilt. This occurs because Valentina uses >>> different formats for indexing unique and non unique fields. The unique >>> index has more compact format because it does not need to store a count of >>> records for each value (for unique field it is always 1). " >>> >>> But what happens if you set the unique flag, and the table gets indexed, but >>> it turns out the value really aren't unique? >> >> Valentina refuse add such record. > > I'm not sure what that means. If you have already added all the records and > then enable the unique flag, what happens? Ah, I see > Does Valentina refuse to set the unique flag? YES. It will try to build index and fail, And set OFF flag again > Or does the flag get set but the items are not unique? No, because this give us incorrect state. Valentina try ALWAYS refuse incorrect states. > Or does Valentina throw away any non-unique records? Of course no. this is data of users!!! -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 09:18:50 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 01:19:57 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 9:01 AM, "Ed Kleban" wrote: > What sorting algorithm does Valentina use? Is the algorithm similar to > Quicksort that will perform poorly when sorting or indexing an array that is > already correctly ordered? Or is it an IntroSort algorithm that degrades > nicely? Or does it randomize a few elements prior to sorting to avoid the > Quicksort performance problems? Or something else? Uses where ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 01:31:24 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 01:32:26 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: When you set the fIndexed EVFlag, then V2 builds an index, correct? Presumably it uses some sort algorithm to do this. Similarly when you call the Vtable.Sort command, it again uses some algorithm to sort. I am curious what the algorithm used is, because I would like to know the behavior when the column of items to index, or the column of items to sort is already sorted. I'm curious if when the items are already sorted: 1) Performing a sort on them may be very slow, because QuickSort is used, and Quicksort has poor performance when used to sort a list that is already sorted, or... 2) Performing a sort on them is very fast because a serialization check is made to see if they are already sorted and if so nothing is done, or... 3) Performing a sort on them is not terrible, because IntroSort is used instead of QuickSort which has much better worst-case sorting efficiency when the list is already sorted, or... 4) QuickSort is used, however before it is use several of list elements are randomly reordered to avoid the performance problems that occur when the list is already sorted. 5) I'm totally clueless because you're simply not indexing or sorting in any of these ways. On 11/20/05 1:18 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 9:01 AM, "Ed Kleban" wrote: > >> What sorting algorithm does Valentina use? Is the algorithm similar to >> Quicksort that will perform poorly when sorting or indexing an array that is >> already correctly ordered? Or is it an IntroSort algorithm that degrades >> nicely? Or does it randomize a few elements prior to sorting to avoid the >> Quicksort performance problems? Or something else? > > Uses where ? From sunshine at public.kherson.ua Sun Nov 20 09:39:23 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 01:40:27 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 9:31 AM, "Ed Kleban" wrote: > When you set the fIndexed EVFlag, then V2 builds an index, correct? > Presumably it uses some sort algorithm to do this. Similarly when you call > the Vtable.Sort command, it again uses some algorithm to sort. I am curious > what the algorithm used is, because I would like to know the behavior when > the column of items to index, or the column of items to sort is already > sorted. I'm curious if when the items are already sorted: > 1) Performing a sort on them may be very slow, because QuickSort is used, > and Quicksort has poor performance when used to sort a list that is already > sorted, or... > > 2) Performing a sort on them is very fast because a serialization check is > made to see if they are already sorted and if so nothing is done, or... > 3) Performing a sort on them is not terrible, because IntroSort is used > instead of QuickSort which has much better worst-case sorting efficiency > when the list is already sorted, or... > > 4) QuickSort is used, however before it is use several of list elements are > randomly reordered to avoid the performance problems that occur when the > list is already sorted. > > 5) I'm totally clueless because you're simply not indexing or sorting in any > of these ways. Actually nature of DBMS indexing is sort of inserts. For some cases of small sets we use just quicksort, but not recursive -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 01:55:40 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 01:56:42 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 1:39 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 9:31 AM, "Ed Kleban" wrote: > >> When you set the fIndexed EVFlag, then V2 builds an index, correct? > >> >> 5) I'm totally clueless because you're simply not indexing or sorting in any >> of these ways. > > Actually nature of DBMS indexing is sort of inserts. Doh! Of course. It's incremental to deal with a single record add at a time. Makes perfect sense for adding individual records. So then re-indexing an already or mostly sorted array would be very fast. Yep, yep. But that also suggests that if I'm going to supply a very large array of randomly ordered elements and then ebabke the "Unique" or "Indexed" flag, that I might be better off presorting them first myself, since an insertion sort in that circumstance could take much longer... that is if I happened to have a plugin or RB-written sort routine that could compete with the speed of your C++ compiled insertion sort. True? Doesn't much matter. Most of the arrays I'm concerned with right now all happen to be pre-sorted. Thanks! > For some cases of small sets we use just quicksort, but not recursive From Ed at Kleban.com Sun Nov 20 01:58:05 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 01:59:07 2005 Subject: Setting Unique In-Reply-To: Message-ID: >>>> But what happens if you set the unique flag, and the table gets indexed, >>>> but >>>> it turns out the value really aren't unique? >>> >>> Valentina refuse add such record. >> >> I'm not sure what that means. If you have already added all the records and >> then enable the unique flag, what happens? > > Ah, I see > >> Does Valentina refuse to set the unique flag? > > YES. It will try to build index and fail, > And set OFF flag again > Ah, good. So I can either look for an error being thrown, and/or test the state of the Unique flag after setting it. Great. From sunshine at public.kherson.ua Sun Nov 20 10:11:38 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 02:12:51 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 9:58 AM, "Ed Kleban" wrote: >>>>> But what happens if you set the unique flag, and the table gets indexed, >>>>> but >>>>> it turns out the value really aren't unique? >>>> >>>> Valentina refuse add such record. >>> >>> I'm not sure what that means. If you have already added all the records and >>> then enable the unique flag, what happens? >> >> Ah, I see >> >>> Does Valentina refuse to set the unique flag? >> >> YES. It will try to build index and fail, >> And set OFF flag again >> > > Ah, good. So I can either look for an error being thrown, and/or test the > state of the Unique flag after setting it. Actually I no self wonder. v2 allow you set fUnique, but still do not have fIndexed. I.e. Index building will happens later. I not remember if we have protect SUCH case ** in V1 when you set fUnique, Valentina self have set fIndexed in the same time. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 02:13:15 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 02:14:17 2005 Subject: Bloblets Message-ID: Is there any way to read just a little bit of data from a Blob or Text without having to read the entire amount of data into memory? It would be nice to be able to request 100 bytes starting from byte 10293 for example. From sunshine at public.kherson.ua Sun Nov 20 10:16:48 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 02:17:51 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 9:55 AM, "Ed Kleban" wrote: > On 11/20/05 1:39 AM, "Ruslan Zasukhin" wrote: > >> On 11/20/05 9:31 AM, "Ed Kleban" wrote: >> >>> When you set the fIndexed EVFlag, then V2 builds an index, correct? >> >>> >>> 5) I'm totally clueless because you're simply not indexing or sorting in any >>> of these ways. >> >> Actually nature of DBMS indexing is sort of inserts. > > Doh! Of course. It's incremental to deal with a single record add at a > time. Makes perfect sense for adding individual records. So then > re-indexing an already or mostly sorted array would be very fast. Yep, yep. > But that also suggests that if I'm going to supply a very large array of > randomly ordered elements and then ebabke the "Unique" or "Indexed" flag, > that I might be better off presorting them first myself, since an insertion > sort in that circumstance could take much longer... that is if I happened to > have a plugin or RB-written sort routine that could compete with the speed > of your C++ compiled insertion sort. True? > > Doesn't much matter. Most of the arrays I'm concerned with right now all > happen to be pre-sorted. Ed, Valentina take care self to do best things. Valentina do very complex indexing. We have size of code related to indexing, bigger than SqlLite itself is. :-) Again I will repeat. IT is good idea at first make prototype using simple logic. And ONLY if you will see some performance problems there is sense start think on improvements. Most probably you will not see them at all. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 10:17:59 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 02:19:00 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 10:13 AM, "Ed Kleban" wrote: > Is there any way to read just a little bit of data from a Blob or Text > without having to read the entire amount of data into memory? > > It would be nice to be able to request 100 bytes starting from byte 10293 > for example. We did have such feature in 1.x but nobody have use it, So we have not open it for v2 for now. Also always remember about disk, cache, ... Here not always faster to read 100 bytes then then e.g. 4Kb -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 07:14:02 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 07:15:05 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 2:11 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 9:58 AM, "Ed Kleban" wrote: > >>>>>> But what happens if you set the unique flag, and the table gets indexed, >>>>>> but >>>>>> it turns out the value really aren't unique? >>>>> >>>>> Valentina refuse add such record. >>>> >>>> I'm not sure what that means. If you have already added all the records >>>> and >>>> then enable the unique flag, what happens? >>> >>> Ah, I see >>> >>>> Does Valentina refuse to set the unique flag? >>> >>> YES. It will try to build index and fail, >>> And set OFF flag again >>> >> >> Ah, good. So I can either look for an error being thrown, and/or test the >> state of the Unique flag after setting it. > > Actually I no self wonder. > > v2 allow you set fUnique, but still do not have fIndexed. > I.e. Index building will happens later. > I not remember if we have protect SUCH case Ok, then let me rephrase that: So I can set first the unique and then the indexed flags, and look to catch any error that may be thrown, and then check the value of the unique flag? > > ** in V1 when you set fUnique, Valentina self have set fIndexed in the same > time. > From Ed at Kleban.com Sun Nov 20 07:32:24 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 07:33:25 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 2:16 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 9:55 AM, "Ed Kleban" wrote: > >> On 11/20/05 1:39 AM, "Ruslan Zasukhin" wrote: >> >>> On 11/20/05 9:31 AM, "Ed Kleban" wrote: >>> >>>> When you set the fIndexed EVFlag, then V2 builds an index, correct? >>> >>>> >>>> 5) I'm totally clueless because you're simply not indexing or sorting in >>>> any >>>> of these ways. >>> >>> Actually nature of DBMS indexing is sort of inserts. >> >> Doh! Of course. It's incremental to deal with a single record add at a >> time. Makes perfect sense for adding individual records. So then >> re-indexing an already or mostly sorted array would be very fast. Yep, yep. > > >> But that also suggests that if I'm going to supply a very large array of >> randomly ordered elements and then ebabke the "Unique" or "Indexed" flag, > >> that I might be better off presorting them first myself, since an insertion >> sort in that circumstance could take much longer... that is if I happened to >> have a plugin or RB-written sort routine that could compete with the speed >> of your C++ compiled insertion sort. True? >> >> Doesn't much matter. Most of the arrays I'm concerned with right now all >> happen to be pre-sorted. > > Ed, Valentina take care self to do best things. > > Valentina do very complex indexing. We have size of code related to > indexing, bigger than SqlLite itself is. :-) > > > Again I will repeat. IT is good idea at first make prototype using simple > logic. And ONLY if you will see some performance problems there is sense > start think on improvements. Most probably you will not see them at all. > That is very good advice, and well worth repeating. It is good advice for any programming project to do a decent design, make a rapid prototype, meter the heck out of it, and tune the performance where and if needed. You get far more progress in far less time. That said, I ask such questions for several reasons, including: 1) I have an inherent curiosity not only about how to use Valentina, but how it works. You're certainly under no obligation to share that information -- it may well even be confidential. That's fine! I don't want to press you regarding the recipe for the "special sauce" that distinguishes Valentina from its Rivals. To the extent you are willing to answer such questions however, it helps me better understand the capabilities, limitations, and design tradeoffs in using the product as tool. To the extent you prefer not to answer or simply don't know (and don't even care about knowing) I can write and run some tests if it's so important to me that I really need to know -- typically it's not. 2) I'm not writing code from scratch. Rather I'm tacking on Valentina to a very large existing code base which gives me a large number of options I normally wouldn't have if implementing a design from scratch. As an example, when I want to populate a table, I may well have the data available to do so in two or three different forms -- some random, some already sorted, some hashed for faster access than Valentina will ever be able to match. Better understanding of Valentina's capabilities and limitations helps me to make better choices for using or transferring these sources to V2. 3) As previously discussed, we share a common addiction and I simply can't help myself :-) ====== Sounds like I'm probably at the point of diminishing returns at this point and am rapidly becoming more of a pest than a list contributor. I'll endeavor to tighten the clamp on my curiosity and make less demands on your generous penchant for sharing information. I certainly don't want to wear out my welcome with one of the best tutors on structural efficiency I've run into in a very long time. In that vein, I'll stop short of listing reasons 4), 5), and 6). From Ed at Kleban.com Sun Nov 20 07:47:27 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 07:48:28 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 2:17 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 10:13 AM, "Ed Kleban" wrote: > >> Is there any way to read just a little bit of data from a Blob or Text >> without having to read the entire amount of data into memory? >> >> It would be nice to be able to request 100 bytes starting from byte 10293 >> for example. > > We did have such feature in 1.x but nobody have use it, > So we have not open it for v2 for now. > That is most unfortunate, since it means I will have to do so manually outside of the database if I want to avoid unneccessary disk access -- which is one of the key things I'm looking to the database to manage for me so that I don't have to. Specifically, I need to store a large number of strings that will range in size from typically, say... 2 MB to 20 MB. I then need to perform completely random accesses to pull out substrings from those strings, the position and length of which are stored in database tables. Sometimes some of those 2-20 MB string will be in memory already, but other times they will only be on disk and likely not in either Valentina's or the OS's cache. Since Valentina doesn't support substring access, I'll have to store every such string in a separate file outside of the database, store the filenames in V2, and use the OS to perform the random accesses -- or read in the whole thing and suffer the performance consequences. Not a difficult or unmanageable task, but like I said... unfortunate. > Also always remember about disk, cache, ... > > Here not always faster to read 100 bytes then then e.g. 4Kb > I fully expect to have to do a disk access to pull in a cache page or two, both into the OS cache as well as Valentina's, if I need a 100 byte substring from a 10 MB source. I don't expect to have to read 10 MB worth of pages when I only need 100 bytes. From sunshine at public.kherson.ua Sun Nov 20 15:52:18 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 07:53:19 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 3:14 PM, "Ed Kleban" wrote: > > So I can set first the unique and then the indexed flags, and look to catch > any error that may be thrown, and then check the value of the unique flag? > Yes. Just not remember, what exactly flag will be set OFF by Valentina 2 -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 08:00:20 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 08:01:21 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 7:52 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 3:14 PM, "Ed Kleban" wrote: > >> >> So I can set first the unique and then the indexed flags, and look to catch >> any error that may be thrown, and then check the value of the unique flag? >> > > Yes. > > Just not remember, what exactly flag will be set OFF by Valentina 2 Pardon? Are you saying that you don't remember whether in Valentina 2 if the unique flag will be turned off if the data turns out to not be unique? Or did you mean something else? From sunshine at public.kherson.ua Sun Nov 20 16:00:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 08:01:37 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 3:32 PM, "Ed Kleban" wrote: Hi Ed, > hat said, I ask such questions for several reasons, including: > > 1) I have an inherent curiosity not only about how to use Valentina, but how > it works. This is other question :-) Just I have worried that you try go in too difficult way. > You're certainly under no obligation to share that information -- it may > well even be confidential. That's fine! I don't want to press you > regarding the recipe for the "special sauce" that distinguishes Valentina > from its Rivals. To the extent you are willing to answer such questions > however, it helps me better understand the capabilities, limitations, and > design tradeoffs in using the product as tool. To the extent you prefer not > to answer or simply don't know (and don't even care about knowing) I can > write and run some tests if it's so important to me that I really need to > know -- typically it's not. Ok :-) > 2) I'm not writing code from scratch. Rather I'm tacking on Valentina to a > very large existing code base which gives me a large number of options I > normally wouldn't have if implementing a design from scratch. > As an example, when I want to populate a table, I may well have the data > available to do so in two or three different forms -- some random, some > already sorted, some hashed for faster access than Valentina will ever be > able to match. Better understanding of Valentina's capabilities and > limitations helps me to make better choices for using or transferring these > sources to V2. right > 3) As previously discussed, we share a common addiction and I simply can't > help myself :-) > > > ====== > > Sounds like I'm probably at the point of diminishing returns at this point > and am rapidly becoming more of a pest than a list contributor. I'll > endeavor to tighten the clamp on my curiosity and make less demands on your > generous penchant for sharing information. I certainly don't want to wear > out my welcome with one of the best tutors on structural efficiency I've run > into in a very long time. > > In that vein, I'll stop short of listing reasons 4), 5), and 6). You are welcome. And questions are great. Just I have learned few times for last 2-3 years that "theoretical thinking" sometimes do not happens in life. Some "very cool" optimizations, as I did think, in fact force system to be slower. When I have disable them system have work faster. Just not go into the same trap :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 16:03:34 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 08:04:37 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 3:47 PM, "Ed Kleban" wrote: >> We did have such feature in 1.x but nobody have use it, >> So we have not open it for v2 for now. >> > > That is most unfortunate, since it means I will have to do so manually > outside of the database if I want to avoid unneccessary disk access -- which > is one of the key things I'm looking to the database to manage for me so > that I don't have to. Specifically, I need to store a large number of > strings that will range in size from typically, say... 2 MB to 20 MB. I > then need to perform completely random accesses to pull out substrings from > those strings, the position and length of which are stored in database > tables. Hmm. 3 times Hmm. You sure this is best possible solution ?! I doubt. > Sometimes some of those 2-20 MB string will be in memory already, > but other times they will only be on disk and likely not in either > Valentina's or the OS's cache. > > Since Valentina doesn't support substring access, I'll have to store every > such string in a separate file outside of the database, store the filenames > in V2, and use the OS to perform the random accesses -- or read in the whole > thing and suffer the performance consequences. Not a difficult or > unmanageable task, but like I said... unfortunate. >> Also always remember about disk, cache, ... >> >> Here not always faster to read 100 bytes then then e.g. 4Kb >> > > I fully expect to have to do a disk access to pull in a cache page or two, > both into the OS cache as well as Valentina's, if I need a 100 byte > substring from a 10 MB source. I don't expect to have to read 10 MB worth > of pages when I only need 100 bytes. 10MB yes, this is many. Just I think can be better solution to avoid such huge data of ATOMIC sets. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 16:04:38 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 08:05:40 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 4:00 PM, "Ed Kleban" wrote: >>> So I can set first the unique and then the indexed flags, and look to catch >>> any error that may be thrown, and then check the value of the unique flag? >>> >> >> Yes. >> >> Just not remember, what exactly flag will be set OFF by Valentina 2 > > Pardon? Are you saying that you don't remember whether in Valentina 2 if > the unique flag will be turned off if the data turns out to not be unique? > > Or did you mean something else? Can be 2 case-sensitive here: flag fUnique is set OFF flag fIndexed is set OFF Not sure how it is implemented. It needs to check sources and tests -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 08:10:08 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 08:11:10 2005 Subject: Indexing or sorting ordered arrays. In-Reply-To: Message-ID: On 11/20/05 8:00 AM, "Ruslan Zasukhin" wrote: > You are welcome. And questions are great. > > Just I have learned few times for last 2-3 years that "theoretical thinking" > sometimes do not happens in life. Some "very cool" optimizations, as I did > think, in fact force system to be slower. When I have disable them system > have work faster. > > Just not go into the same trap :-) > Unfortunately, such traps are a hazard of the vocation. With experience, we learn better how to spot and avoid 'em. It still no assurance we won't fall into one. I fully expect to fall into several. I just hope I'm smart enough to figure out when I'm in one when I do. From Ed at Kleban.com Sun Nov 20 08:29:55 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 08:30:58 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 8:03 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 3:47 PM, "Ed Kleban" wrote: > >>> We did have such feature in 1.x but nobody have use it, >>> So we have not open it for v2 for now. >>> >> >> That is most unfortunate, since it means I will have to do so manually >> outside of the database if I want to avoid unneccessary disk access -- which >> is one of the key things I'm looking to the database to manage for me so >> that I don't have to. Specifically, I need to store a large number of >> strings that will range in size from typically, say... 2 MB to 20 MB. I >> then need to perform completely random accesses to pull out substrings from >> those strings, the position and length of which are stored in database >> tables. > > Hmm. 3 times Hmm. > > You sure this is best possible solution ?! > I doubt. > Great! I welcome any suggestions you may have to offer. Any other options come to mind? If you are asking am I sure that I need to be able to pull out 100-byte substrings out 10 MB strings, then the answer is yes. In terms of access patterns however, it's not clear how much of a performance hit it will be. If I'm only pulling out one or two such strings based on a couple of pieces of text the user wants to view and is waiting on, who cares how long it takes. The user won't see any difference. On the other hand if I'm going to be pull out dozens, hundreds, or thousands of such strings say, for building some sort of cross-reference or analysis, then pretty rapidly the entire file will get cached into the OS and perhaps the better thing to do if I expect this is indeed to read the entire string into memory first and just operate on it there. But: I'd don't really know in advance whether there will be shades of grey in between where the tradeoff would make a great difference. And since I'm designing a tool that will allow the User rather than the Designer to determine the access patterns based on their data viewing needs, I'm not sure I'll ever really know. I'd rather not have to guess or figure such issues out in advance. I'd rather rely on an efficient database to simply adapt, use its caching algorithms, and do the job for me. To my thinking this is one of the fundamental reasons for using a database rather than managing the data myself. > >> Sometimes some of those 2-20 MB string will be in memory already, >> but other times they will only be on disk and likely not in either >> Valentina's or the OS's cache. >> >> Since Valentina doesn't support substring access, I'll have to store every >> such string in a separate file outside of the database, store the filenames >> in V2, and use the OS to perform the random accesses -- or read in the whole >> thing and suffer the performance consequences. Not a difficult or >> unmanageable task, but like I said... unfortunate. > >>> Also always remember about disk, cache, ... >>> >>> Here not always faster to read 100 bytes then then e.g. 4Kb >>> >> >> I fully expect to have to do a disk access to pull in a cache page or two, >> both into the OS cache as well as Valentina's, if I need a 100 byte >> substring from a 10 MB source. I don't expect to have to read 10 MB worth >> of pages when I only need 100 bytes. > > 10MB yes, this is many. It is one of many since I may need 100 or 1000 byte substrings out of a great many separate 10MB sources. > > Just I think can be better solution to avoid such huge data of ATOMIC sets. > There are, and I implement many of them. This is not my sole source of access to the data I need. I have a massive amount of this data pre-scanned, pre-sorted, and pre-linked for rapid access in anticipation of things the user might want to do. And to be more precise, even if I don't have this information, when I do have a need to access it and pull it out of some of the 10MB sources it will get scanned, sorted, linked, crossreferenced, and cached by my software on the fly so that I don't have to go get it again if I'm going to use again in a similar way real soon. But all that said, this "get a small substring" mechanism is the access method of last resort, and will occur -- a lot. All my code can do after that is work to prevent it from having to happen again too soon. ======= I have a car of a certain model year. When you go look up the list of most common customer complaints, do you know what heads the list? That it knows the temperature inside and outside the car but won't tell you. It has to know to be able to administer the thermostat which you set by selecting the desired temperature on a digital display. The manufacturer heard that complaint loud and clear, because in later model years they fixed the car to display the temperature. This is a similar kind of thing. Your Kernel manual has very elegant diagrams in it showing the segmentation scheme that you use to take a very large file and segment it to make it manageable on disk. But then I discover that despite this fine organization, I can't take advantage of it but rather have to find some other means of either duplicating it myself or relying on some other mechanism, like the OS file system that can do it for me. It's not a show-stopper. It's not even particularly hard or painful to code. It's just frustrating knowing that I have to when the provision of simple single function call -- one in fact that was already written and used to be there -- was removed due lack of popularity. From Ed at Kleban.com Sun Nov 20 08:32:07 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 08:33:09 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 8:04 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 4:00 PM, "Ed Kleban" wrote: > >>>> So I can set first the unique and then the indexed flags, and look to catch >>>> any error that may be thrown, and then check the value of the unique flag? >>>> >>> >>> Yes. >>> >>> Just not remember, what exactly flag will be set OFF by Valentina 2 >> >> Pardon? Are you saying that you don't remember whether in Valentina 2 if >> the unique flag will be turned off if the data turns out to not be unique? >> >> Or did you mean something else? > > Can be 2 case-sensitive here: > > flag fUnique is set OFF > flag fIndexed is set OFF > > Not sure how it is implemented. > > It needs to check sources and tests > Case sensitive? Sorry, I don't understand. But I believe you're saying you're not sure and would have to check the source to really know what's going on or how it will behave. If you find out some day, I'd be delighted to know. Thanks! --Ed From sunshine at public.kherson.ua Sun Nov 20 16:40:12 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 08:41:16 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 4:29 PM, "Ed Kleban" wrote: > > If you are asking am I sure that I need to be able to pull out 100-byte > substrings out 10 MB strings, then the answer is yes. In terms of access > patterns however, it's not clear how much of a performance hit it will be. Question is: why you cannot split that 10Mb into many small records ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 09:17:05 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 09:18:07 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 8:40 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 4:29 PM, "Ed Kleban" wrote: > >> >> If you are asking am I sure that I need to be able to pull out 100-byte >> substrings out 10 MB strings, then the answer is yes. In terms of access >> patterns however, it's not clear how much of a performance hit it will be. > > Question is: > > why you cannot split that 10Mb into many small records ? I can provide more detail off-list. But the main reason is essentially because I have numerous positional references into these large strings and that this particular view or structuring is the natural paradigm I have to work with given the data I have. My choices are therefore: 1) Restructure my data, such as chopping it up into many small records, so that it is more manageable in the structuring tools I have -- such as the database and/or OS filesystem -- and then write an abstraction layer of code that will allow me to work in terms of the natural positional span of substrings, while translating this to the lower-level structural representation. In other words write a whole new segmentation scheme that can deal with spans that cross segments, and other messes. I don't want to mess with the additional layer of complexity, and I don't relish the idea of taking a performance hit for building and accessing through yet another layer of segmentation -- especially one that I have to manage. 2) Use a tool that already implements segmentation logic so I don't have to, such as a database or filesystem -- which, again, is what I see as a main purpose of these tools. The other reason that I need to preserve the ability to operate in this natural positional framework, is that it allows me to do direct comparisons of database content with that of external files that have not yet been incorporated into my database already. There are cases when these large 10 MB files can simply be discarded and are no longer needed because all of the information of immediate value has been summarized into other structures -- including span hashes so I can do direct comparisons with external files without having to have the old data around any more. But there are also cases and intermediate periods where the proper thing to do -- or in initial stages of development the far easier thing to do -- is to merely keep a fully assembled copy of these large text spans rather than taking the performance hit of regenerating them. I don't know how clear all that is. But maybe you can get a hint or two out of it. Again, much of my "concern" comes not from the fact that I have to do more effort to work around this, but the amusing frustration that the database structure has ALREADY done all this work for me... but the access method to take advantage of it was removed from V2 "Because nobody used it". Guess I should have been an user earlier, eh? ;-) --Ed From Ed at Kleban.com Sun Nov 20 09:27:38 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 09:28:39 2005 Subject: Setting Unique In-Reply-To: Message-ID: On 11/20/05 8:32 AM, "Ed Kleban" wrote: > > On 11/20/05 8:04 AM, "Ruslan Zasukhin" wrote: > >> On 11/20/05 4:00 PM, "Ed Kleban" wrote: >> >>>>> So I can set first the unique and then the indexed flags, and look to >>>>> catch >>>>> any error that may be thrown, and then check the value of the unique flag? >>>>> >>>> >>>> Yes. >>>> >>>> Just not remember, what exactly flag will be set OFF by Valentina 2 >>> >>> Pardon? Are you saying that you don't remember whether in Valentina 2 if >>> the unique flag will be turned off if the data turns out to not be unique? >>> >>> Or did you mean something else? >> >> Can be 2 case-sensitive here: >> >> flag fUnique is set OFF >> flag fIndexed is set OFF >> >> Not sure how it is implemented. >> >> It needs to check sources and tests >> > > Case sensitive? Sorry, I don't understand. > But I believe you're saying you're not sure and would have to check the > source to really know what's going on or how it will behave. > > If you find out some day, I'd be delighted to know. > > Thanks! > --Ed Ah, I think I get it. You're saying you're not sure whether when the database detects that the field contents is not unique whether it clears the fUnique flag or cleas the fIndexed flag, or both. So if I care I should check them both. If that's what you mean, I understand. No rush on looking it up. I can code as above if I care, and I likely wont'. I'll know in advance with 100% accuracy whether the field contents I'm providing are indeed unique or not. I was just curious how Valentina handled this. Only thing I really need to know, you've already told me. Namely that if I disable fIndexed and fUnique before filling the table and then turn them back on afterwards that: 1) It's faster than filling the table with both these turned on, and 2) It will immediately reindex the data again once these bits are set. That's all I really need for coding on my end. Thanks! --Ed From sunshine at public.kherson.ua Sun Nov 20 17:50:33 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 09:51:39 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 5:17 PM, "Ed Kleban" wrote: > Again, much of my "concern" comes not from the fact that I have to do more > effort to work around this, but the amusing frustration that the database > structure has ALREADY done all this work for me... but the access method to > take advantage of it was removed from V2 "Because nobody used it". Guess I > should have been an user earlier, eh? ;-) Add this as feature request. Actually this is just one more parameter in methods ReadData WriteData -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 17:52:07 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 09:53:09 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 5:17 PM, "Ed Kleban" wrote: > I don't know how clear all that is. But maybe you can get a hint or two out > of it. It looks that this big BLOB has structure. You say that you will keep in some parallel tables offsets that keep info about structure. So I do not understand why you can't just split that BLOB... And then no need for this parallel table(s). -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 10:23:03 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 10:24:08 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 9:52 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 5:17 PM, "Ed Kleban" wrote: > >> I don't know how clear all that is. But maybe you can get a hint or two out >> of it. > > It looks that this big BLOB has structure. > You say that you will keep in some parallel tables offsets that keep info > about structure. > > So I do not understand why you can't just split that BLOB... > And then no need for this parallel table(s). > The short answer is simple: Because I don't want to. In my opinion this is the wrong way to solve this problem. Here's a longer answer: Because I don't want to have to maintain two separate representations for positional spans along with the associated processing overhead. I need to retain the natural positional spans for comparison with substring spans that are in files that have not yet been processed into the database. By doing so, the same span can be used to either access a span within some OS file, as well as a positional span within a big Blob. This is mostly of value during the process of either parsing a file to incorporate it's contents into the database, or comparing the contents of information already in the database with some external file. Essentially, by simply storing a copy of a file as a blob I can use the same position and length parameters for a span whether the span is internal or external to the database. This also allows me to bootstrap my implementation by initially just copying a file into the database as a quick and dirty way of representing it, and eventually replacing this simplistic access method with something more clever or efficient -- a technique I gather you recommend. After a period of transition, these big blobs which are essentially copies of existing files can be thrown away entirely, because all of the information is stored in very small database pieces. In fact it is stored at a tiny fraction of it's original size because all common substrings are stored as references rather than duplicate text, and all the substrings that will never be looked at again by the user unless they want to do a full reconstruction for export are optionally compressed thanks to the ability provided by Valentina to ZIP with individual Blob fields -- i.e. not all of the condensed data is stored with in the same blob fields; it's more clever than that. However, during this time of transition: 1) Retrieving the substring for the blob string position spans from the very compact versions would require an immense amount of processing, which is a concern when you have 64063 spans to process for a mere 2.8 MB file as an\ very small example I am testing with. 2) I have many options available. I don't have to use the Valentina database for this purpose. I already have a full parse tree at hand and code written that will extract these substrings using a very efficient hash pool that's faster than Valentina will ever be. However, I'm striving to see how far I can push Valentina to use it for this and similar applications. Ideally I'd like to move everything into a single data manager such as Valentina and punt all the old pre-database code. From Ed at Kleban.com Sun Nov 20 10:38:05 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 10:39:06 2005 Subject: Bloblets In-Reply-To: Message-ID: Done. The Mantis form brings up a selection on the Product Version menu for version 2.1 I'm running 2.0.5 Is there a newer version than I have? On 11/20/05 9:50 AM, "Ruslan Zasukhin" wrote: > On 11/20/05 5:17 PM, "Ed Kleban" wrote: > >> Again, much of my "concern" comes not from the fact that I have to do more >> effort to work around this, but the amusing frustration that the database >> structure has ALREADY done all this work for me... but the access method to >> take advantage of it was removed from V2 "Because nobody used it". Guess I >> should have been an user earlier, eh? ;-) > > Add this as feature request. > > Actually this is just one more parameter in methods > > ReadData > WriteData From Ed at Kleban.com Sun Nov 20 10:43:54 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 10:44:57 2005 Subject: IndexbyWords // Vstring.SplitToWords( text as string) In-Reply-To: Message-ID: > >> Unfortunately it turns out that "words" as defined by SplitToString include >> "." and exclude "_". > > I believe they follow natural language rules. > >> The later is unfortunate for my application. The >> former seems to be unfortunate for most every application I can think of >> that might want to otherwise use IndexedByWords. > > right > >> It's not good for prose, >> verse, or code since "foo" and "foo." and ".foo" would all parse out to be >> different search terms. That is unless a search for "foo" would return >> entries that included all three of these (and other) combinations. I'll >> have to go back and read the fine print on query for IndexedByWords fields. > > Add feature request :-) Done. From jda at his.com Sun Nov 20 12:08:25 2005 From: jda at his.com (jda) Date: Sun Nov 20 11:09:28 2005 Subject: IndexbyWords // Vstring.SplitToWords( text as string) In-Reply-To: References: Message-ID: > > >>> Unfortunately it turns out that "words" as defined by SplitToString include >>> "." and exclude "_". >> >> I believe they follow natural language rules. >> >>> The later is unfortunate for my application. The >>> former seems to be unfortunate for most every application I can think of > >> that might want to otherwise use IndexedByWords. > > I really hate to contribute to this endless series of messages, but Ruslan, be careful what you do. The ICU library is smart and ignores periods if they end words, but not if they are in the middle of "words". The last time I tested this, "foo." becomes "foo", but "foo.bar" stays as "foo.bar". I assume this is mostly done so that decimal numbers aren't split into two (21.50 isn't split into 21 and 50). Jon From sunshine at public.kherson.ua Sun Nov 20 21:09:03 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 13:10:03 2005 Subject: IndexbyWords // Vstring.SplitToWords( text as string) In-Reply-To: Message-ID: On 11/20/05 7:08 PM, "jda" wrote: >>>> Unfortunately it turns out that "words" as defined by SplitToString >>>> include >>>> "." and exclude "_". >>> >>> I believe they follow natural language rules. >>> >>>> The later is unfortunate for my application. The >>>> former seems to be unfortunate for most every application I can think of >>>> that might want to otherwise use IndexedByWords. >>> > > I really hate to contribute to this endless series of messages, but > Ruslan, be careful what you do. The ICU library is smart and ignores > periods if they end words, but not if they are in the middle of > "words". The last time I tested this, "foo." becomes "foo", but > "foo.bar" stays as "foo.bar". I assume this is mostly done so that > decimal numbers aren't split into two (21.50 isn't split into 21 and > 50). Do not worry Jon. I am not going change behavior of ICU. What we have point is ability for DEVELOPER to change/tune something -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 20 21:11:14 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 20 13:12:16 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 6:38 PM, "Ed Kleban" wrote: > Done. The Mantis form brings up a selection on the Product Version menu for > version 2.1 I'm running 2.0.5 Is there a newer version than I have? Exists 2.1 betas You can subscribe to beta list to discuss them there -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 20 14:11:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 20 14:12:58 2005 Subject: Bloblets In-Reply-To: Message-ID: On 11/20/05 1:11 PM, "Ruslan Zasukhin" wrote: > On 11/20/05 6:38 PM, "Ed Kleban" wrote: > >> Done. The Mantis form brings up a selection on the Product Version menu for >> version 2.1 I'm running 2.0.5 Is there a newer version than I have? > > Exists 2.1 betas > > You can subscribe to beta list to discuss them there LOL. You're poor users are already complaining that I'm engaged in endless discussions on this list. Others may already regret you asked me to move our correspondence here, much less join in on another. --Ed From Ed at Kleban.com Mon Nov 21 08:36:05 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 08:37:09 2005 Subject: kPreferIndexed Message-ID: My understanding from reading the V4RB reference manual, is that if you disable Indexing for a field, and then later perform a search such as FindSingle on that field with kPreferIndexed, then at that time V4RB will first create an index for that field as if you had enabled isIndexed=true, and that the index will then be retained and updated as additional records are added until you subsequently changed to isIndexed=false. So then, if I remove an index then it will be recreated automatically the next time I do a search that needs it. Is all of this correct? Thanks! --Ed From Ed at Kleban.com Mon Nov 21 11:17:52 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 11:18:59 2005 Subject: Table of a VFeld from a VCursor Message-ID: When you get a VField from a VCursor, such as with: aVField = aVCursor.Field( "SomeFieldName" ) What is the value of aVField.Table ? Is it nil? Is it aVCursor? Is it the VTable from which the vField was extracted by SQL and added to the Cursor? I'm hoping that given you have a VField obtained from a VCursor in this manner that there is some way to determine which VCursor the VField comes from. Is there some way to find out? Thanks! --Ed From yeomans at desuetude.com Mon Nov 21 12:51:23 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Mon Nov 21 11:52:26 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: References: Message-ID: <45029a1c2d04a3efbcf2dcbf59c62442@desuetude.com> On Nov 21, 2005, at 12:17 PM, Ed Kleban wrote: > > When you get a VField from a VCursor, such as with: > > aVField = aVCursor.Field( "SomeFieldName" ) > > What is the value of aVField.Table ? > > Is it nil? > Is it aVCursor? > Is it the VTable from which the vField was extracted by SQL and > added to > the Cursor? > > I'm hoping that given you have a VField obtained from a VCursor in this > manner that there is some way to determine which VCursor the VField > comes > from. Is there some way to find out? I'm not sure that VField.Table makes sense here. A cursor can be thought of as a new, temporary table, and its fields are fields of this temporary table. -------------- Charles Yeomans From sunshine at public.kherson.ua Mon Nov 21 20:02:48 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 12:03:51 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: <45029a1c2d04a3efbcf2dcbf59c62442@desuetude.com> Message-ID: On 11/21/05 7:51 PM, "Charles Yeomans" wrote: >> When you get a VField from a VCursor, such as with: >> >> aVField = aVCursor.Field( "SomeFieldName" ) >> >> What is the value of aVField.Table ? >> >> Is it nil? >> Is it aVCursor? >> Is it the VTable from which the vField was extracted by SQL and >> added to >> the Cursor? >> >> I'm hoping that given you have a VField obtained from a VCursor in this >> manner that there is some way to determine which VCursor the VField >> comes >> from. Is there some way to find out? > > I'm not sure that VField.Table makes sense here. A cursor can be > thought of as a new, temporary table, and its fields are fields of this > temporary table. Absolutely right. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 21 20:06:02 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 12:07:04 2005 Subject: kPreferIndexed In-Reply-To: Message-ID: On 11/21/05 4:36 PM, "Ed Kleban" wrote: > My understanding from reading the V4RB reference manual, is that if you > disable Indexing for a field, and then later perform a search such as > FindSingle on that field with kPreferIndexed, then at that time V4RB will > first create an index for that field as if you had enabled isIndexed=true, > and that the index will then be retained and updated as additional records > are added until you subsequently changed to isIndexed=false. > > So then, if I remove an index then it will be recreated automatically the > next time I do a search that needs it. > > Is all of this correct? No. At least for now logic of engine is: if( mpFldStorage->get_IndexFile() && inSearchPref == kPreferIndexed ) { // we do indexed search } else // we scan the whole table. { } So you can use this option to AVOID search by index. This can be useful, if you already have small set of records to iterate. You could get this set from searches by prev fields. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Mon Nov 21 12:06:13 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 12:07:20 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: Makes sense. However my question still remains: >>> I'm hoping that given you have a VField obtained from a VCursor in this >>> manner that there is some way to determine which VCursor the VField >>> comes >>> from. Is there some way to find out? Is there a way to find out from a VField what it's VCursor is, or to as a VCursor what it's VFields are or whether a given VField belongs to it? Any unorthodox, implementation-dependent, dangerous "cheats" that might accomplish this? On 11/21/05 12:02 PM, "Ruslan Zasukhin" wrote: > On 11/21/05 7:51 PM, "Charles Yeomans" wrote: > >>> When you get a VField from a VCursor, such as with: >>> >>> aVField = aVCursor.Field( "SomeFieldName" ) >>> >>> What is the value of aVField.Table ? >>> >>> Is it nil? >>> Is it aVCursor? >>> Is it the VTable from which the vField was extracted by SQL and >>> added to >>> the Cursor? >>> >>> I'm hoping that given you have a VField obtained from a VCursor in this >>> manner that there is some way to determine which VCursor the VField >>> comes >>> from. Is there some way to find out? >> >> I'm not sure that VField.Table makes sense here. A cursor can be >> thought of as a new, temporary table, and its fields are fields of this >> temporary table. > > Absolutely right. From Ed at Kleban.com Mon Nov 21 12:16:39 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 12:17:42 2005 Subject: kPreferIndexed In-Reply-To: Message-ID: On 11/21/05 12:06 PM, "Ruslan Zasukhin" wrote: > On 11/21/05 4:36 PM, "Ed Kleban" wrote: > >> My understanding from reading the V4RB reference manual, is that if you >> disable Indexing for a field, and then later perform a search such as >> FindSingle on that field with kPreferIndexed, then at that time V4RB will >> first create an index for that field as if you had enabled isIndexed=true, >> and that the index will then be retained and updated as additional records >> are added until you subsequently changed to isIndexed=false. >> >> So then, if I remove an index then it will be recreated automatically the >> next time I do a search that needs it. >> >> Is all of this correct? > > No. > > At least for now logic of engine is: > > > if( mpFldStorage->get_IndexFile() && inSearchPref == kPreferIndexed ) > { > // we do indexed search > } > else // we scan the whole table. > { > } > > > So you can use this option to AVOID search by index. > This can be useful, if you already have small set of records to iterate. > You could get this set from searches by prev fields. Ah, I see. So you can avoid the search by index, but the table won't get indexed until or unless you explicitly set fIndexed = true. Ok, got it. From yeomans at desuetude.com Mon Nov 21 13:35:27 2005 From: yeomans at desuetude.com (Charles Yeomans) Date: Mon Nov 21 12:36:29 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: References: Message-ID: On Nov 21, 2005, at 1:06 PM, Ed Kleban wrote: > Makes sense. However my question still remains: > >>>> I'm hoping that given you have a VField obtained from a VCursor in >>>> this >>>> manner that there is some way to determine which VCursor the VField >>>> comes >>>> from. Is there some way to find out? > > Is there a way to find out from a VField what it's VCursor is, or to > as a > VCursor what it's VFields are or whether a given VField belongs to it? > Any unorthodox, implementation-dependent, dangerous "cheats" that might > accomplish this? > I did a little experiment, and it appears that VField.Table returns a reference to the VCursor. But VCursor is not a subclass of VTable; that is, theCursor IsA VTable returns false. Ruslan, you've got some explaining to do. -------------- Charles Yeomans From sunshine at public.kherson.ua Mon Nov 21 20:55:59 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 12:57:01 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/21/05 8:35 PM, "Charles Yeomans" wrote: >> Makes sense. However my question still remains: >> >>>>> I'm hoping that given you have a VField obtained from a VCursor in >>>>> this >>>>> manner that there is some way to determine which VCursor the VField >>>>> comes >>>>> from. Is there some way to find out? >> >> Is there a way to find out from a VField what it's VCursor is, or to >> as a >> VCursor what it's VFields are or whether a given VField belongs to it? >> Any unorthodox, implementation-dependent, dangerous "cheats" that might >> accomplish this? >> > > I did a little experiment, and it appears that VField.Table returns a > reference to the VCursor. But VCursor is not a subclass of VTable; > that is, theCursor IsA VTable returns false. Ruslan, you've got some > explaining to do. In C++ I_Cursor is subclass of I_Table. But in REALbasic we have not implement this on tech reasons. interfaces have many bugs in RB -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 21 20:56:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 12:57:56 2005 Subject: kPreferIndexed In-Reply-To: Message-ID: On 11/21/05 8:16 PM, "Ed Kleban" wrote: >>> My understanding from reading the V4RB reference manual, is that if you >>> disable Indexing for a field, and then later perform a search such as >>> FindSingle on that field with kPreferIndexed, then at that time V4RB will >>> first create an index for that field as if you had enabled isIndexed=true, >>> and that the index will then be retained and updated as additional records >>> are added until you subsequently changed to isIndexed=false. >>> >>> So then, if I remove an index then it will be recreated automatically the >>> next time I do a search that needs it. >>> >>> Is all of this correct? >> >> No. >> >> At least for now logic of engine is: >> >> >> if( mpFldStorage->get_IndexFile() && inSearchPref == kPreferIndexed ) >> { >> // we do indexed search >> } >> else // we scan the whole table. >> { >> } >> >> >> So you can use this option to AVOID search by index. >> This can be useful, if you already have small set of records to iterate. >> You could get this set from searches by prev fields. > > Ah, I see. So you can avoid the search by index, but the table won't get > indexed until or unless you explicitly set fIndexed = true. Ok, got it. Right, Your idea that it can self create index is also interesting ... We will keep it in mind -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Mon Nov 21 13:37:57 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 13:39:02 2005 Subject: kPreferIndexed In-Reply-To: Message-ID: >> Ah, I see. So you can avoid the search by index, but the table won't get >> indexed until or unless you explicitly set fIndexed = true. Ok, got it. > > Right, > > Your idea that it can self create index is also interesting ... > We will keep it in mind Yes, it would incredibly nice to have, perhaps as an alternate flag to kPreferIndexed, say kIndexOnFind That makes it very easy to simply disable indexing on some fields, add a ton of records, and not have to worry about which indexes you need to turn on again. I'll add a feature request... eventually. From Ed at Kleban.com Mon Nov 21 13:47:23 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 13:48:27 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: >> I did a little experiment, and it appears that VField.Table returns a >> reference to the VCursor. But VCursor is not a subclass of VTable; >> that is, theCursor IsA VTable returns false. Ruslan, you've got some >> explaining to do. > > In C++ I_Cursor is subclass of I_Table. > > But in REALbasic we have not implement this on tech reasons. > interfaces have many bugs in RB > This is great news. I'd like to request that if and when you choose to clean this up that you NOT remove the functionality that permits access to the VCursor. Some possible ways to this are: 1) implement VField.Table and VField.Cursor as methods rather than properties... But the method invocations would incur more overhead than a simple property access, so I'm not thrilled with this one. 2) Change the name of the property from "Table" to something else like Source -- a really bad idea because it is not backward compatible. 3) Leave functionality the way it is but update the document to explain the way things really work; namely that in the case of a VField obtained by means of one of a VCursor's field methods, that the Table property returns the associated VCursor, not a VTable. -- May not be the best naming convention, may not be the best implementation, but it's darn valuable to be able to access this. I should probably write up a request for this as well under the "Fix Documentation" category. From chuck at mediamacros.com Mon Nov 21 14:53:46 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 21 13:50:52 2005 Subject: LoadDump making too many files In-Reply-To: References: Message-ID: <438225CA.4060603@mediamacros.com> Things are getting more complicated... First, any dump comes in as 4 files, no matter what the dump file says. I also have an existing project database that I need to use (so names can't change). It uses some table names that Valentina is fine with but does not like in a load dump. (It created the dump fine but will not read it back in.) It loaded the dump but skipped a number of tables. So far the ones I see that are a problem in this file... Class Language_Phrase Option Parameter Technique Technique_Group Technique_Option Nothing looks incorrect in the syntax and these are scattered throughout the other items that did export. Is LoadDump just not working (especially in the tool?) -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/19/05 12:16 AM, "Chuck Neal" wrote: > > Hi Chuck, > > So you make SQL dump, then do SQL load. And getwrong number of files? > > Ivan, sounds like a bug. > > > >>I am using Valentina 2 for Director. When I do a load dump it is >>creating 4 files instead of everything in one. The DB the dump was made >>from was a single file and the first part seems to correctly set the >>type to kDscDatBlbInd >> >>This is the start of my SQL dump... >> >>SET PROPERTY >> DatabaseMode, >> DatabaseSegment, >> SchemaVersion, >> DateFormat, >> DateSeparator, >> TimeSeparator, >> LocaleName, >> StorageEncoding, >> CollationFrenchCollation, >> CollationAlternateHandling, >> CollationCaseFirst, >> CollationCaseLevel, >> CollationNormalizationMode, >> CollationStrength, >> CollationHiraganaQuaternaryMode, >> CollationNumericCollation >>of DATABASE TO >> 'kDscDatBlbInd', >> 32768, >> 1, >> 'kMDY', >> '/', >> ':', >> 'en_US', >> 'UTF-16', >> 'kOff', >> 'kNonIgnorable', >> 'kOff', >> 'kOff', >> 'kOff', >> 'kTertiary', >> 'kOff', >> 'kOff'; > > From sunshine at public.kherson.ua Mon Nov 21 22:25:17 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 14:26:33 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/21/05 9:47 PM, "Ed Kleban" wrote: >>> I did a little experiment, and it appears that VField.Table returns a >>> reference to the VCursor. But VCursor is not a subclass of VTable; >>> that is, theCursor IsA VTable returns false. Ruslan, you've got some >>> explaining to do. >> >> In C++ I_Cursor is subclass of I_Table. >> >> But in REALbasic we have not implement this on tech reasons. >> interfaces have many bugs in RB >> > > This is great news. I'd like to request that if and when you choose to > clean this up that you NOT remove the functionality that permits access to > the VCursor. > > Some possible ways to this are: > > 1) implement VField.Table and VField.Cursor as methods rather than > properties... But the method invocations would incur more overhead than a > simple property access, so I'm not thrilled with this one. > > 2) Change the name of the property from "Table" to something else like > Source -- a really bad idea because it is not backward compatible. > > 3) Leave functionality the way it is but update the document to explain the > way things really work; namely that in the case of a VField obtained by > means of one of a VCursor's field methods, that the Table property returns > the associated VCursor, not a VTable. -- May not be the best naming > convention, may not be the best implementation, but it's darn valuable to be > able to access this. > > I should probably write up a request for this as well under the "Fix > Documentation" category. What is so great in this info ??? I do not see any useful usage of it -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 21 22:27:16 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 14:28:20 2005 Subject: LoadDump making too many files In-Reply-To: <438225CA.4060603@mediamacros.com> Message-ID: On 11/21/05 9:53 PM, "Chuck Neal" wrote: > Things are getting more complicated... > > First, any dump comes in as 4 files, no matter what the dump file says. > > I also have an existing project database that I need to use (so names > can't change). It uses some table names that Valentina is fine with but > does not like in a load dump. (It created the dump fine but will not > read it back in.) It loaded the dump but skipped a number of tables. > > So far the ones I see that are a problem in this file... > Class > Language_Phrase > Option > Parameter > Technique > Technique_Group > Technique_Option Looks to be legal names > Nothing looks incorrect in the syntax and these are scattered throughout > the other items that did export. > > Is LoadDump just not working (especially in the tool?) Tool? What you mean? Vstudio ? Chuck, you should us send something to reproduce, Also please add report into Mantis. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Mon Nov 21 14:34:21 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 14:35:27 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/21/05 2:25 PM, "Ruslan Zasukhin" wrote: >> 3) Leave functionality the way it is but update the document to explain the >> way things really work; namely that in the case of a VField obtained by >> means of one of a VCursor's field methods, that the Table property returns >> the associated VCursor, not a VTable. -- May not be the best naming >> convention, may not be the best implementation, but it's darn valuable to be >> able to access this. >> >> I should probably write up a request for this as well under the "Fix >> Documentation" category. > > What is so great in this info ??? > > I do not see any useful usage of it Yeah, it's pretty esoteric. I'll explain in more detail later, but basicaly it's valueable if your are writing a FieldCursor class as described in earlier emails. As it turns out I've decided to write a hierarchy of subclasses of FieldCursor that parallels the subclases of Vfield, for a variety of reasons including execution performance. But if I had wanted to write it all into a single class and not make a lot of subclasses, having this would be quite useful. I'll explain later in detail if you'd like. In fact, I'll probably publish, or send you to publish, the FieldCursor class hierarchy to use as a V4RB add-on or to include as an example. It essentially provides the lazy read feature in code until such time as you get around to adding it V4RB. If I didn't have a way to do this I would be willing to move nearly as much code over to V4RB and feel ok about burning bridges behind myself. Or I'd have to restructure my use of the database in a far more painful way to manage. Later! --Ed From sunshine at public.kherson.ua Mon Nov 21 23:30:46 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 15:31:48 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/21/05 10:34 PM, "Ed Kleban" wrote: > In fact, I'll probably publish, or send you to publish, the FieldCursor > class hierarchy to use as a V4RB add-on or to include as an example. It > essentially provides the lazy read feature in code until such time as you > get around to adding it V4RB. If I didn't have a way to do this I would be > willing to move nearly as much code over to V4RB and feel ok about burning > bridges behind myself. Or I'd have to restructure my use of the database in > a far more painful way to manage. Can you show small example of usage of V4RB now, and with your classes To see difference ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Mon Nov 21 17:06:21 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 21 16:05:39 2005 Subject: SQL Error Message-ID: <438244DD.3090406@mediamacros.com> I am probably doing something obvious here but does anyone see anything wrong with this. INSERT INTO class (name,text,ID) VALUES (:1 ,:2 ,:3 ) This is the data - ["Test Class", "This is a test of this class", "I2G8L5-11O3VGE-FRP6HU-1"] This comes up with this error... ERROR: 462848 I could not find that error in the list of error codes. I also tried inserting it directly in the SQL and got the same results. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer From IvanSmahin at public.kherson.ua Tue Nov 22 00:15:10 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Mon Nov 21 16:16:12 2005 Subject: SQL Error In-Reply-To: <438244DD.3090406@mediamacros.com> References: <438244DD.3090406@mediamacros.com> Message-ID: <982643583.20051122001510@public.kherson.ua> Hello Chuck, Tuesday, November 22, 2005, 12:06:21 AM, you wrote: CN> I am probably doing something obvious here but does anyone see anything CN> wrong with this. CN> INSERT INTO class (name,text,ID) VALUES (:1 ,:2 ,:3 ) CN> This is the data - ["Test Class", "This is a test of this class", CN> "I2G8L5-11O3VGE-FRP6HU-1"] CN> This comes up with this error... CN> ERROR: 462848 CN> I could not find that error in the list of error codes. I also tried CN> inserting it directly in the SQL and got the same results. You should convert it to the hex. It would be 71000 - ERR_SQL_PARSER_ERROR. 'text' seems to be a keyword. The best solution is to avoid using such an identifiers. -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From chuck at mediamacros.com Mon Nov 21 17:22:28 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 21 16:19:45 2005 Subject: LoadDump making too many files In-Reply-To: References: Message-ID: <438248A4.8010707@mediamacros.com> I think I found one of the issues. Its an error on export not import. The loaddump does make 4 files, no matter what. The other table error was due to a field in the table being named with an "illegal" name. All of those tables had fields named "Text". In most any SQL you can get around this by putting the name in quotes, but the dump should, in my opinion, include all names in quotes by default to avoid this. That would prevent this from happening. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/21/05 9:53 PM, "Chuck Neal" wrote: > > >>Things are getting more complicated... >> >>First, any dump comes in as 4 files, no matter what the dump file says. >> >>I also have an existing project database that I need to use (so names >>can't change). It uses some table names that Valentina is fine with but >>does not like in a load dump. (It created the dump fine but will not >>read it back in.) It loaded the dump but skipped a number of tables. >> >>So far the ones I see that are a problem in this file... >>Class >>Language_Phrase >>Option >>Parameter >>Technique >>Technique_Group >>Technique_Option > > > Looks to be legal names > > >>Nothing looks incorrect in the syntax and these are scattered throughout >>the other items that did export. >> >>Is LoadDump just not working (especially in the tool?) > > > Tool? What you mean? Vstudio ? > > Chuck, you should us send something to reproduce, > Also please add report into Mantis. > From Ed at Kleban.com Mon Nov 21 16:21:49 2005 From: Ed at Kleban.com (Ed Kleban) Date: Mon Nov 21 16:22:55 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/21/05 3:30 PM, "Ruslan Zasukhin" wrote: > On 11/21/05 10:34 PM, "Ed Kleban" wrote: > >> In fact, I'll probably publish, or send you to publish, the FieldCursor >> class hierarchy to use as a V4RB add-on or to include as an example. It >> essentially provides the lazy read feature in code until such time as you >> get around to adding it V4RB. If I didn't have a way to do this I would be >> willing to move nearly as much code over to V4RB and feel ok about burning >> bridges behind myself. Or I'd have to restructure my use of the database in >> a far more painful way to manage. > > Can you show small example of usage of V4RB now, and with your classes > To see difference ? > > Sure. I'm still coding this, nothing is tested, all subject to change. That said, here is some code to stare at. (Actually this was quite a useful exercise; I found 3 bugs in the process of pasting text here). // When you define your VTable subclasses you define the FieldCursors // you want to be able to use in addition to table fields: Sub CTblForms(db as CRbProjectDB, tableName as String) // Initialize the receiver and its fields. self.name = tableName // Define the Table Fields fFrmFlags = new VByte( "FFrmFlags" ) fFrmLabel = new VVarChar( "FFrmLabel", 504 ) fFrmTagTyp = new VVarChar( "FFrmTagTyp", 504) fFrmXmlTag = new VVarChar( "FFrmXmlTag", 504) // Define the FieldCursors cursorOnFFrmXmlTag = new CFCVarChar( fFrmXmlTag ) cursorOnFFrmFlags = new CFCByte( fFrmFlags ) End Sub // Subsequently, in your code you can use the FieldCursors // to randomly access a single field of a given record without // reading in every single table field when all you need is // a single field's value: flagByteFromRecord32 = MyTblForms.cursorOnFFrmFlags.integerValue( 32 ) MyTblForms.cursorOnFFrmFlags.integerValue( 21 ) = kFlagValForRec21 // To make this possible... // You need to define a subclass of CFCField for each different // kind of Vfield subclass that you want to be able create a // FieldCursor for. This gets done once and reused in all the // applications where you want to use FieldCursors. It is essentially // the common FieldCursor add-in code. Here are some example guts // out of CFCByte: Function integerValue(recId as Integer, doUpdateCheck = true) As Integer // Return an Integer with the value of the reciever's fcCursorField at // recId from the updated fcCursor on fcTable in fcDatabase. // If doUpdateCheck is false then assume that no rows have been // added to fcTable and that the fcCursor is valid. if doUpdateCheck and updateFieldCursor then // fcCursorChanged fcFCByte = fcCursor.ByteField( 1 ) end fcCursor.position = recId return fcFCByte.Value End Function Function integerValue(recId as Integer, newIntVal as Integer, _ doUpdateCheck = true, assigns newValStr as string) As String // Change the value of the reciever's fcCursorField at recId // from the updated fcCursor on fcTable in fcDatabase to newIntVal. // If doUpdateCheck is false then assume that no rows have been // added to fcTable and that the fcCursor is valid. if doUpdateCheck and updateFieldCursor then // fcCursorChanged fcFCByte = fcCursor.ByteField( 1 ) end fcCursor.position = recId fcFCByte.Value = newIntVal fcCursor.UpdateRecord End Function // The central magic that makes all this work transparently so you // don't have worry about the state of the FieldCursor is the // updateFieldCursor method, and the common code called from the // constructors for all CFCField subclasses: Protected Function updateFieldCursor() As Boolean // Update the receiver's cursor from it's field if necessary. // Return true if the Cursor was updated or false if the // former fcCursor is fine to use. const here = "updateFieldCursor" if fcCursor is nil or fcCursor.RecordCount <> fcTable.RecordCount then fcCursor = fcDatabase.SqlSelect( _ fcSqlString, EVCursorLocation.kClientSide, _ EVLockType.kNoLocks, EVCursorDirection.kRandom ) #if DebugBuild then if fcCursor.RecordCount <> fcTable.RecordCount then fail( here, "Update error" ) end if #endif return true // Cursor was modified else return false // Cursor is fine as-is. end End Function Sub CFCField(aVField as VField) // Initialize the receiver to be a cursor on aVField const here = "CFCField" #if DebugBuild then if aVField is nil or aVField.Table is nil _ or aVField.Table.Database is nil then fail( here, "Implementation Error" ) end #endif if not self.checkFieldType( aVField ) then fail( here, "Invalid VField subclass" ) end // Cache useful values in properties fcTable = aVField.Table fcSqlString = glue( "Select ", aVField.Name, " from ", fcTable.Name ) fcDatabase = aVField.Table.Database // There is no need or reason to cache the argument aVField. End Sub From chuck at mediamacros.com Mon Nov 21 17:27:37 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 21 16:28:21 2005 Subject: SQL Error In-Reply-To: <982643583.20051122001510@public.kherson.ua> References: <438244DD.3090406@mediamacros.com> <982643583.20051122001510@public.kherson.ua> Message-ID: <438249D9.5050807@mediamacros.com> Unfortunately I do not have the luxury of changing it since its an existing project. Any work arounds other than using the API syntax? I really prefer to stick with SQL for now since this will be paralleling another app that uses the same structure. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ivan Smahin wrote: > Hello Chuck, > > Tuesday, November 22, 2005, 12:06:21 AM, you wrote: > > CN> I am probably doing something obvious here but does anyone see anything > CN> wrong with this. > > CN> INSERT INTO class (name,text,ID) VALUES (:1 ,:2 ,:3 ) > CN> This is the data - ["Test Class", "This is a test of this class", > CN> "I2G8L5-11O3VGE-FRP6HU-1"] > > CN> This comes up with this error... > CN> ERROR: 462848 > > CN> I could not find that error in the list of error codes. I also tried > CN> inserting it directly in the SQL and got the same results. > > > You should convert it to the hex. > It would be 71000 - ERR_SQL_PARSER_ERROR. > > 'text' seems to be a keyword. > The best solution is to avoid using such an identifiers. > > > From sunshine at public.kherson.ua Tue Nov 22 00:29:21 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 16:31:27 2005 Subject: SQL Error In-Reply-To: <438249D9.5050807@mediamacros.com> Message-ID: On 11/22/05 12:27 AM, "Chuck Neal" wrote: > Unfortunately I do not have the luxury of changing it since its an > existing project. Any work arounds other than using the API syntax? I > really prefer to stick with SQL for now since this will be paralleling > another app that uses the same structure. You can try double quotes: INSERT INTO class (name,"text",ID) Or [] INSERT INTO class (name,[text],ID) Usually this is used when identifier have spaces. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From chuck at mediamacros.com Mon Nov 21 17:45:28 2005 From: chuck at mediamacros.com (Chuck Neal) Date: Mon Nov 21 16:43:13 2005 Subject: SQL Error In-Reply-To: References: Message-ID: <43824E08.4000403@mediamacros.com> The quotes did not seem to work but I will try brackets. I might need to talk to the client and see if we can work out a translation between the 2 apps so I can safely change the names as this looks like it will be e repeat problem. -Chuck -------------------------- Chuck Neal CEO, MediaMacros, Inc. chuck@mediamacros.com http://www.mediamacros.com -------------------------- Check out the Developers Mall Your one stop shop for all your Director Xtra Needs http://www.mediamacros.net/customer Ruslan Zasukhin wrote: > On 11/22/05 12:27 AM, "Chuck Neal" wrote: > > >>Unfortunately I do not have the luxury of changing it since its an >>existing project. Any work arounds other than using the API syntax? I >>really prefer to stick with SQL for now since this will be paralleling >>another app that uses the same structure. > > > You can try double quotes: > > INSERT INTO class (name,"text",ID) > > Or [] > > INSERT INTO class (name,[text],ID) > > > Usually this is used when identifier have spaces. > > > From sunshine at public.kherson.ua Tue Nov 22 00:44:44 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Mon Nov 21 16:45:47 2005 Subject: LoadDump making too many files In-Reply-To: <438248A4.8010707@mediamacros.com> Message-ID: On 11/22/05 12:22 AM, "Chuck Neal" wrote: > I think I found one of the issues. Its an error on export not import. > > The loaddump does make 4 files, no matter what. > The other table error was due to a field in the table being named with > an "illegal" name. All of those tables had fields named "Text". In most > any SQL you can get around this by putting the name in quotes, but the > dump should, in my opinion, include all names in quotes by default to > avoid this. That would prevent this from happening. Absolutely correct !!! And V2 do not wrap them by quotes? Ivan put this into Mantis please -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 22 09:19:08 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 22 01:20:13 2005 Subject: Data compression issue for Valentina 2.0 Beta In-Reply-To: Message-ID: On 11/22/05 8:16 AM, "? ??" wrote: > We feel it is almost same speed with 1.0. > I have questions. > We are trying to load another test file and we come to know there is big > differences > according to the number of null columns. > Let's suppose there are A, B, C, D columns in a table and we are just using > A,B columns. > In this case, is Valentina requiring disk space for C, D, though we didn't > give values for them? In general yes. Numeric fields, String[N] -- eat bytes as declared. VarChar eats 8 bytes only. empty BLOB/Text field eat 4 bytes. > And we'd like to know whether the length of column name can affect the size of > database files or not ? Name of column? Of course no. it is stored once in .vdb file. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Tue Nov 22 23:36:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Tue Nov 22 15:37:38 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/22/05 12:21 AM, "Ed Kleban" wrote: > // When you define your VTable subclasses you define the FieldCursors > // you want to be able to use in addition to table fields: > > Sub CTblForms(db as CRbProjectDB, tableName as String) > // Initialize the receiver and its fields. > > self.name = tableName > > // Define the Table Fields > fFrmFlags = new VByte( "FFrmFlags" ) > fFrmLabel = new VVarChar( "FFrmLabel", 504 ) > fFrmTagTyp = new VVarChar( "FFrmTagTyp", 504) > fFrmXmlTag = new VVarChar( "FFrmXmlTag", 504) > > // Define the FieldCursors > cursorOnFFrmXmlTag = new CFCVarChar( fFrmXmlTag ) > cursorOnFFrmFlags = new CFCByte( fFrmFlags ) > End Sub > > > // Subsequently, in your code you can use the FieldCursors > // to randomly access a single field of a given record without > // reading in every single table field when all you need is > // a single field's value: > > flagByteFromRecord32 = MyTblForms.cursorOnFFrmFlags.integerValue( 32 ) > MyTblForms.cursorOnFFrmFlags.integerValue( 21 ) = kFlagValForRec21 Okay, I see idea of hacking :-) Only problem you can meet is record locks. May be you will prefer use kNoLocks. > // To make this possible... > > // You need to define a subclass of CFCField for each different > // kind of Vfield subclass that you want to be able create a > // FieldCursor for. This gets done once and reused in all the > // applications where you want to use FieldCursors. It is essentially > // the common FieldCursor add-in code. Here are some example guts > // out of CFCByte: > > Function integerValue(recId as Integer, doUpdateCheck = true) As Integer > // Return an Integer with the value of the reciever's fcCursorField at > // recId from the updated fcCursor on fcTable in fcDatabase. > // If doUpdateCheck is false then assume that no rows have been > // added to fcTable and that the fcCursor is valid. > > if doUpdateCheck and updateFieldCursor then // fcCursorChanged > fcFCByte = fcCursor.ByteField( 1 ) > end > > fcCursor.position = recId > return fcFCByte.Value > End Function > > > Function integerValue(recId as Integer, newIntVal as Integer, _ > doUpdateCheck = true, assigns newValStr as string) As String > > // Change the value of the reciever's fcCursorField at recId > // from the updated fcCursor on fcTable in fcDatabase to newIntVal. > // If doUpdateCheck is false then assume that no rows have been > // added to fcTable and that the fcCursor is valid. > > if doUpdateCheck and updateFieldCursor then // fcCursorChanged > fcFCByte = fcCursor.ByteField( 1 ) > end > > fcCursor.position = recId > fcFCByte.Value = newIntVal > fcCursor.UpdateRecord > End Function > > > // The central magic that makes all this work transparently so you > // don't have worry about the state of the FieldCursor is the > // updateFieldCursor method, and the common code called from the > // constructors for all CFCField subclasses: > > Protected Function updateFieldCursor() As Boolean > // Update the receiver's cursor from it's field if necessary. > // Return true if the Cursor was updated or false if the > // former fcCursor is fine to use. > > const here = "updateFieldCursor" > > if fcCursor is nil or fcCursor.RecordCount <> fcTable.RecordCount then > fcCursor = fcDatabase.SqlSelect( _ > fcSqlString, EVCursorLocation.kClientSide, _ > EVLockType.kNoLocks, EVCursorDirection.kRandom ) KClientSide -- give you READ ONLY cursor. Better use ServerSide > #if DebugBuild then > if fcCursor.RecordCount <> fcTable.RecordCount then > fail( here, "Update error" ) > end if > #endif > > return true // Cursor was modified > else > return false // Cursor is fine as-is. > end > End Function > > > Sub CFCField(aVField as VField) > // Initialize the receiver to be a cursor on aVField > > const here = "CFCField" > > #if DebugBuild then > if aVField is nil or aVField.Table is nil _ > or aVField.Table.Database is nil then > fail( here, "Implementation Error" ) > end > #endif > > if not self.checkFieldType( aVField ) then > fail( here, "Invalid VField subclass" ) > end > > // Cache useful values in properties > fcTable = aVField.Table > fcSqlString = glue( "Select ", aVField.Name, " from ", fcTable.Name ) > fcDatabase = aVField.Table.Database > // There is no need or reason to cache the argument aVField. > End Sub -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Wed Nov 23 20:45:53 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 23 13:47:03 2005 Subject: Crash during startup Message-ID: Hi, I get a crash with this line: mType = new VByte ("myType") mType is of type VByte. The crash is always on this line, after going thru several tables and a lot of field. I get no logs, just a crash. I have tried to increase cache size with no results. RB2005r4 + V4RB 2.0.5 I am porting several app to V4RB 2.0.5 but so far I get only crashes and no info... I have followed the references "Converting V4RB 1.x projects to 2.0" completely. I can't compile but nothing else. ~/Stan From maxprog at mac.com Wed Nov 23 22:52:39 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 23 15:53:47 2005 Subject: Crash during startup In-Reply-To: References: Message-ID: <9A9B9871-92DB-4D3E-A980-FDDC16A7E6A9@mac.com> Hi, I respond to myself as I have found the error, a duplicate field name. Only a question, is: vdb.Table("myTable").Field(14).Name = "fieldnewname" correct to rename a field? Stan > Hi, > > I get a crash with this line: mType = new VByte ("myType") > mType is of type VByte. The crash is always on this line, after > going thru several tables and a lot of field. I get no logs, just a > crash. I have tried to increase cache size with no results. > > RB2005r4 + V4RB 2.0.5 > > I am porting several app to V4RB 2.0.5 but so far I get only > crashes and no info... I have followed the references "Converting > V4RB 1.x projects to 2.0" completely. I can't compile but nothing > else. > > ~/Stan > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From IvanSmahin at public.kherson.ua Thu Nov 24 00:04:43 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Wed Nov 23 16:05:49 2005 Subject: Crash during startup In-Reply-To: <9A9B9871-92DB-4D3E-A980-FDDC16A7E6A9@mac.com> References: <9A9B9871-92DB-4D3E-A980-FDDC16A7E6A9@mac.com> Message-ID: <2910033178.20051124000443@public.kherson.ua> Hello Stan, Wednesday, November 23, 2005, 11:52:39 PM, you wrote: SB> Hi, SB> I respond to myself as I have found the error, a duplicate field name. SB> Only a question, is: SB> vdb.Table("myTable").Field(14).Name = "fieldnewname" SB> correct to rename a field? Seems to be correct. But you need to be insure: 1. Table myTable exists. 2. Field with index 14 exists. -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From sunshine at public.kherson.ua Thu Nov 24 09:15:47 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 01:16:56 2005 Subject: Crash during startup In-Reply-To: <9A9B9871-92DB-4D3E-A980-FDDC16A7E6A9@mac.com> Message-ID: On 11/23/05 11:52 PM, "Stan Busk" wrote: Hi Stan, I have report this into Mantis, issue #1151 Stan, please give us more info. where was duplicate name? We need reproduce to catch this and return error > Hi, > > I respond to myself as I have found the error, a duplicate field name. > > Only a question, is: > > vdb.Table("myTable").Field(14).Name = "fieldnewname" > > correct to rename a field? > > Stan > >> Hi, >> >> I get a crash with this line: mType = new VByte ("myType") >> mType is of type VByte. The crash is always on this line, after >> going thru several tables and a lot of field. I get no logs, just a >> crash. I have tried to increase cache size with no results. >> >> RB2005r4 + V4RB 2.0.5 >> >> I am porting several app to V4RB 2.0.5 but so far I get only >> crashes and no info... I have followed the references "Converting >> V4RB 1.x projects to 2.0" completely. I can't compile but nothing >> else. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Thu Nov 24 16:08:44 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 09:10:00 2005 Subject: Crash during startup In-Reply-To: References: Message-ID: Hi, One of my table (I use classes) was using the same field name twice: mTransactionType = new VULong( "Scheduling_Type" ) and later in the constructor: mType = new VByte( "Scheduling_Type" ) When reaching that second line the app crashes with no error message on screen nor on log file. I activated the debugger and went thru my VDatabase class until I reach that table and then went thru the table until the mType field. Stan > On 11/23/05 11:52 PM, "Stan Busk" wrote: > > Hi Stan, > > I have report this into Mantis, issue #1151 > > Stan, please give us more info. > where was duplicate name? > > We need reproduce to catch this and return error > >> Hi, >> >> I respond to myself as I have found the error, a duplicate field >> name. >> >> Only a question, is: >> >> vdb.Table("myTable").Field(14).Name = "fieldnewname" >> >> correct to rename a field? >> >> Stan >> >>> Hi, >>> >>> I get a crash with this line: mType = new VByte ("myType") >>> mType is of type VByte. The crash is always on this line, after >>> going thru several tables and a lot of field. I get no logs, just a >>> crash. I have tried to increase cache size with no results. >>> >>> RB2005r4 + V4RB 2.0.5 >>> >>> I am porting several app to V4RB 2.0.5 but so far I get only >>> crashes and no info... I have followed the references "Converting >>> V4RB 1.x projects to 2.0" completely. I can't compile but nothing >>> else. > > -- > Best regards, > > Ruslan Zasukhin > VP Engineering and New Technology > Paradigma Software, Inc > > Valentina - Joining Worlds of Information > http://www.paradigmasoft.com > > [I feel the need: the need for speed] > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From maxprog at mac.com Thu Nov 24 16:29:26 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 09:30:37 2005 Subject: VAdatabse initialization crash Message-ID: <68202D13-F573-4F4B-B5DF-2597B921BF28@mac.com> Hi, The following crashes: Dim vdb as vDatabase vdb = new vDataBase() vdb.open( inFile ) <-Crash here bur the following works: Dim vdb as vDatabase vdb = new vDataBase( EVStorageType.kDisk ) vdb.open( inFile ) However in the doc I have read that if no parameter is passed to the VDatabase constructor, the default is EVStorageType.kDisk. Seems that or the doc is not correct or the function has a bug because the function returns nil if no parameter is passed. ~/Stan From maxprog at mac.com Thu Nov 24 17:11:09 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 10:12:23 2005 Subject: Replacing LIKE with REGEX doubts Message-ID: <3EDA2E39-E4E3-4BF7-B48B-CA1320F02283@mac.com> Hi, Just wondering some of the Valentina v1 LIKE equivalents using v2 REGEX of Is equal to: myField LIKE '\Amytext\Z' no_case Is not equal to: myField NOT LIKE '\Amytext\Z' no_case Contains: myField LIKE 'mytext' no_case Doesn't contain: myField NOT LIKE 'mytext' no_case Starts with: myField LIKE '\Amytext' no_case Ends with: myField LIKE 'mytext\Z' no_case are they?: Is equal to: myField REGEX '(?i)mytext' Is not equal to: ??? Contains: myField REGEX '(?i)[a-zA-Z0-9]+mytext[a-zA-Z0-9]+' Doesn't contain: ??? Starts with: myField REGEX '(?i)[a-zA-Z0-9]+mytext' Ends with: myField REGEX '(?i)mytext[a-zA-Z0-9]+' If I am right with [a-zA-Z0-9]+ is there something better and wider? I mean that could consider more possible characters? TIA, ~/Stan From maxprog at mac.com Thu Nov 24 17:32:59 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 10:34:11 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: <3EDA2E39-E4E3-4BF7-B48B-CA1320F02283@mac.com> References: <3EDA2E39-E4E3-4BF7-B48B-CA1320F02283@mac.com> Message-ID: <171A6C84-34EF-4A30-BD96-E738333F1729@mac.com> Hi, Am I right if I continue using LIKE this way?: Is equal to: myField LIKE 'mytext' Is not equal to: myField NOT LIKE 'mytext' Contains: myField LIKE '%mytext%' Doesn't contain: myField NOT LIKE '%mytext%' Starts with: myField LIKE '%mytext' Ends with: myField LIKE 'mytext%' Is new Valentina 2 LIKE case insensitive? or is it case sensitive? What characters do I need to escape, the '%', '_' and '/' ? Is it faster than REGEX? Right now it looks easier. Stan > Hi, > > Just wondering some of the Valentina v1 LIKE equivalents using v2 > REGEX of > > Is equal to: myField LIKE '\Amytext\Z' no_case > Is not equal to: myField NOT LIKE '\Amytext\Z' no_case > Contains: myField LIKE 'mytext' no_case > Doesn't contain: myField NOT LIKE 'mytext' no_case > Starts with: myField LIKE '\Amytext' no_case > Ends with: myField LIKE 'mytext\Z' no_case > > > are they?: > > > Is equal to: myField REGEX '(?i)mytext' > Is not equal to: ??? > Contains: myField REGEX '(?i)[a-zA-Z0-9]+mytext[a-zA-Z0-9]+' > Doesn't contain: ??? > Starts with: myField REGEX '(?i)[a-zA-Z0-9]+mytext' > Ends with: myField REGEX '(?i)mytext[a-zA-Z0-9]+' > > If I am right with [a-zA-Z0-9]+ is there something better and > wider? I mean that could consider more possible characters? > > TIA, > ~/Stan > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Thu Nov 24 18:42:17 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 10:43:27 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: <3EDA2E39-E4E3-4BF7-B48B-CA1320F02283@mac.com> Message-ID: On 11/24/05 6:11 PM, "Stan Busk" wrote: Hi Stan, > Hi, > > Just wondering some of the Valentina v1 LIKE equivalents using v2 > REGEX of > > Is equal to: myField LIKE '\Amytext\Z' no_case > Is not equal to: myField NOT LIKE '\Amytext\Z' no_case > Contains: myField LIKE 'mytext' no_case > Doesn't contain: myField NOT LIKE 'mytext' no_case > Starts with: myField LIKE '\Amytext' no_case > Ends with: myField LIKE 'mytext\Z' no_case > > > are they?: > > > Is equal to: myField REGEX '(?i)mytext' > Is not equal to: ??? myField REGEX NOT '(?i)mytext' myField NOT REGEX '(?i)mytext' > Contains: myField REGEX '(?i)[a-zA-Z0-9]+mytext[a-zA-Z0-9]+' > Doesn't contain: ??? > Starts with: myField REGEX '(?i)[a-zA-Z0-9]+mytext' > Ends with: myField REGEX '(?i)mytext[a-zA-Z0-9]+' > > If I am right with [a-zA-Z0-9]+ is there something better and wider? > I mean that could consider more possible characters? I believe that in Valentina SQL.pdf exists text about how todo INDEXED searches StartsWith, Like, ... 2) note new feature: db.CollationAttribute( kStrength ) = kPrimary This give you the wole db case-insensitive. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 24 18:43:30 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 10:44:39 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: <171A6C84-34EF-4A30-BD96-E738333F1729@mac.com> Message-ID: On 11/24/05 6:32 PM, "Stan Busk" wrote: > Hi, > > Am I right if I continue using LIKE this way?: > > Is equal to: myField LIKE 'mytext' > Is not equal to: myField NOT LIKE 'mytext' > Contains: myField LIKE '%mytext%' > Doesn't contain: myField NOT LIKE '%mytext%' > Starts with: myField LIKE '%mytext' > Ends with: myField LIKE 'mytext%' > > Is new Valentina 2 LIKE case insensitive? or is it case sensitive? It is depend on Database/Table/Field settings. > What characters do I need to escape, the '%', '_' and '/' ? only 2: ' \ > Is it faster than REGEX? Right now it looks easier. I think it is little faster because simpler rules. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 24 18:45:31 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 10:46:42 2005 Subject: Crash during startup In-Reply-To: Message-ID: On 11/24/05 5:08 PM, "Stan Busk" wrote: > Hi, > > One of my table (I use classes) was using the same field name twice: > > mTransactionType = new VULong( "Scheduling_Type" ) > > and later in the constructor: > > mType = new VByte( "Scheduling_Type" ) > > When reaching that second line the app crashes with no error message > on screen nor on log file. Aha, so easy to reproduce... thanks > I activated the debugger and went thru my > VDatabase class until I reach that table and then went thru the table > until the mType field. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 24 18:47:22 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 10:48:32 2005 Subject: VAdatabse initialization crash. Bug ? In-Reply-To: <68202D13-F573-4F4B-B5DF-2597B921BF28@mac.com> Message-ID: On 11/24/05 5:29 PM, "Stan Busk" wrote: > Hi, > > The following crashes: > > Dim vdb as vDatabase > vdb = new vDataBase() > vdb.open( inFile ) <-Crash here > > bur the following works: > > Dim vdb as vDatabase > vdb = new vDataBase( EVStorageType.kDisk ) > vdb.open( inFile ) Strange, yes it is default. Optional, I believe many our examples do first way and not crash. > However in the doc I have read that if no parameter is passed to the > VDatabase constructor, the default is EVStorageType.kDisk. > Seems that or the doc is not correct or the function has a bug because the > function returns nil if no parameter is passed. Okay we will check this point. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Thu Nov 24 18:06:17 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 11:07:28 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: References: Message-ID: Is it EVColAttributeValue.kPrimary like in: myReference = new VvarChar ( "myReference", 64, EVFlag.fIndexed + EVFlag.fUnique + EVColAttributeValue.kPrimary ) If I edit those fields, should I also update schema on old database files? Is it?: vdb.Table("myTable").Field("myField").CollationAttribute = EVColAttributeValue.kPrimary Stan >> Am I right if I continue using LIKE this way?: >> >> Is equal to: myField LIKE 'mytext' >> Is not equal to: myField NOT LIKE 'mytext' >> Contains: myField LIKE '%mytext%' >> Doesn't contain: myField NOT LIKE '%mytext%' >> Starts with: myField LIKE '%mytext' >> Ends with: myField LIKE 'mytext%' >> >> Is new Valentina 2 LIKE case insensitive? or is it case sensitive? > > It is depend on Database/Table/Field settings. From maxprog at mac.com Thu Nov 24 18:08:57 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 11:10:09 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: References: Message-ID: Hi, Can you use that before a Query? or is it an initialization function? If it is, where do you place it? Stan > 2) note new feature: > > db.CollationAttribute( kStrength ) = kPrimary > > This give you the wole db case-insensitive. From sunshine at public.kherson.ua Thu Nov 24 21:02:55 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 13:04:03 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: Message-ID: On 11/24/05 7:06 PM, "Stan Busk" wrote: > Is it EVColAttributeValue.kPrimary like in: > > myReference = new VvarChar ( "myReference", 64, EVFlag.fIndexed + > EVFlag.fUnique + EVColAttributeValue.kPrimary ) > > If I edit those fields, should I also update schema on old database > files? Is it?: > > vdb.Table("myTable").Field("myField").CollationAttribute = > EVColAttributeValue.kPrimary This is permanent option. It is bad idea change it too often. On ach change index(es) must be rebuilt Usually people set it only for db. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Thu Nov 24 21:03:45 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 13:04:54 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: Message-ID: On 11/24/05 7:08 PM, "Stan Busk" wrote: > Hi, > > Can you use that before a Query? or is it an initialization function? > If it is, where do you place it? I think good idea is near to db.Create > Stan > >> 2) note new feature: >> >> db.CollationAttribute( kStrength ) = kPrimary >> >> This give you the wole db case-insensitive. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Thu Nov 24 20:31:42 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 13:32:54 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: References: Message-ID: And in a schema update for old database I guess if I want the change to be permanent and for old databases to be in sync with new database settings, am I right? Last question for today, is the word 'Transactions' reserved? In ValentinaSQL document 'List of Key Words of Valentina SQL' part only singular 'Transaction' is listed. In two different apps, SQLSelect scrash the application when using my 'Transactions' table. I haven't tried to rename yet though, Thanks for all your responses, I am almost finished porting 3 apps. Stan > On 11/24/05 7:08 PM, "Stan Busk" wrote: > >> Hi, >> >> Can you use that before a Query? or is it an initialization function? >> If it is, where do you place it? > > I think good idea is near to db.Create > > >> Stan >> >>> 2) note new feature: >>> >>> db.CollationAttribute( kStrength ) = kPrimary >>> >>> This give you the wole db case-insensitive. > > -- > Best regards, > > Ruslan Zasukhin > VP Engineering and New Technology > Paradigma Software, Inc > > Valentina - Joining Worlds of Information > http://www.paradigmasoft.com > > [I feel the need: the need for speed] > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Thu Nov 24 21:39:48 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Thu Nov 24 13:40:56 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: Message-ID: On 11/24/05 9:31 PM, "Stan Busk" wrote: > And in a schema update for old database I guess if I want the change > to be permanent and for old databases to be in sync with new database > settings, am I right? Sorry I have not catch question. > Last question for today, is the word 'Transactions' reserved? You can find alphabetical list of all keywords in ValentinaSQL.pdf In its Appendix. Transactions - no Transaction - yes > In > ValentinaSQL document 'List of Key Words of Valentina SQL' part only > singular 'Transaction' is listed. I type before read :-) Right > In two different apps, SQLSelect > scrash the application when using my 'Transactions' table. I haven't > tried to rename yet though, Hmm. Sounds like a bug. > Thanks for all your responses, I am almost finished porting 3 apps. Good. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Thu Nov 24 21:12:28 2005 From: maxprog at mac.com (Stan Busk) Date: Thu Nov 24 14:13:39 2005 Subject: Replacing LIKE with REGEX doubts In-Reply-To: References: Message-ID: Confirmed, I have change the table name to 'tblTransactions' and now it works. Definitively the name 'Transactions' is not allowed in some way. I should have named tables with the tbl suffix years ago. Stan >> In two different apps, SQLSelect >> scrash the application when using my 'Transactions' table. I haven't >> tried to rename yet though, > > Hmm. Sounds like a bug. From Ed at Kleban.com Thu Nov 24 18:25:48 2005 From: Ed at Kleban.com (Ed Kleban) Date: Thu Nov 24 18:37:19 2005 Subject: When FindSingle fails Message-ID: Presumably if FindSingle fails to find a match it returns recID 0 ? From maxprog at mac.com Fri Nov 25 18:04:50 2005 From: maxprog at mac.com (Stan Busk) Date: Fri Nov 25 11:06:02 2005 Subject: [V4RB] Query crash when indexed values Message-ID: <52514ECD-8A66-4D93-B775-D52924A528B9@mac.com> Hi, I have the follwing query that always crash: SELECT * FROM myTable WHERE myValue1='11' and myValue2='2005' ORDER BY myValue3 I use the EVCursorLocation.kServerSide, EVLockType.kNoLocks and EVCursorDirection.kRandom when perforing the Query. myValue1 and myValue2 are declared that way: myValue1 = new VByte ("myValue1", EVFlag.fIndexed) myValue2 = new VUShort ("myValue2", EVFlag.fIndexed) If I remove the EVFlag.fIndexed flags, automatically the Query works. Am I using the wrong flag? or are VByte and VUShort no longer indexable? TIA, ~/Stan From sunshine at public.kherson.ua Fri Nov 25 21:11:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 13:12:21 2005 Subject: When FindSingle fails In-Reply-To: Message-ID: On 11/25/05 2:25 AM, "Ed Kleban" wrote: > Presumably if FindSingle fails to find a match it returns recID 0 ? Yes Ed, And note, there is no records with RecID = 0. So it is legal to use 0 as sign of search failure -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 25 21:13:33 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 13:14:43 2005 Subject: [V4RB] Query crash when indexed values In-Reply-To: <52514ECD-8A66-4D93-B775-D52924A528B9@mac.com> Message-ID: On 11/25/05 7:04 PM, "Stan Busk" wrote: Hi Stan > Hi, > > I have the follwing query that always crash: > > SELECT * FROM myTable WHERE myValue1='11' and myValue2='2005' ORDER > BY myValue3 > > I use the EVCursorLocation.kServerSide, EVLockType.kNoLocks and > EVCursorDirection.kRandom when perforing the Query. ok > myValue1 and myValue2 are declared that way: > > myValue1 = new VByte ("myValue1", EVFlag.fIndexed) > myValue2 = new VUShort ("myValue2", EVFlag.fIndexed) ok > If I remove the EVFlag.fIndexed flags, automatically the Query works. Because it will not use index. > Am I using the wrong flag? or are VByte and VUShort no longer indexable? In any case it should not crash. Aha, why you use quotes? Please try this SELECT * FROM myTable WHERE myValue1=11 and myValue2=2005 ORDER -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jda at his.com Fri Nov 25 14:43:57 2005 From: jda at his.com (jda) Date: Fri Nov 25 13:45:08 2005 Subject: VChar vs VText In-Reply-To: References: Message-ID: Hi Ruslan, I'm going to add about 12 new fields to my db (API way). They could be VText (which I mostly use) or VChar (I don't care if they are limited to 1K or so of text). Is there any *real* advantage of one over the other (storage-wise)? I expect these fields will generally store from 5-200 characters or so. If I use VChar, what's the smallest segment size that conserves space? I recall something like 5024. If VText, what's the smallest that makes sense (32)? Thanks, Jon From sunshine at public.kherson.ua Fri Nov 25 22:07:54 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 14:09:06 2005 Subject: VChar vs VText In-Reply-To: Message-ID: On 11/25/05 9:43 PM, "jda" wrote: Hi Jon, > I'm going to add about 12 new fields to my db (API way). They could > be VText (which I mostly use) or VChar (I don't care if they are > limited to 1K or so of text). Is there any *real* advantage of one > over the other (storage-wise)? I expect these fields will generally > store from 5-200 characters or so. > > If I use VChar, what's the smallest segment size that conserves > space? I recall something like 5024. > > If VText, what's the smallest that makes sense (32)? Vtext MUST be choose if you know that document can be very big, If you say 1K is okay, then use VarChar -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Fri Nov 25 21:09:38 2005 From: maxprog at mac.com (Stan Busk) Date: Fri Nov 25 14:10:52 2005 Subject: [V4RB] Query crash when indexed values In-Reply-To: References: Message-ID: Hi, Do you mean the ' is no longer required? Why? > Aha, why you use quotes? > Please try this > > SELECT * FROM myTable WHERE myValue1=11 and myValue2=2005 ORDER Also, I have a crash here: aCursor = myDatabase.SQLselect("SELECT myTable_Field FROM myTable ORDER BY myTable_Date DESC", EVCursorLocation.kServerSide, EVLockType.kNoLocks, EVCursorDirection.kRandom) It always crash. However: aCursor = myDatabase.SQLselect("SELECT myTable_Field,myTable_Date FROM myTable ORDER BY myTable_Date DESC", EVCursorLocation.kServerSide, EVLockType.kNoLocks, EVCursorDirection.kRandom) works. Seems I can no longer SELECT a particular field without including the field I order with. Am I right? Stan From sunshine at public.kherson.ua Fri Nov 25 22:16:31 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 14:17:42 2005 Subject: [V4RB] Query crash when indexed values In-Reply-To: Message-ID: On 11/25/05 10:09 PM, "Stan Busk" wrote: > Do you mean the ' is no longer required? Why? Stan, it Was required and IS required only for string and datetime constants. You should not use quotes for numbers. >> Aha, why you use quotes? >> Please try this >> >> SELECT * FROM myTable WHERE myValue1=11 and myValue2=2005 ORDER -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Fri Nov 25 22:18:33 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 14:19:44 2005 Subject: [V4RB] Query crash when indexed values In-Reply-To: Message-ID: On 11/25/05 10:09 PM, "Stan Busk" wrote: May be try REINDEX your db ? Or test on fresh ? > Also, I have a crash here: > > aCursor = myDatabase.SQLselect("SELECT myTable_Field FROM myTable > ORDER BY myTable_Date DESC", EVCursorLocation.kServerSide, > EVLockType.kNoLocks, EVCursorDirection.kRandom) > > It always crash. However: > > aCursor = myDatabase.SQLselect("SELECT myTable_Field,myTable_Date > FROM myTable ORDER BY myTable_Date DESC", > EVCursorLocation.kServerSide, EVLockType.kNoLocks, > EVCursorDirection.kRandom) > > works. Seems I can no longer SELECT a particular field without > including the field I order with. > > Am I right? Yes this is correct. SQL92 require that you can ORDER BY only on field which is SELECTed Ivan, but we should not crash in first case also. Strange, I believe we have such tests. Please put on TODO -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From jda at his.com Fri Nov 25 17:04:41 2005 From: jda at his.com (jda) Date: Fri Nov 25 16:05:58 2005 Subject: VChar vs VText In-Reply-To: References: Message-ID: >On 11/25/05 9:43 PM, "jda" wrote: > >Hi Jon, > >> I'm going to add about 12 new fields to my db (API way). They could >> be VText (which I mostly use) or VChar (I don't care if they are >> limited to 1K or so of text). Is there any *real* advantage of one >> over the other (storage-wise)? I expect these fields will generally >> store from 5-200 characters or so. >> >> If I use VChar, what's the smallest segment size that conserves >> space? I recall something like 5024. >> >> If VText, what's the smallest that makes sense (32)? > >Vtext MUST be choose if you know that document can be very big, > >If you say 1K is okay, then use VarChar > Is there any advantage to VarChar (remember, typical data of 5-200 chars per fields, UTF-16 encoding)? And what segment size is optimal (for saving storage space)? Jon From macsforever2000 at goodeast.com Fri Nov 25 15:21:15 2005 From: macsforever2000 at goodeast.com (Frank Schima) Date: Fri Nov 25 16:22:29 2005 Subject: VChar vs VText In-Reply-To: References: Message-ID: Hi Jon, On Nov 25, 2005, at 3:04 PM, jda wrote: > Is there any advantage to VarChar (remember, typical data of 5-200 > chars per fields, UTF-16 encoding)? And what segment size is > optimal (for saving storage space)? You do not set the segment size with a VarChar, instead it is the maximum size. 1022 is the minimum value you should use, at least for double byte encodings like UTF-16 which is the default. I'm not sure what the header size is, so if you need to jump to the next optimal size, I don't think 2044 is the proper value, probably it should be less. Perhaps Ruslan can comment on this? My understanding is that VarChar is faster and more space efficient than VText. Really, Vtext is only for holding potentially large amounts of text. Best regards, Frank Schima I'm currently job hunting: From jda at his.com Fri Nov 25 17:33:20 2005 From: jda at his.com (jda) Date: Fri Nov 25 16:35:53 2005 Subject: VChar vs VText In-Reply-To: References: Message-ID: Hi Frank, >You do not set the segment size with a VarChar, instead it is the >maximum size. Ah, of course. Should have remembered that... > 1022 is the minimum value you should use, at least for double byte >encodings like UTF-16 which is the default. Wasn't that for V1? I thought it changed in V2 to something over 5000. Ruslan? > I'm not sure what the header size is, so if you need to jump to the >next optimal size, I don't think 2044 is the proper value, probably >it should be less. Perhaps Ruslan can comment on this? > >My understanding is that VarChar is faster and more space efficient >than VText. Really, Vtext is only for holding potentially large >amounts of text. > Sounds like CVhar then. Thanks. Jon From sunshine at public.kherson.ua Sat Nov 26 00:41:05 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Fri Nov 25 16:42:16 2005 Subject: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 12:33 AM, "jda" wrote: > Hi Frank, > >> You do not set the segment size with a VarChar, instead it is the >> maximum size. > > Ah, of course. Should have remembered that... > >> 1022 is the minimum value you should use, at least for double byte >> encodings like UTF-16 which is the default. > > Wasn't that for V1? I thought it changed in V2 to something over 5000. Ruslan? Best VarChar size in v2 for UTF16 strings is 1022 1022 * 2 = 2044 2 max big string on page 2044 * 2 = 4088. So we get 4Kb page -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Fri Nov 25 21:35:29 2005 From: Ed at Kleban.com (Ed Kleban) Date: Fri Nov 25 21:36:48 2005 Subject: VChar vs VText In-Reply-To: Message-ID: I was using 504 because I was thinking about UTF-8 strings. But UTF-8 doesn't work well in V4RB v2.0 correct? So should I change to using 1022? Also, when I get a string into a RB variable with GetString, is there a way to retrive a UTF-8 string? Is it slower if I do rather than retrieveing UTF-16 strings? Either way, I'm sure glad I'm doing my searches on string hashes rather than actual strings. On 11/25/05 4:41 PM, "Ruslan Zasukhin" wrote: > On 11/26/05 12:33 AM, "jda" wrote: > >> Hi Frank, >> >>> You do not set the segment size with a VarChar, instead it is the >>> maximum size. >> >> Ah, of course. Should have remembered that... >> >>> 1022 is the minimum value you should use, at least for double byte >>> encodings like UTF-16 which is the default. >> >> Wasn't that for V1? I thought it changed in V2 to something over 5000. >> Ruslan? > > Best VarChar size in v2 for UTF16 strings is 1022 > > 1022 * 2 = 2044 > > 2 max big string on page 2044 * 2 = 4088. > > So we get 4Kb page > From sunshine at public.kherson.ua Sat Nov 26 09:31:12 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 27 13:23:05 2005 Subject: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 5:35 AM, "Ed Kleban" wrote: > I was using 504 because I was thinking about UTF-8 strings. But UTF-8 > doesn't work well in V4RB v2.0 correct? So should I change to using 1022? Yes for TF16 Actually in v2 for UTF8 the best size of VarChar will be 2044. > Also, when I get a string into a RB variable with GetString, is there a way > to retrive a UTF-8 string? You get it as UTF8 now right now !!! REALbasic work with UTF8 on default. > Is it slower if I do rather than retrieveing > UTF-16 strings? > > Either way, I'm sure glad I'm doing my searches on string hashes rather than > actual strings. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sat Nov 26 09:31:48 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 27 13:23:17 2005 Subject: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 5:35 AM, "Ed Kleban" wrote: > Either way, I'm sure glad I'm doing my searches on string hashes rather than > actual strings. Hashing can work only for search fld = "xxx" But you cannot do range searches using hashing, right ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From maxprog at mac.com Sat Nov 26 11:53:39 2005 From: maxprog at mac.com (Stan Busk) Date: Sun Nov 27 13:23:29 2005 Subject: [V4RB] Queries with boolean fields Message-ID: Hi, The following Query crashes: SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND myField3=0 AND myField4=1 myField3 and myField4 are Boolean. The following Query doesn't crash: SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND NOT myField3 AND myField4 Am I correct to not use the =0 and =1 anymore but rather the field directly and 'NOT' for false values? ~/Stan From maxprog at mac.com Sat Nov 26 13:41:12 2005 From: maxprog at mac.com (Stan Busk) Date: Sun Nov 27 13:23:41 2005 Subject: [V4RB] Queries with boolean fields Message-ID: <3387D19E-3011-4B01-A87A-DDA543D7837F@mac.com> Hi, The following Query crashes: SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND myField3=0 AND myField4=1 myField3 and myField4 are Boolean. The following Query doesn't crash: SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND NOT myField3 AND myField4 Am I correct to not use the =0 and =1 anymore but rather the field directly and 'NOT' for false values? ~/Stan From cbpelto at pcisys.net Sat Nov 26 09:55:02 2005 From: cbpelto at pcisys.net (Chuck Pelto) Date: Sun Nov 27 13:24:01 2005 Subject: Errors In V-Studio Message-ID: <600B0C7D-C957-49DF-81E5-DE962D1AEB81@pcisys.net> Getting error messages when trying to create a new database of the V2 persuasion in VStudio. The first error reads as follows: 09:48:32: Kernel error: 0x59505 09:48:32: File "/private/var/tmp/folders.501/TemporaryItems/28247452" is not open. When I click OK in the error message dialog and try to open the just created database, I get THIS: 09:49:50: Kernel error: 0x59502 09:49:50: File "0x100030000" does not exist. 09:49:50: Database could not be opened! What needs to be done? Regards, Chuck From maxprog at mac.com Sat Nov 26 20:32:43 2005 From: maxprog at mac.com (Stan Busk) Date: Sun Nov 27 13:24:08 2005 Subject: [V4RB] valentina.Convert_1_2 function not working with my files Message-ID: Hi, I have tried the valentina.Convert_1_2 function. None of the converted files can be used, they all crash both my App and Valentina Studio. What can I do? Stan From cbpelto at pcisys.net Sat Nov 26 13:33:57 2005 From: cbpelto at pcisys.net (Chuck Pelto) Date: Sun Nov 27 13:24:16 2005 Subject: A Simple VStudio Question Message-ID: <917E768B-5C51-4E7B-98FE-3D1FACB66221@pcisys.net> Can I develop and populate a V2 database structure in VStudio and then plop it into an RB2005 application without having to go through the tediousness of defining the VDatabase and VTables and CreateStructure? Or can I simple define the VDatabase and VTables with their respective fields and not go through the CreateStructure, as the structure is already defined via the database created in VStudio? Regards, Chuck From jda at his.com Sat Nov 26 16:35:48 2005 From: jda at his.com (jda) Date: Sun Nov 27 13:24:23 2005 Subject: Method char limit followup In-Reply-To: References: Message-ID: Hi Ruslan, I did eventually get this to work...must be a typo buried somewhere in the first query I sent. Jon From jda at his.com Sat Nov 26 11:41:24 2005 From: jda at his.com (jda) Date: Sun Nov 27 13:24:40 2005 Subject: Method length limit? In-Reply-To: References: Message-ID: Hi Ruslan, I use this method to search all fields in my db (all VText): "CONCAT(IFNULL(authors,''),' ',IFNULL(title,''),' ',IFNULL(editors,''),' ',IFNULL(journal,''),' ',IFNULL(volume,''),' ',IFNULL(pages,''),' ',IFNULL(thedate,''),' ',IFNULL(publisher,''),' ', IFNULL(location,''),' ',IFNULL(url,''),' ',IFNULL(title2,''),' ',IFNULL(user1,''),' ',IFNULL(user2,''),' ',IFNULL(user3,''),' ',IFNULL(user4,''),' ',IFNULL(abstract,''),' ',IFNULL(keywords,''),' ',IFNULL(notes,'')" ) I added 12 new fields (all VarChar), which are NULLABLE, INDEXED, and INDEXEDBYWORDS I extended the method to include the new fields "CONCAT(IFNULL(authors,''),' ',IFNULL(title,''),' ',IFNULL(editors,''),' ',IFNULL(journal,''),' ',IFNULL(volume,''),' ',IFNULL(pages,''),' ',IFNULL(thedate,''),' ',IFNULL(publisher,''),' ', IFNULL(location,''),' ',IFNULL(url,''),' ',IFNULL(title2,''),' ',IFNULL(user1,''),' ',IFNULL(user2,''),' ',IFNULL(user3,''),' ',IFNULL(user4,''),' ',IFNULL(abstract,''),' ',IFNULL(keywords,''),' ',IFNULL(notes,''),' ',IFNULL(user5,''),' ',IFNULL(user6,''),' ',IFNULL(user7,''),' ',IFNULL(user8,''),' ',IFNULL(user9,''),' ',IFNULL(user10,''),' ',IFNULL(user11,''),' ',IFNULL(user12,''),' ',IFNULL(user13,''),' ',IFNULL(user14,''),' ',IFNULL(user15,''),' ',IFNULL(user16,'')" ) If I open the db in ViSQL I see that the method returns NULL even when there are fields with data in them. Is there a limit to the length of methods? Is this new method too long? Jon From j.peters at valentina-db.de Sun Nov 27 20:31:24 2005 From: j.peters at valentina-db.de (Jochen Peters) Date: Sun Nov 27 13:31:14 2005 Subject: Errors In V-Studio In-Reply-To: <20051127192451.02E053D8278@edison.macserve.net> References: <20051127192451.02E053D8278@edison.macserve.net> Message-ID: Hi Chuck, please download the latest beta (b17, Win) and try with this. -- Best regards, Jochen Peters PIIT GmbH ------------------------------------ http://www.valentina-db.de > Getting error messages when trying to create a new database of the V2 > persuasion in VStudio. > > The first error reads as follows: > > 09:48:32: Kernel error: 0x59505 > 09:48:32: File "/private/var/tmp/folders.501/TemporaryItems/28247452" > is not open. > > When I click OK in the error message dialog and try to open the just > created database, I get THIS: > > 09:49:50: Kernel error: 0x59502 > 09:49:50: File "0x100030000" does not exist. > 09:49:50: Database could not be opened! > > What needs to be done? > > Regards, > > Chuck From sunshine at public.kherson.ua Sun Nov 27 21:55:42 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 27 13:56:07 2005 Subject: [V4RB] Queries with boolean fields In-Reply-To: <3387D19E-3011-4B01-A87A-DDA543D7837F@mac.com> Message-ID: On 11/26/05 2:41 PM, "Stan Busk" wrote: > Hi, > > The following Query crashes: > > SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND myField3=0 > AND myField4=1 > > myField3 and myField4 are Boolean. > > The following Query doesn't crash: > > SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND NOT > myField3 AND myField4 > > Am I correct to not use the =0 and =1 anymore but rather the field > directly and 'NOT' for false values? What if you right SELECT * FROM myTable WHERE (myField1=2 OR myField2=3) AND myField3=false AND myField4=true ? I think your case also should work. Please send us db for check -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 27 21:56:57 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 27 13:57:20 2005 Subject: [V4RB] valentina.Convert_1_2 function not working with my files In-Reply-To: Message-ID: On 11/26/05 9:32 PM, "Stan Busk" wrote: > Hi, > > I have tried the valentina.Convert_1_2 function. None of the > converted files can be used, they all crash both my App and Valentina > Studio. What can I do? Stan, Ivan have fix one bug in convert_1_2, which have report MacKiev guys. I think tomorrow I will make new beta build. If still not work, then we need one your db of v 1.x -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Sun Nov 27 21:58:54 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Sun Nov 27 13:59:21 2005 Subject: A Simple VStudio Question In-Reply-To: <917E768B-5C51-4E7B-98FE-3D1FACB66221@pcisys.net> Message-ID: On 11/26/05 10:33 PM, "Chuck Pelto" wrote: Hi Chuck, > Can I develop and populate a V2 database structure in VStudio and > then plop it into an RB2005 application without having to go through > the tediousness of defining the VDatabase and VTables and > CreateStructure? Yes of course. ViSQL do this for example. > Or can I simple define the VDatabase and VTables with their > respective fields and not go through the CreateStructure, as the > structure is already defined via the database created in VStudio? Also yes. But will be important to use the same order of creation. We hope in near future add into Valentina Studio support of JavaScript. And then add scripts which will produce code on different languages. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Sun Nov 27 19:57:00 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 27 19:57:25 2005 Subject: Default maximal length for VarChar Re: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 1:31 AM, "Ruslan Zasukhin" wrote: > On 11/26/05 5:35 AM, "Ed Kleban" wrote: > >> I was using 504 because I was thinking about UTF-8 strings. But UTF-8 >> doesn't work well in V4RB v2.0 correct? So should I change to using 1022? > > Yes for TF16 You meant "Yes for UTF-16", presumably. > > Actually in v2 for UTF8 the best size of VarChar will be 2044. > Why? Does this have to do with the natural page size of MacOS X or something? I understood the logic described in the Kernel manual which makes good sense: " The default maximal length for VarChar field is 504 bytes. This is the maximum number that allows us to work with 1,024 byte pages. Indeed: 8 bytes of header + 2 records * (504 + 4) = 1024 bytes. You can specify lower values for maximal length (e.g., 20), but 1,024 byte pages will still be used. The only advantage is that you will truncate longer strings to 20 bytes. " Is this no longer true? Has the change to unicode in V2 made it the case that the default maximal length has changed? Is the Kernel manual incorrect? If the minimal page size is still 1024 bytes, then wouldn't that mean since each character now takes up 2 bytes in UTF-16 when stored, instead of one by as when previously stored in UTF-8, that the default maximal length for a VarChar field should now be 504 / 2 = 252 bytes? I don't have enough detail yet for this to make sense. > >> Also, when I get a string into a RB variable with GetString, is there a way >> to retrive a UTF-8 string? > > You get it as UTF8 now right now !!! > > REALbasic work with UTF8 on default. > >> Is it slower if I do rather than retrieveing >> UTF-16 strings? >> >> Either way, I'm sure glad I'm doing my searches on string hashes rather than >> actual strings. From Ed at Kleban.com Sun Nov 27 20:24:11 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 27 20:24:36 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 1:31 AM, "Ruslan Zasukhin" wrote: > On 11/26/05 5:35 AM, "Ed Kleban" wrote: > >> Either way, I'm sure glad I'm doing my searches on string hashes rather than >> actual strings. > > Hashing can work only for search fld = "xxx" If I understand you correctly, then yes. > > But you cannot do range searches using hashing, right ? Yes, that is correct. You mentioned this also the last time I mentioned using Hashes, and both time have essentially implied that there is some drawback to not having an ability to do > or >= or < or <= searches. I suppose that for some applications that may be true, but it is virtually never a drawback for my applications. In fact I've never even considered wanting to do such a thing. Question: What applications do you have in mind that range searches on strings would be useful for? For 99% of my applications I want to be able to: 1) See if I already have a given string (or object, or structure, or whatever) in my table. 2) If I do, then find that item, or lack thereof, as fast as possible. 3) And then typically add the item into the table if it is not already there. I therefore am doing this on a column of say strings for example, which I always know will be unique. For a table of bytes, shorts, or longs, simply performing a binary search by using a FindSingle on an indexed unique column in Valentina is going to be pretty darn fast. For a table of strings however, it's a whole 'nuther story. First you have the overhead for comparing strings which is much, much slower than comparing integers, On top of that as of V2 we now have the reality that strings are stored in tables in Unicode and are therefore twice the size, -- but which in reality does not imply twice the time since most matches will fail on a comparison of the first few characters. But the really scary part that I have no clue about is that there are so many parameters you can choose from for the CollationAttribute, plus the fact that you're calling into some foreign package to perform those comparisons. I have no clue what extra overhead burden there is for making these comparisons, but I'm presuming the answer is "at least a little bit". So the simple way is to include a StringHash as Vlong (NOT as VULong) column in the table beside your VarChar( N ) column in which you store a hash encoding. RealBasic now provides a very fast has for objects (essentially the address I'm presuming) and strings (position-dependent shift and Xor of string chars it appears) as well as all other types. You simply store the value (or cast it if it is an instantiated Object) in a variant and request theVariant.hash to get RB's hash value. When you store a string the VarChar field of a record, you also store the hash of the string in the VLong hash field. To perform a fast lookup on any string, you then hash the string, do a FindSingle call on the hashfield. If it finds none, you know it's not in the table. If it finds one then you still have to compare the strings to ensure you don't have a hash collision. If it matches, you've found it. In the exceedingly rare cases that it does not match, then you need to do a second Find search to obtain an array of all hash matches, and if there is more than one, compare full strings against all except the one you alredy checked. The end result is that you essentially do a binary search on a non-unique, indexed VULong field instead of a binary search on full strings, plus a typical worst case of a single full string comparison to verify that the match you found is indeed the actual string. I haven't done any metering yet to be able to tell you how much faster this is but I'm guessing the answer is, "quite a bit" -- especially if you are writing an application such as mine where you are continually doing thousands of string lookups. So this is how I use Valentina to do exact searches for strings. If I haven't bored you to tears with this then you can continue reading my next wish-list email in which I describe how I'd like to do inexact searches for strings with genetic encodings rather than hashes... From Ed at Kleban.com Sun Nov 27 21:08:48 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 27 21:09:12 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/26/05 1:31 AM, "Ruslan Zasukhin" wrote: > On 11/26/05 5:35 AM, "Ed Kleban" wrote: > >> Either way, I'm sure glad I'm doing my searches on string hashes rather than >> actual strings. > > Hashing can work only for search fld = "xxx" > > But you cannot do range searches using hashing, right ? Correct. But now I'd like elaborate on an earlier note I made about a feature I'd like to see: Of all the features on my wish-list for future Valentina versions, top of the list would be some VField.FindMasked functions; perhaps something like: VField.FindMasked( inMask as Integer, inKeep as Integer, inSelection as VSet ) as VBitSet VField.FindMaskedAsArraySet( inMask as Integer, inKeep as Integer, inSelection as VSet ) as VArraySet VField.FindFirstMasked( inMask as Integer, inKeep as Integer, inSelection as VSet ) as Integer // Returns RecID High-order bits of the inMask argument would be truncated to the size of the VField. The inKeep argument would be one of: EVKeep.includeExactMatches EVKeep.excludeExactMatches EVKeep.includePartialMatches EVKeep.excludePartialMatches The functions would simply perform a BitAnd between the field value and inMask. If the result was equal to inMask it would qualify as an exact match. If the result was non-zero it would qualify as a partial match. This would allow you to do some very sophisticated match processing that could greatly improve the performance of many algorithms. Let's take an example where we want to perform a very rapid search among a collection of arbitrary strings that have been "interned" (stored uniquely) in a column. Perhaps we already have a hash field as well for performing rapid exact-match lookups. But what if we want to rapidly find all words that include the letters "bet" and therefore want to match "Alphabet", "Better", and the phrase "You Betcha" -- all of which are included in the collection. One could certainly use the regular expression matching facility, and eventually come up with a list of candidates. But it would take an awful lot of processing to come up with that list. A vastly faster way would be to include a genetic encoding of every interned string as another field. This would likely be an integer field, probably VULong for optimal flexibility and performance. A simplistic encoding might be to simply lowercase the word and then allocate 26 of the 32 bits in the VULong for each of the characters of the alphabet. Thus when we call FindMasked our inMask value would have bits set for the "b", "e", and "t" characters. The list we get back from a such FindMasked search would have some good candidates like "better" and some bad candidates in the list like "tabbed" which also has "b", "e", and "t" in its genetic encoding. We'd still have to run the result through a regular expression search or examine them in some other manner to cull out the bad candidates, but the list to cull would be a small fraction of the table contents. With a more sophisticated encoding that included aspects of string length, presences of whitespace and usage of punctuatation the list retuned by FindMasked could be an EXCEEDINGLY small fraction. One could even include a second VULong field with a different genetic encoding to perform a second-pass culling using another application of a FindMasked function call. What ya think? From Ed at Kleban.com Sun Nov 27 22:01:19 2005 From: Ed at Kleban.com (Ed Kleban) Date: Sun Nov 27 22:01:44 2005 Subject: Unable to disable indexing Message-ID: Should I be able to disable indexing? Execution of the following code results in an "Indexing still enabled" failure. // Disable indexing while filling the table spanSrcIdField = LTblSrcSpan.fSpanObRef spanSrcIdField.IsIndexed = false spanSrcIdField.IsUnique = false #if DebugBuild then if spanSrcIdField.IsUnique then fail( here, "Unique still set" ) elseif spanSrcIdField.isIndexed then fail( here, "Indexing still enabled" ) end #endif Running V4RB on Mac OS X 10.4.2 Thanks! --Ed From Ed at Kleban.com Mon Nov 28 01:05:29 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:10:33 2005 Subject: Unable to disable indexing In-Reply-To: Message-ID: Interestingly, the problem noted below does not appear to occur when the field is not declared as "fUnique" (and therefore indexed) in the field declaration. I.e after changing the term "fUnique" to "fNone" in the following line, the problem noted below does not occur: fSpanObRef = new VLong( "FSpanObRef" , EVFlag.fUnique ) I'm not sure yet whether having enabled indexing dynamically as opposed to at instantiation time, whether it is any more ammenable to being disabled. Interestingly however, the speed improvement of disabling indexing is surprisingly slight, shaving 7 seconds off of 61 seconds for building a table of 100,533 in-RAM records. I'm not sure whether that's bad because the savings is so slight, or good because it means management of the indices is so efficient -- but I think it's the later. The fact that it takes on the order of a minute to do this however is a big concern. I'll spend some more time tomorrow figuring out where all the time is being spent. It may make more sense to just build this as a table with a single indexed column rather than as a table having this as one column out of five. But the good news is I've finally got Valentina up and running, er uh, well up and limping at least, and have some empirical results to start wading around in. On 11/27/05 10:01 PM, "Ed Kleban" wrote: > Should I be able to disable indexing? > > Execution of the following code results in an "Indexing still enabled" > failure. > > > // Disable indexing while filling the table > spanSrcIdField = LTblSrcSpan.fSpanObRef > spanSrcIdField.IsIndexed = false > spanSrcIdField.IsUnique = false > > #if DebugBuild then > if spanSrcIdField.IsUnique then > fail( here, "Unique still set" ) > elseif spanSrcIdField.isIndexed then > fail( here, "Indexing still enabled" ) > end > #endif > > Running V4RB on Mac OS X 10.4.2 > > Thanks! > --Ed > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From Ed at Kleban.com Mon Nov 28 01:35:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:10:39 2005 Subject: Creating new fields in classes way Re: Telling tables what their fields are In-Reply-To: Message-ID: Is there any reason I can't use methods such as CreateShortField or CreateVarCharField to add additional fields after a table has been created when using the Classes way? I.e. instead of including the following line in the Table constructor... fSpanLen = new VULong( "FSpanLen" ) ... simply create only some fields in the constructor and leave some field variables such as fSpanLen equal to nil... then much later, when needed, come along and call... fSpanLen = myTable.CreateULongField( "FSpanLen" ) ----- If I do this, then presumably when fSpanLen is created it will have the same number of records as all other fields of myTable, and the fSpanLen field value for each row will be initialzed to 0 (SetBlank) when the column is created, correct? On 11/19/05 9:00 AM, "Ed Kleban" wrote: > > On 11/19/05 8:43 AM, "Ruslan Zasukhin" wrote: > >> On 11/19/05 3:22 PM, "Ed Kleban" wrote: >> >>>> Tools as Valentina Studio and viSQL cannot use Class Way. NEVER. >>>> because they do not know structure of any db they going to open >>> >>> Yes, but I want to "eat my cake and have it too" as we say here. I want to >>> use classes way to define my databases, and then be able to inspect them on >>> disk with VS :) >> >> You can this. No problems here. > > Ah great! Then I'm a happy camper. :) > >> I think you have missunderstnad. >> >> you can develop YOUR application with Class way. >> viSQL was developed with API way >> > Yes, I did misunderstand. Thanks for the clarification. I see what you > meant now after re-reading your comment in a different context. > > Thanks! > --Ed > From cbpelto at pcisys.net Mon Nov 28 04:41:20 2005 From: cbpelto at pcisys.net (Chuck Pelto) Date: Wed Nov 30 08:10:52 2005 Subject: Different Ways.... Message-ID: <6A2D1D4B-1CB3-4D0A-B771-82EEAF7EDDCC@pcisys.net> ...different set-ups? I get the distinct impression that if one wants to go a specific way using Valentina, e.g., SQL or API, one has to do the set up of the Valentina database differently. That the SQL way has a different set up than the API. Is this correct? Or can one set up the database in one manner and then access/use it with either SQL or API or RBDB? Regards, Chuck From sunshine at public.kherson.ua Mon Nov 28 13:59:57 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:10:59 2005 Subject: Default maximal length for VarChar Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 3:57 AM, "Ed Kleban" wrote: >> Actually in v2 for UTF8 the best size of VarChar will be 2044. >> > > Why? Does this have to do with the natural page size of MacOS X or > something? Yes. Best size of page is 4K We assume that page should keep at least 2 records. So max size *in bytes* for one record is (4K - header) / 2 = 2044 > I understood the logic described in the Kernel manual which makes good > sense: > > " The default maximal length for VarChar field is 504 bytes. This is the > maximum number that allows us to work with 1,024 byte pages. Indeed: > 8 bytes of header + 2 records * (504 + 4) = 1024 bytes. > You can specify lower values for maximal length (e.g., 20), but 1,024 byte > pages will still be used. The only advantage is that you will truncate > longer strings to 20 bytes. > " Ops, this is old logic !!! It count that page size is 1K. Now it is 4K. > Is this no longer true? Has the change to unicode in V2 made it the case > that the default maximal length has changed? Is the Kernel manual > incorrect? > > If the minimal page size is still 1024 bytes, then wouldn't that mean since > each character now takes up 2 bytes in UTF-16 when stored, instead of one by > as when previously stored in UTF-8, that the default maximal length for a > VarChar field should now be 504 / 2 = 252 bytes? > > I don't have enough detail yet for this to make sense. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 28 14:59:26 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:11:07 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 4:24 AM, "Ed Kleban" wrote: >> But you cannot do range searches using hashing, right ? > > Yes, that is correct. > > You mentioned this also the last time I mentioned using Hashes, and both > time have essentially implied that there is some drawback to not having an > ability to do > or >= or < or <= searches. I suppose that for some > applications that may be true, but it is virtually never a drawback for my > applications. In fact I've never even considered wanting to do such a > thing. > > Question: What applications do you have in mind that range searches on > strings would be useful for? Strings have alphabetical order. So we must not prohibit to developers usage of range searches. But you are right. IT is good idea to add one more option to choose: ala-hashing kind of index. You can add this as request into Mantis > For 99% of my applications I want to be able to: > > 1) See if I already have a given string (or object, or structure, or > whatever) in my table. > > 2) If I do, then find that item, or lack thereof, as fast as possible. > > 3) And then typically add the item into the table if it is not already > there. > > I therefore am doing this on a column of say strings for example, which I > always know will be unique. > > For a table of bytes, shorts, or longs, simply performing a binary search by > using a FindSingle on an indexed unique column in Valentina is going to be > pretty darn fast. > > For a table of strings however, it's a whole 'nuther story. First you have > the overhead for comparing strings which is much, much slower than comparing > integers, On top of that as of V2 we now have the reality that strings are > stored in tables in Unicode and are therefore twice the size, -- but which > in reality does not imply twice the time since most matches will fail on a > comparison of the first few characters. But the really scary part that I > have no clue about is that there are so many parameters you can choose from > for the CollationAttribute, plus the fact that you're calling into some > foreign package to perform those comparisons. I have no clue what extra > overhead burden there is for making these comparisons, but I'm presuming the > answer is "at least a little bit". Well, we use IBM library. I believe that IBM guys have did their job perfectly. And we have no other choice. You cannot have OWN string compare method for each natural language. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 28 15:06:25 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:11:13 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 5:08 AM, "Ed Kleban" wrote: >>> Either way, I'm sure glad I'm doing my searches on string hashes rather than >>> actual strings. >> >> Hashing can work only for search fld = "xxx" >> >> But you cannot do range searches using hashing, right ? > > Correct. But now I'd like elaborate on an earlier note I made about a > feature I'd like to see: > > Of all the features on my wish-list for future Valentina versions, top of > the list would be some VField.FindMasked functions; perhaps something like: > > VField.FindMasked( > inMask as Integer, inKeep as Integer, inSelection as VSet ) > as VBitSet > > VField.FindMaskedAsArraySet( > inMask as Integer, inKeep as Integer, inSelection as VSet ) > as VArraySet > > VField.FindFirstMasked( > inMask as Integer, inKeep as Integer, inSelection as VSet ) > as Integer // Returns RecID > > High-order bits of the inMask argument would be truncated to the size of the > VField. The inKeep argument would be one of: > > EVKeep.includeExactMatches > EVKeep.excludeExactMatches > EVKeep.includePartialMatches > EVKeep.excludePartialMatches > > The functions would simply perform a BitAnd between the field value and > inMask. If the result was equal to inMask it would qualify as an exact > match. If the result was non-zero it would qualify as a partial match. Well, I believe you can get this right now, Ed in 2 ways: 1) using SQL and methods 2) using calculated fields and methods. Valentina support a lots of functions. And I believe it support bit And. So you can do something as: SELECT .. FROM T WHERE fld & mask Or you can make Table Mathod -- calcualted field: fldM = t.CreateBooleanField( "name", "fld & mask" ) You can even index it. And you can change formula on the fly. And you can have many calculated fields for table... Check docs on this feature -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Mon Nov 28 15:08:35 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:11:18 2005 Subject: Unable to disable indexing In-Reply-To: Message-ID: On 11/28/05 6:01 AM, "Ed Kleban" wrote: Hi Ed, > Should I be able to disable indexing? Yes of course. > Execution of the following code results in an "Indexing still enabled" > failure. > // Disable indexing while filling the table > spanSrcIdField = LTblSrcSpan.fSpanObRef > spanSrcIdField.IsIndexed = false > spanSrcIdField.IsUnique = false > #if DebugBuild then > if spanSrcIdField.IsUnique then > fail( here, "Unique still set" ) > elseif spanSrcIdField.isIndexed then > fail( here, "Indexing still enabled" ) > end > #endif Try change order > spanSrcIdField.IsUnique = false > spanSrcIdField.IsIndexed = false Or better just remove line with Unique flag > spanSrcIdField.IsIndexed = false Valentina 2 allow this. And later when you set indexing ON, Valentina will just use unique flag. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Mon Nov 28 11:43:04 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:11:23 2005 Subject: LastInsertedRecID Message-ID: The V4RB reference manual mentions a "Table method" named LastInsertedRecID which the manual mentions is "r/o" on page 43, gives an example for, and notes "returns a RecID". But the description offers no clue as to which table this record was added to. Is it simply the last table to which a record was added? If so wouldn't it be useful to have a function that also returned this information, perhaps LastTableInsertedInto? Or is the RecID here a 64-bit value encoding both the table and record Id? From Ed at Kleban.com Mon Nov 28 11:58:10 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:11:27 2005 Subject: SetBlank with inAccess = EvValueAccess.forAdd Message-ID: I'm having a bit of difficulty understanding the following paragraph under "SetBlank": " Parameter inAccess can be used to speed up SetBlank() if you add records. In this case you can specify its value forAdd, so Valentina will not save copies of previouse field values. In the same time you can always use the default value forUpdate and everyhting will work correctly. " Does this mean that when you use SetBlank with forUpdate instead of with forAdd, that it always writes the current value of every field in the buffer to the current record before setting all fields to blank values? Or is it clever enough to only write out field values that have changed? I would think that unless the user issues an explicit "Table.UpdateRecord()" call that they should not expect modified field values to be transferred to the table record. Or is the default value of forUpdate intended to be a "convenience" so that users don't have to remember to call "Table.UpdateRecord()"? Somehow I think I'm missing something here. Thanks! --Ed From maxprog at mac.com Tue Nov 29 13:12:13 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 08:11:56 2005 Subject: [V4RB] Query returning unmatched records Message-ID: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> Hi, I have the following query: SELECT SUM(myField) FROM myTable WHERE ( myDate BETWEEN '1/1/2001' AND '1/31/2001' ) AND aMisc1 = false AND aMisc2 = false It returns records however they are not matching the query, actually it never returns what it should. Is there anything I should know? I can't see why that query doesn't work right now. TIA, ~/Stan From Ed at Kleban.com Tue Nov 29 08:36:27 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:12:04 2005 Subject: Performance metrics - some early conclusions Message-ID: I was getting rather pissed that that Valentina appeared to be much slower that my intuition led me to believe would be the case. It shouldn't take 20 times longer to add a 2-field entry into a table record for every node in an XML Parse tree than the time required to build the parse tree from XML in the first place. Then I noticed that the disk was thrashing for what was supposed to be an all in-memory table. I eventually figured out the problem, tweaked things up, and have come to the following VERY preliminary conclusions about Valentina performance under RB 2005: ========= Generating a 400 MB file by failing to turn of V4RB debug logging may cost you at least factor of 20 in performance degradation. Variant.Hash is not unique for in-memory objects in RB 2004 r5 despite the language reference claim to the contrary. It appears this may be a bug in multiple RB versions going back to 5.x. Adding a record in a table with twice as many fields will likely take about twice as long, as you would expect. Re-indexing or enabling indexing on a non-indexed field is very expensive. Indexing a field after first populating it for all your records can save you time, perhaps 15%. It takes about half as long to store an object hash in a memory buffer field as it does for Variant.hash to calculate the value. Considering that Variant.hash is presumably basing the hash on the address of an object, that's darn impressive. It is indeed possible to list all of the nodes in an XML parse tree in an in-memory table in perhaps 20% less time that it takes to build the parse tree from XML source using Theo's bat-out-of-hell XML Engine. That bodes very well for overall performance. Bottom line: I'm a happy camper :) From Ed at Kleban.com Tue Nov 29 08:44:15 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:12:09 2005 Subject: Table of a VFeld from a VCursor In-Reply-To: Message-ID: On 11/22/05 3:36 PM, "Ruslan Zasukhin" wrote: > On 11/22/05 12:21 AM, "Ed Kleban" wrote: > >> // When you define your VTable subclasses you define the FieldCursors >> // you want to be able to use in addition to table fields: >> >> Sub CTblForms(db as CRbProjectDB, tableName as String) >> // Initialize the receiver and its fields. >> >> self.name = tableName >> >> // Define the Table Fields >> fFrmFlags = new VByte( "FFrmFlags" ) >> fFrmLabel = new VVarChar( "FFrmLabel", 504 ) >> fFrmTagTyp = new VVarChar( "FFrmTagTyp", 504) >> fFrmXmlTag = new VVarChar( "FFrmXmlTag", 504) >> >> // Define the FieldCursors >> cursorOnFFrmXmlTag = new CFCVarChar( fFrmXmlTag ) >> cursorOnFFrmFlags = new CFCByte( fFrmFlags ) >> End Sub >> >> >> // Subsequently, in your code you can use the FieldCursors >> // to randomly access a single field of a given record without >> // reading in every single table field when all you need is >> // a single field's value: >> >> flagByteFromRecord32 = MyTblForms.cursorOnFFrmFlags.integerValue( 32 ) >> MyTblForms.cursorOnFFrmFlags.integerValue( 21 ) = kFlagValForRec21 > > Okay, I see idea of hacking :-) > > Only problem you can meet is record locks. > May be you will prefer use kNoLocks. Yes, if you look in the updateFieldCursor method below, I do indeed use kNoLocks. > > >> // To make this possible... >> >> // You need to define a subclass of CFCField for each different >> // kind of Vfield subclass that you want to be able create a >> // FieldCursor for. This gets done once and reused in all the >> // applications where you want to use FieldCursors. It is essentially >> // the common FieldCursor add-in code. Here are some example guts >> // out of CFCByte: >> >> Function integerValue(recId as Integer, doUpdateCheck = true) As Integer >> // Return an Integer with the value of the reciever's fcCursorField at >> // recId from the updated fcCursor on fcTable in fcDatabase. >> // If doUpdateCheck is false then assume that no rows have been >> // added to fcTable and that the fcCursor is valid. >> >> if doUpdateCheck and updateFieldCursor then // fcCursorChanged >> fcFCByte = fcCursor.ByteField( 1 ) >> end >> >> fcCursor.position = recId >> return fcFCByte.Value >> End Function >> >> >> Function integerValue(recId as Integer, newIntVal as Integer, _ >> doUpdateCheck = true, assigns newValStr as string) As String >> >> // Change the value of the reciever's fcCursorField at recId >> // from the updated fcCursor on fcTable in fcDatabase to newIntVal. >> // If doUpdateCheck is false then assume that no rows have been >> // added to fcTable and that the fcCursor is valid. >> >> if doUpdateCheck and updateFieldCursor then // fcCursorChanged >> fcFCByte = fcCursor.ByteField( 1 ) >> end >> >> fcCursor.position = recId >> fcFCByte.Value = newIntVal >> fcCursor.UpdateRecord >> End Function >> >> >> // The central magic that makes all this work transparently so you >> // don't have worry about the state of the FieldCursor is the >> // updateFieldCursor method, and the common code called from the >> // constructors for all CFCField subclasses: >> >> Protected Function updateFieldCursor() As Boolean >> // Update the receiver's cursor from it's field if necessary. >> // Return true if the Cursor was updated or false if the >> // former fcCursor is fine to use. >> >> const here = "updateFieldCursor" >> >> if fcCursor is nil or fcCursor.RecordCount <> fcTable.RecordCount then >> fcCursor = fcDatabase.SqlSelect( _ >> fcSqlString, EVCursorLocation.kClientSide, _ >> EVLockType.kNoLocks, EVCursorDirection.kRandom ) > > KClientSide -- give you READ ONLY cursor. Better use ServerSide > > >> #if DebugBuild then >> if fcCursor.RecordCount <> fcTable.RecordCount then >> fail( here, "Update error" ) >> end if >> #endif >> >> return true // Cursor was modified >> else >> return false // Cursor is fine as-is. >> end >> End Function >> >> >> Sub CFCField(aVField as VField) >> // Initialize the receiver to be a cursor on aVField >> >> const here = "CFCField" >> >> #if DebugBuild then >> if aVField is nil or aVField.Table is nil _ >> or aVField.Table.Database is nil then >> fail( here, "Implementation Error" ) >> end >> #endif >> >> if not self.checkFieldType( aVField ) then >> fail( here, "Invalid VField subclass" ) >> end >> >> // Cache useful values in properties >> fcTable = aVField.Table >> fcSqlString = glue( "Select ", aVField.Name, " from ", fcTable.Name ) >> fcDatabase = aVField.Table.Database >> // There is no need or reason to cache the argument aVField. >> End Sub From fb at memedia.de Tue Nov 29 20:33:22 2005 From: fb at memedia.de (Florian Bogeschdorfer) Date: Wed Nov 30 08:12:21 2005 Subject: SQL question In-Reply-To: Message-ID: <0MKwpI-1EhBE83vBd-0006zo@mrelayeu.kundenserver.de> Hi all! Say I have two tables: TABLE CUSTOMER with fields ID,NAME,ADRESS And TABLE PAYBACKCARD with fields CUSTOMER_ID,ID,CARDNUMBER If I search for customers with cards, I do "SELECT * FROM CUSTOMER,PAYBACKCARD WHERE CUSTOMER.ID=PAYBACKCARD.CUSTOMER_ID" Fine. But now I want to find all customers WITHOUT card. How can I do this? In this case there is no link between the two tables, I need "SELECT * FROM CUSTOMER WHERE there is no link between the tables..." TIA Florian From rbarber at yhb.att.ne.jp Wed Nov 30 15:09:24 2005 From: rbarber at yhb.att.ne.jp (Ron) Date: Wed Nov 30 08:12:31 2005 Subject: list problems In-Reply-To: References: Message-ID: <35caf7c121c439731f844bde4b45baf6@yhb.att.ne.jp> Hi Ruslan I haven't received anything for several days on this list or the beta list, including your crosspost to the Rev list. I did get your post to the Rev list but not on this one. I requested but did not receive a password reminder either so I'm wondering if their is a problem with the list or my account or? Thanks Ron From Ed at Kleban.com Mon Nov 28 12:08:02 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:12:52 2005 Subject: Fast Allocate Message-ID: Is there a faster way to allocate, say 1000 blank or common-value records all at once rather than simply calling SetBlank or filling up the memory buffer and then calling Table.AddRecord in a loop 1000 times? From Ed at Kleban.com Mon Nov 28 12:13:14 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:13:01 2005 Subject: VCursor.AddRecord Message-ID: Since: VCursor.DropRecord removes a record from the cursor, but not from the tables the info was extracted from, whereas: VCursor.DeleteRecord removes a record both from the cursor and the original table, and VCursor.UpdateRecord writes values through the cursor back into the original table, ...then I'm presuming that... Vcursor.AddRecord() will add a new record to BOTH the cursor as well as to the original table, correct? And what is done for fields in the original table that are not included in the cursor? Presumably they are set to their blank value equivalents? Thanks! --Ed From joergp at three-2-one.com Wed Nov 30 15:17:58 2005 From: joergp at three-2-one.com (Joerg Pressel) Date: Wed Nov 30 08:18:24 2005 Subject: [OFF] Looking for help porting to V4RB 2.x Message-ID: <6815BD04-F1AE-45C1-BD46-FC26E30F6142@three-2-one.com> Hi all, my app CoverBuddy (http://www.coverbuddy.com) is written in RB 5.5 and uses a Valentina version 1.x database to store information about music files in UTF8. My short-term plan for CoverBuddy is creating a windows version ? this would be a good opportunity to update the Valentina database to version 2 ... Real problem here is my extremely limited time ? so I'm looking for somebody who could help me porting the database. Budget is limited, but of course this had to be paid. Anybody experienced in V4RB 2.x and RB 5.5 and interested in helping out? Please contact me off-list ... Thanks, -- J?rg ____________________________________________ three-2-one interaktive Medien GmbH joerg.pressel@three-2-one.com http://www.three-2-one.com fon: +49 2151 31945-11 From IvanSmahin at public.kherson.ua Wed Nov 30 16:29:42 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Wed Nov 30 08:30:05 2005 Subject: [V4RB] Query returning unmatched records In-Reply-To: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> References: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> Message-ID: <1462311288.20051130162942@public.kherson.ua> Hello Stan, Tuesday, November 29, 2005, 2:12:13 PM, you wrote: SB> Hi, SB> I have the following query: SB> SELECT SUM(myField) SB> FROM myTable SB> WHERE ( myDate BETWEEN '1/1/2001' AND '1/31/2001' ) SB> AND aMisc1 = false SB> AND aMisc2 = false SB> It returns records however they are not matching the query, actually SB> it never returns what it should. SB> Is there anything I should know? I can't see why that query doesn't SB> work right now. I guess myDate is VDate field? You should use the date-literals instead of string one. ... WHERE ( myDate BETWEEN date'1/1/2001' AND date'1/31/2001' ) ... -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From maxprog at mac.com Wed Nov 30 15:31:30 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 08:31:53 2005 Subject: [V4RB] Query returning unmatched records // fixed. In-Reply-To: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> References: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> Message-ID: <3B7A6E42-78A1-49E9-A614-89AFFCE2044F@mac.com> I have found the solution, use of kMDY as the dateformat. ~/Stan > Hi, > > I have the following query: > > SELECT SUM(myField) > FROM myTable > WHERE ( myDate BETWEEN '1/1/2001' AND '1/31/2001' ) > AND aMisc1 = false > AND aMisc2 = false > > It returns records however they are not matching the query, > actually it never returns what it should. > Is there anything I should know? I can't see why that query doesn't > work right now. > > TIA, > ~/Stan > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From maxprog at mac.com Wed Nov 30 15:38:18 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 08:38:41 2005 Subject: [OFF] Looking for help porting to V4RB 2.x In-Reply-To: <6815BD04-F1AE-45C1-BD46-FC26E30F6142@three-2-one.com> References: <6815BD04-F1AE-45C1-BD46-FC26E30F6142@three-2-one.com> Message-ID: Hi, I have ported 3 apps last week and made a document with all the changes I made. I used the info from Valentina documentation in part but my document have much more info. Let me know if you want that document. Stan > Hi all, > > my app CoverBuddy (http://www.coverbuddy.com) is written in RB 5.5 > and uses a Valentina version 1.x database to store information > about music files in UTF8. > > My short-term plan for CoverBuddy is creating a windows version ? > this would be a good opportunity to update the Valentina database > to version 2 ... > > Real problem here is my extremely limited time ? so I'm looking for > somebody who could help me porting the database. Budget is limited, > but of course this had to be paid. > > Anybody experienced in V4RB 2.x and RB 5.5 and interested in > helping out? Please contact me off-list ... > > Thanks, > -- > J?rg > > ____________________________________________ > three-2-one interaktive Medien GmbH > > joerg.pressel@three-2-one.com > http://www.three-2-one.com > fon: +49 2151 31945-11 > > > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From maxprog at mac.com Wed Nov 30 15:39:33 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 08:39:57 2005 Subject: [V4RB] Query returning unmatched records In-Reply-To: <1462311288.20051130162942@public.kherson.ua> References: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> <1462311288.20051130162942@public.kherson.ua> Message-ID: <7154A734-C5EC-41EA-B136-7C7E02976410@mac.com> Interesting, I searched the web on how do to that, so: WHERE ( myDate BETWEEN date'1/1/2001' AND date'1/31/2001' ) is correct? If so I will try it. myDate is a DateTime field. Stan > Hello Stan, > > Tuesday, November 29, 2005, 2:12:13 PM, you wrote: > > SB> Hi, > > SB> I have the following query: > > SB> SELECT SUM(myField) > SB> FROM myTable > SB> WHERE ( myDate BETWEEN '1/1/2001' AND '1/31/2001' ) > SB> AND aMisc1 = false > SB> AND aMisc2 = false > > SB> It returns records however they are not matching the query, > actually > SB> it never returns what it should. > SB> Is there anything I should know? I can't see why that query > doesn't > SB> work right now. > > I guess myDate is VDate field? > You should use the date-literals instead of string one. > ... > WHERE ( myDate BETWEEN date'1/1/2001' AND date'1/31/2001' ) > ... > > > -- > Best regards, > Ivan mailto:IvanSmahin@public.kherson.ua > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Wed Nov 30 16:45:29 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:45:51 2005 Subject: [OFF] Looking for help porting to V4RB 2.x In-Reply-To: Message-ID: On 11/30/05 4:38 PM, "Stan Busk" wrote: > Hi, > > I have ported 3 apps last week and made a document with all the > changes I made. I used the info from Valentina documentation in part > but my document have much more info. Let me know if you want that > document. Me too want it :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 16:46:41 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:47:01 2005 Subject: [OFF] Looking for help porting to V4RB 2.x In-Reply-To: Message-ID: On 11/30/05 4:38 PM, "Stan Busk" wrote: > Hi, > > I have ported 3 apps last week and made a document with all the > changes I made. I used the info from Valentina documentation in part > but my document have much more info. Let me know if you want that > document. Or better idea !!! We have setup Valentina WIKI For now it is located here: http://paradigma.ukrcom.kherson.ua/dokuwiki/ So you all can add info right there -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 16:47:28 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:47:51 2005 Subject: [V4RB] Query returning unmatched records In-Reply-To: <7154A734-C5EC-41EA-B136-7C7E02976410@mac.com> Message-ID: On 11/30/05 4:39 PM, "Stan Busk" wrote: > > Interesting, I searched the web on how do to that, so: > > WHERE ( myDate BETWEEN date'1/1/2001' AND date'1/31/2001' ) > > is correct? If so I will try it. myDate is a DateTime field. You ask about syntax of BETWEEN ? Yes it is correct. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From IvanSmahin at public.kherson.ua Wed Nov 30 16:51:06 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Wed Nov 30 08:51:24 2005 Subject: [V4RB] Query returning unmatched records In-Reply-To: <7154A734-C5EC-41EA-B136-7C7E02976410@mac.com> References: <9A14FBE3-C290-4991-94A9-C8FB7BEB8733@mac.com> <1462311288.20051130162942@public.kherson.ua> <7154A734-C5EC-41EA-B136-7C7E02976410@mac.com> Message-ID: <761100794.20051130165106@public.kherson.ua> Hello Stan, Wednesday, November 30, 2005, 4:39:33 PM, you wrote: SB> Interesting, I searched the web on how do to that, so: SB> WHERE ( myDate BETWEEN date'1/1/2001' AND date'1/31/2001' ) SB> is correct? If so I will try it. myDate is a DateTime field. SB> Stan You still should use the dates as defined by current dateFormat. Date/DateTime/Time literals just gives a way to remove some internal ambiguities. BTW. Using such literals is required by SQL-92. -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From sunshine at public.kherson.ua Wed Nov 30 16:52:03 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:52:24 2005 Subject: Different Ways.... In-Reply-To: <6A2D1D4B-1CB3-4D0A-B771-82EEAF7EDDCC@pcisys.net> Message-ID: On 11/28/05 1:41 PM, "Chuck Pelto" wrote: Hi Chuck, > ...different set-ups? > > I get the distinct impression that if one wants to go a specific way > using Valentina, e.g., SQL or API, one has to do the set up of the > Valentina database differently. That the SQL way has a different set > up than the API. > > Is this correct? NO > Or can one set up the database in one manner and > then access/use it with either SQL or API or RBDB? Yes, the way you access database files -- this is your codding style only. It is NOT stored in database files -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 16:54:47 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 08:55:13 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/29/05 4:36 PM, "Ed Kleban" wrote: > Variant.Hash is not unique for in-memory objects in RB 2004 r5 despite the > language reference claim to the contrary. It appears this may be a bug in > multiple RB versions going back to 5.x. Ed, WARNING by big letters!!! Do not use Vfield.Value it is based on VARIANT of REALbasic. it is 10 times slower than usage of type-specific values as Vlong.Value Vshort.Value This is REALbasic problem/bug. At least it was in RB 5.x and it seems first RB 2005. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 08:57:19 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 08:57:44 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 7:06 AM, "Ruslan Zasukhin" wrote: > On 11/28/05 5:08 AM, "Ed Kleban" wrote: > >>>> Either way, I'm sure glad I'm doing my searches on string hashes rather >>>> than >>>> actual strings. >>> >>> Hashing can work only for search fld = "xxx" >>> >>> But you cannot do range searches using hashing, right ? >> >> Correct. But now I'd like elaborate on an earlier note I made about a >> feature I'd like to see: >> >> The functions would simply perform a BitAnd between the field value and >> inMask. If the result was equal to inMask it would qualify as an exact >> match. If the result was non-zero it would qualify as a partial match. > > Well, I believe you can get this right now, Ed in 2 ways: > > 1) using SQL and methods > 2) using calculated fields and methods. > > Valentina support a lots of functions. And I believe it support bit And. > > So you can do something as: > > SELECT .. FROM T WHERE fld & mask > > Or you can make Table Mathod -- calcualted field: > > fldM = t.CreateBooleanField( "name", "fld & mask" ) > > You can even index it. > And you can change formula on the fly. > And you can have many calculated fields for table... > > Check docs on this feature > Well, I'd be thrilled if this were true, but I can't find any info about this anywhere in the Kernel, V4RB Reference, or SQL_2 manuals. Several times in the past I've searched for "bit", which turns up nothing in these documents except "bitset" of course. Searching for "xor" does indeed take me to the "Logical Operators" section of the SQL manual on page VSQL-76, however the Not, And, Xor, and OR operators defined there are all defined in terms of strictly boolean values defined as 0 = false or non-zero = true. I.e. the description for "OR" is: " Returns 1 if any operand is non-zero, otherwise returns 0. " This implies to me that these are not bitwise functions, but treat an entire ULong as a single binary value with 31 wasted bits. I hadn't thought to search for "&", but all that does is lead me to the "&&" operator which is an abbreviation for Logical "AND" defined as: " Returns 1 if all operands are non-zero, otherwise returns 0. " Actually, I'm unable to find nuch info at all in the manuals about calculated fields. There are a dozen references or so desrcibing how certain things work or not in the case of a calculated field, but nowhere have I found a description of usage, available operators, and limitations for making these work. Is there perhaps some other SQL reference work you could refer me to that does? As far as boolean fields go, this is not really a solution. I'm not looking for a way to create 16 separate single-bit boolean fields that I write complex separate logic for in RB methods that operate on returned values or with calculated field methods with long involved code. I'm looking for a fast bitwise operation that can perform 32 parallel bit comparisons with a single "&" operator and move on to check the next record. That make sense? Or am I really missing something here? From ayu8 at cdc.gov Wed Nov 30 09:46:44 2005 From: ayu8 at cdc.gov (Sims, John) Date: Wed Nov 30 09:01:41 2005 Subject: SQL question Message-ID: > -----Original Message----- > From: valentina-bounces@lists.macserve.net > [mailto:valentina-bounces@lists.macserve.net] On Behalf Of > Florian Bogeschdorfer > Sent: Tuesday, November 29, 2005 2:33 PM > To: 'Valentina Developers' > Subject: SQL question > > Hi all! > > Say I have two tables: > > TABLE CUSTOMER with fields ID,NAME,ADRESS > > And > > TABLE PAYBACKCARD with fields CUSTOMER_ID,ID,CARDNUMBER > > If I search for customers with cards, I do "SELECT * FROM > CUSTOMER,PAYBACKCARD WHERE CUSTOMER.ID=PAYBACKCARD.CUSTOMER_ID" > > Fine. But now I want to find all customers WITHOUT card. How > can I do this? > In this case there is no link between the two tables, I need > "SELECT * FROM CUSTOMER WHERE there is no link between the tables..." > > TIA > > Florian > Hi Florian, I'm not 100% sure that Valentina supports this, but the most common way I know of would be... SELECT * FROM Customer WHERE NOT Customer.ID IN (SELECT Customer.ID FROM Customer,PaybackCard WHERE Customer.ID=PaybackCard.Customer_ID) If that doesn't work, look into the EXISTS keyword of SQL. Good luck. -John From Ed at Kleban.com Wed Nov 30 09:08:26 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:08:50 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 6:59 AM, "Ruslan Zasukhin" wrote: > On 11/28/05 4:24 AM, "Ed Kleban" wrote: > >>> But you cannot do range searches using hashing, right ? >> >> Yes, that is correct. >> >> Question: What applications do you have in mind that range searches on >> strings would be useful for? > > Strings have alphabetical order. So we must not prohibit to developers usage > of range searches. Yes, of course. The use of hashing I'm talking about is for a completely different purpose. You certainly still need ordered string comparison, searching, and range operations. I'd never suggest otherwise. > But you are right. IT is good idea to add one more option to choose: > ala-hashing kind of index. > > You can add this as request into Mantis Hmm, I'm not sure what you're suggesting I submit to Mantis. I'm able to do hashed string searches just fine right now by simply storing a parallel hash field in a record that has string, text, or varchar field. FindSingle et. al. do a perfectly fine, fast job of doing lookups on them. Amusingly, the hardest part has been finding a fast c-coded hashing function to use. I was delighted to see that in RB 2005 they had included a fast hashing function that sounded exactly like what I needed. I then discovered that counter to the documentation the hash was not unique for Objects, and hasn't been since 2003. They're going to fix that in the next beta release. I then discovered that , Doh! The hash is case-insenstive, so "text" and "TEXT" have the same hash. For the moment that's fine -- I just get more collisions. I think Bjorn is going to expose the internal hash method used for the Dictionary inside of the Einhugur tools for us to use. But I could certainly submit a request to Mantis for Valentina to provide a hash function -- I'd even be delighted to supply you with the recommended one-line of C code I use for the hash... probably just a 5-bit rotate and Xor for each byte. Is there some other aspect of incorporating hashes into indices somehow that you're recommending I submit to Mantis? > >> For a table of strings however, it's a whole 'nuther story. First you have >> the overhead for comparing strings which is much, much slower than comparing >> integers, On top of that as of V2 we now have the reality that strings are >> stored in tables in Unicode and are therefore twice the size, -- but which >> in reality does not imply twice the time since most matches will fail on a >> comparison of the first few characters. But the really scary part that I >> have no clue about is that there are so many parameters you can choose from >> for the CollationAttribute, plus the fact that you're calling into some >> foreign package to perform those comparisons. I have no clue what extra >> overhead burden there is for making these comparisons, but I'm presuming the >> answer is "at least a little bit". > > Well, we use IBM library. I believe that IBM guys have did their job > perfectly. I'm delighted to hear it. You're approval of their code means a great deal to me. > And we have no other choice. You cannot have OWN string compare method for > each natural language. > Sure. But my whole point is that when you want to find a string fast, you use a hash, not comparisons embedded in some string package -- no matter who wrote it or how well. In fact even the REALbaic "=" operator is one of the worst possible things to use to do string comparisons. This is simply the wrong approach from my perspective. From ayu8 at cdc.gov Wed Nov 30 09:54:04 2005 From: ayu8 at cdc.gov (Sims, John) Date: Wed Nov 30 09:09:18 2005 Subject: Fast Allocate Message-ID: > -----Original Message----- > From: valentina-bounces@lists.macserve.net > [mailto:valentina-bounces@lists.macserve.net] On Behalf Of Ed Kleban > Sent: Monday, November 28, 2005 1:08 PM > To: Valentina Developers > Subject: Fast Allocate > > Is there a faster way to allocate, say 1000 blank or > common-value records all at once rather than simply calling > SetBlank or filling up the memory buffer and then calling > Table.AddRecord in a loop 1000 times? > > Hi Ed, If all of the values of all of the fields for all of the records are the same, I am pretty sure you only need to call SetBlank once. Psuedocode... SetBlank Assign values to fields For I = 1 to 1000 iNewRec = DB.AddRecord() (or Call Db.AddRecord if you prefer) Next I'm sure Ruslan will correct me if I am wrong. All that said, if you are changing just a couple of fields in each record, I think you can still just call SetBlank once. Psuedocode... SetBlank Assign values for "static" fields For I = 1 to 1000 Assign values for "varying" fields iNewRec = DB.AddRecord() Next Good luck. -John From Ed at Kleban.com Wed Nov 30 09:09:47 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:10:10 2005 Subject: Default maximal length for VarChar Re: VChar vs VText In-Reply-To: Message-ID: On 11/28/05 5:59 AM, "Ruslan Zasukhin" wrote: > On 11/28/05 3:57 AM, "Ed Kleban" wrote: > >>> Actually in v2 for UTF8 the best size of VarChar will be 2044. >>> >> >> Why? Does this have to do with the natural page size of MacOS X or >> something? > > Yes. Best size of page is 4K > > We assume that page should keep at least 2 records. > > So max size *in bytes* for one record is (4K - header) / 2 = 2044 > Ok, great. I understand. I'll change my code accordingly. You can change the Kernel doc. ;-) Thanks! --Ed From sunshine at public.kherson.ua Wed Nov 30 17:13:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:14:15 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/29/05 4:36 PM, "Ed Kleban" wrote: > Generating a 400 MB file by failing to turn of V4RB debug logging may cost > you at least factor of 20 in performance degradation. > > Variant.Hash is not unique for in-memory objects in RB 2004 r5 despite the > language reference claim to the contrary. It appears this may be a bug in > multiple RB versions going back to 5.x. > > Adding a record in a table with twice as many fields will likely take about > twice as long, as you would expect. > > Re-indexing or enabling indexing on a non-indexed field is very expensive. > > Indexing a field after first populating it for all your records can save you > time, perhaps 15%. > > It takes about half as long to store an object hash in a memory buffer field > as it does for Variant.hash to calculate the value. Considering that > Variant.hash is presumably basing the hash on the address of an object, > that's darn impressive. > > It is indeed possible to list all of the nodes in an XML parse tree in an > in-memory table in perhaps 20% less time that it takes to build the parse > tree from XML source using Theo's bat-out-of-hell XML Engine. That bodes > very well for overall performance. > > Bottom line: I'm a happy camper :) Do you use Vtable to add records? Have you specify Table.SetBlank( forAdd ) ^^^^^^^ This can give 20% speedup -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:19:04 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:19:28 2005 Subject: SetBlank with inAccess = EvValueAccess.forAdd In-Reply-To: Message-ID: On 11/28/05 7:58 PM, "Ed Kleban" wrote: Hi Ed, > I'm having a bit of difficulty understanding the following paragraph under > "SetBlank": > > " Parameter inAccess can be used to speed up SetBlank() if you add records. > In this case you can specify its value forAdd, so Valentina will not save > copies of previouse field values. In the same time you can always use the > default value forUpdate and everyhting will work correctly. " > Does this mean that when you use SetBlank with forUpdate instead of with > forAdd, that it always writes the current value of every field in the buffer > to the current record before setting all fields to blank values? Yes. So this is overhead. I have idea how to improve this to drop down overhead to zero. If success, then this parameter will be deprecated. For now it exists. > Or is it clever enough to only write out field values that have changed? Yes we put on disk only dirty fields > I would think that unless the user issues an explicit "Table.UpdateRecord()" > call that they should not expect modified field values to be transferred to > the table record. Or is the default value of forUpdate intended to be a > "convenience" so that users don't have to remember to call > "Table.UpdateRecord()"? > > Somehow I think I'm missing something here. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:20:50 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:21:19 2005 Subject: LastInsertedRecID In-Reply-To: Message-ID: On 11/28/05 7:43 PM, "Ed Kleban" wrote: > The V4RB reference manual mentions a "Table method" named LastInsertedRecID > which the manual mentions is "r/o" on page 43, gives an example for, and > notes "returns a RecID". But the description offers no clue as to which > table this record was added to. > > Is it simply the last table to which a record was added? YES > If so wouldn't it be useful to have a function that also returned this > information, perhaps LastTableInsertedInto? May be this is good idea. Just it seems any DB do not have this. Ivan, what you think ? > Or is the RecID here a 64-bit value encoding both the table and record Id? No, it is OID 64 bit value. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 09:21:35 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:21:58 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: Thanks for the tip. I don't think I ever would have done that anyway, but nice to know. I mentioned "Variant.Hash" because the way RB implemented access to their hash function is via the variant class. So to use their hash function to get a 32-bit signed hash from an arbitrary string, number, or object you have to use: dim v as variant dim hash as integer v = anArbitraryNumberStringOrObject hash = v.hash But yeah. I'll definitely use the subclasses as you direct for accessing the value. Thanks! --Ed On 11/30/05 8:54 AM, "Ruslan Zasukhin" wrote: > On 11/29/05 4:36 PM, "Ed Kleban" wrote: > >> Variant.Hash is not unique for in-memory objects in RB 2004 r5 despite the >> language reference claim to the contrary. It appears this may be a bug in >> multiple RB versions going back to 5.x. > > Ed, > > WARNING by big letters!!! > > Do not use Vfield.Value > > it is based on VARIANT of REALbasic. > it is 10 times slower than usage of type-specific values as > > Vlong.Value > Vshort.Value > > This is REALbasic problem/bug. At least it was in RB 5.x and it seems first > RB 2005. > From sunshine at public.kherson.ua Wed Nov 30 17:22:15 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:22:38 2005 Subject: Fast Allocate In-Reply-To: Message-ID: On 11/30/05 4:54 PM, "Sims, John" wrote: >> Is there a faster way to allocate, say 1000 blank or >> common-value records all at once rather than simply calling >> SetBlank or filling up the memory buffer and then calling >> Table.AddRecord in a loop 1000 times? >> >> > > Hi Ed, > > If all of the values of all of the fields for all of the records are the > same, I am pretty sure you only need to call SetBlank once. > > Psuedocode... > > SetBlank > Assign values to fields > For I = 1 to 1000 > iNewRec = DB.AddRecord() (or Call Db.AddRecord if you prefer) > Next RIGHT > I'm sure Ruslan will correct me if I am wrong. All that said, if you > are changing just a couple of fields in each record, I think you can > still just call SetBlank once. > > Psuedocode... > > SetBlank > Assign values for "static" fields > For I = 1 to 1000 > Assign values for "varying" fields > iNewRec = DB.AddRecord() > Next > > Good luck. Also right -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From IvanSmahin at public.kherson.ua Wed Nov 30 17:23:56 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Wed Nov 30 09:24:18 2005 Subject: SQL question In-Reply-To: <0MKwpI-1EhBE83vBd-0006zo@mrelayeu.kundenserver.de> References: <0MKwpI-1EhBE83vBd-0006zo@mrelayeu.kundenserver.de> Message-ID: <561316449.20051130172356@public.kherson.ua> Hello Florian, Tuesday, November 29, 2005, 9:33:22 PM, you wrote: FB> Hi all! FB> Say I have two tables: FB> TABLE CUSTOMER with fields ID,NAME,ADRESS FB> And FB> TABLE PAYBACKCARD with fields CUSTOMER_ID,ID,CARDNUMBER FB> If I search for customers with cards, I do "SELECT * FROM FB> CUSTOMER,PAYBACKCARD WHERE CUSTOMER.ID=PAYBACKCARD.CUSTOMER_ID" FB> Fine. But now I want to find all customers WITHOUT card. How can I do this? FB> In this case there is no link between the two tables, I need "SELECT * FROM FB> CUSTOMER WHERE there is no link between the tables..." There is at least 2 ways. 1. SELECT * FROM t1 WHERE t1.id NOT IN ( SELECT RecID FROM T1,T2 ...) 2. SELECT * FROM t1 EXCEPT SELECT RecID FROM T1,T2 ... Also. You can consider to have a link between tables. So you will get a lot of link-specific opportunities. 1. SELECT * FROM t1 WHERE COUNT_LINKED(t1, LNK_t1_t2) = 0 2. SELECT * FROM t1 WHERE t1->CUSTOMER_ID is NULL ... -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From IvanSmahin at public.kherson.ua Wed Nov 30 17:31:17 2005 From: IvanSmahin at public.kherson.ua (Ivan Smahin) Date: Wed Nov 30 09:31:35 2005 Subject: LastInsertedRecID In-Reply-To: References: Message-ID: <17710304523.20051130173117@public.kherson.ua> Hello Ruslan, Wednesday, November 30, 2005, 5:20:50 PM, you wrote: RZ> On 11/28/05 7:43 PM, "Ed Kleban" wrote: >> The V4RB reference manual mentions a "Table method" named LastInsertedRecID >> which the manual mentions is "r/o" on page 43, gives an example for, and >> notes "returns a RecID". But the description offers no clue as to which >> table this record was added to. >> >> Is it simply the last table to which a record was added? RZ> YES >> If so wouldn't it be useful to have a function that also returned this >> information, perhaps LastTableInsertedInto? RZ> May be this is good idea. RZ> Just it seems any DB do not have this. RZ> Ivan, what you think ? It could be done. Anyway please put it as feature request to the Mantis. -- Best regards, Ivan mailto:IvanSmahin@public.kherson.ua From Ed at Kleban.com Wed Nov 30 09:33:25 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:33:48 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 9:13 AM, "Ruslan Zasukhin" < > Do you use Vtable to add records? > Do you mean as opposed to adding them through a cursor? Yes, always. I use: TblSomeTable.AddRecord after populating the memory buffer fields. Why? > Have you specify > Table.SetBlank( forAdd ) > ^^^^^^^ > This can give 20% speedup > Yes, I recently went through and put in "forAdd" in all my SetBlank calls. But since I typically do a SetBlank just once before entering a loop to set memory buffer fields and do an AddRecord for hundreds of records, It doesn't really have much affect. I don't fully understand what forAdd does however. I presume that it prevents the SetBlank method from writing the memory buffer values back to the current table record, but: 1) Why should SetBlank do this at all? Should the user just use UpdateRecord if he wants that done? 2) If no memory fields have been modified does use of forAdd provide any savings? 3) If no records have yet been allocated does use of forAdd provide any savings? From Ed at Kleban.com Wed Nov 30 09:36:43 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:37:07 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 9:13 AM, "Ruslan Zasukhin" wrote: > Do you use Vtable to add records? > > Have you specify > Table.SetBlank( forAdd ) > ^^^^^^^ > > This can give 20% speedup > Also I'm still curious about: Is there a faster way to allocate, say 1000 blank or common-value records all at once rather than simply calling SetBlank or filling up the memory buffer and then calling Table.AddRecord in a loop 1000 times? From sunshine at public.kherson.ua Wed Nov 30 17:37:15 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:37:38 2005 Subject: SQL question -- TIP on Valentina SQL In-Reply-To: <561316449.20051130172356@public.kherson.ua> Message-ID: On 11/30/05 5:23 PM, "Ivan Smahin" wrote: > Hello Florian, > > Tuesday, November 29, 2005, 9:33:22 PM, you wrote: > >> Hi all! >> >> Say I have two tables: >> >> TABLE CUSTOMER with fields ID,NAME,ADRESS >> >> And >> >> TABLE PAYBACKCARD with fields CUSTOMER_ID,ID,CARDNUMBER >> >> If I search for customers with cards, I do "SELECT * FROM >> CUSTOMER,PAYBACKCARD WHERE CUSTOMER.ID=PAYBACKCARD.CUSTOMER_ID" >> >> Fine. But now I want to find all customers WITHOUT card. How can I do >> this? >> In this case there is no link between the two tables, I need >> "SELECT * FROM CUSTOMER WHERE there is no link between the tables..." > There is at least 2 ways. > > 1. SELECT * FROM t1 WHERE t1.id NOT IN ( SELECT RecID FROM T1,T2 ...) > > 2. SELECT * FROM t1 > EXCEPT > SELECT RecID FROM T1,T2 ... > > Also. You can consider to have a link between tables. > So you will get a lot of link-specific opportunities. Florian, It is not clear from your letter if you have FOREIGN KEY for this table. IF you have it, then you have LINK from Valentina point of view. And then wow, you can use MUCH MORE effective SQL than above 2 standard ways: > 1. SELECT * FROM t1 WHERE COUNT_LINKED(t1, LNK_t1_t2) = 0 Right. You can see that in this case Valentina do not need do JOIN and other hard operations. Valentina will just use link. This is much faster. > 2. SELECT * FROM t1 WHERE t1->CUSTOMER_ID is NULL Ivan, this query will require jumps to second table, so it is not so effective. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:41:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:43:04 2005 Subject: VCursor.AddRecord In-Reply-To: Message-ID: On 11/28/05 8:13 PM, "Ed Kleban" wrote: > Since: > > VCursor.DropRecord > removes a record from the cursor, but not from the tables the info was > extracted from, whereas: > > VCursor.DeleteRecord > removes a record both from the cursor and the original table, and > > VCursor.UpdateRecord > writes values through the cursor back into the original table, > > ...then I'm presuming that... > > Vcursor.AddRecord() > will add a new record to BOTH the cursor as well as to the original table, > correct? Yes. Cursor must be ReadOnly = false In fact, such cursor do not have own records. It work over original > And what is done for fields in the original table that are not included in > the cursor? Presumably they are set to their blank value equivalents? BLANK -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 09:42:53 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 09:43:16 2005 Subject: Fast Allocate In-Reply-To: Message-ID: On 11/30/05 9:22 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 4:54 PM, "Sims, John" wrote: > >>> Is there a faster way to allocate, say 1000 blank or >>> common-value records all at once rather than simply calling >>> SetBlank or filling up the memory buffer and then calling >>> Table.AddRecord in a loop 1000 times? >>> >>> >> >> Hi Ed, >> >> If all of the values of all of the fields for all of the records are the >> same, I am pretty sure you only need to call SetBlank once. >> > > RIGHT > >> I'm sure Ruslan will correct me if I am wrong. All that said, if you >> are changing just a couple of fields in each record, I think you can >> still just call SetBlank once. >> > Also right Yes. I presumed that was the case. What I'm curous about however is whether there is a faster way to allocate a large number of table records all at once having blank or common values without having to have a loop that calls AddRecord. In essence I'm looking for the equivalent of the RB "Dim" statement which allows me to say: dim bigTable( 30 ) as Integer rather than: dim bigTable() as Integer for I as integer = 1 to 30 bigTable.append blankValue next From sunshine at public.kherson.ua Wed Nov 30 17:45:23 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:48:11 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 4:57 PM, "Ed Kleban" wrote: > Well, I'd be thrilled if this were true, but I can't find any info about > this anywhere in the Kernel, V4RB Reference, or SQL_2 manuals. > > Several times in the past I've searched for "bit", which turns up nothing in > these documents except "bitset" of course. Searching for "xor" does indeed > take me to the "Logical Operators" section of the SQL manual on page > VSQL-76, however the Not, And, Xor, and OR operators defined there are all > defined in terms of strictly boolean values defined as 0 = false or non-zero > = true. I.e. the description for "OR" is: " Returns 1 if any operand is > non-zero, otherwise returns 0. " This implies to me that these are not > bitwise functions, but treat an entire ULong as a single binary value with > 31 wasted bits. > > I hadn't thought to search for "&", but all that does is lead me to the "&&" > operator which is an abbreviation for Logical "AND" defined as: " Returns 1 > if all operands are non-zero, otherwise returns 0. " > > Actually, I'm unable to find nuch info at all in the manuals about > calculated fields. There are a dozen references or so desrcibing how > certain things work or not in the case of a calculated field, but nowhere > have I found a description of usage, available operators, and limitations > for making these work. Is there perhaps some other SQL reference work you > could refer me to that does? All available methods and operators are descibed in the ValentinaSQL.pdf Check it sin, cos, .. concat, ... Trim ... > As far as boolean fields go, this is not really a solution. I'm not looking > for a way to create 16 separate single-bit boolean fields that I write > complex separate logic for in RB methods that operate on returned values or > with calculated field methods with long involved code. I'm looking for a > fast bitwise operation that can perform 32 parallel bit comparisons with a > single "&" operator and move on to check the next record. > > That make sense? Or am I really missing something here? You should not create 16 calculated fields. You can change formula on the fly. If this field is not indexed, than change is ZERO time. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:47:09 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:50:20 2005 Subject: Fast Allocate In-Reply-To: Message-ID: On 11/30/05 5:42 PM, "Ed Kleban" wrote: > > Yes. I presumed that was the case. What I'm curous about however is whether > there is a faster way to allocate a large number of table records all at > once having blank or common values without having to have a loop that calls > AddRecord. NO. And no sense. Because methods as AddRecord -- are in fact LOW LEVEL methods. You cannot speed up nothing here. > In essence I'm looking for the equivalent of the RB "Dim" statement which > allows me to say: > > > dim bigTable( 30 ) as Integer > > rather than: > > dim bigTable() as Integer > for I as integer = 1 to 30 > bigTable.append blankValue > next DBs do not have such things. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:48:26 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:50:28 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 5:33 PM, "Ed Kleban" wrote: >> Have you specify >> Table.SetBlank( forAdd ) >> ^^^^^^^ >> This can give 20% speedup >> > > Yes, I recently went through and put in "forAdd" in all my SetBlank calls. > But since I typically do a SetBlank just once before entering a loop to set > memory buffer fields and do an AddRecord for hundreds of records, It doesn't > really have much affect. In fact this is wrong. It can kick you IF you have BLOB fields. For BLOB fields, it is VERY important that you SetBlank() on each iteration. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:50:57 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:51:18 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 5:33 PM, "Ed Kleban" wrote: > I don't fully understand what forAdd does however. I presume that it > prevents the SetBlank method from writing the memory buffer values back to > the current table record, but: > 1) Why should SetBlank do this at all? Should the user just use > UpdateRecord if he wants that done? I can wish t.SetBlank t.UpdateRecord() So on default SetBlank() MUST make copies of old values. > 2) If no memory fields have been modified does use of forAdd provide any > savings? SetBlank() itslef ALWAYS copy all and each old values BEFORE you start write new So answer is: number of dirty fields DOES NOT affect > 3) If no records have yet been allocated does use of forAdd provide any > savings? But you have some current record... -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:52:34 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:52:55 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 5:08 PM, "Ed Kleban" wrote: >> You can add this as request into Mantis > > Hmm, I'm not sure what you're suggesting I submit to Mantis. I'm able to do > hashed string searches just fine right now by simply storing a parallel hash > field in a record that has string, text, or varchar field. FindSingle et. > al. do a perfectly fine, fast job of doing lookups on them. But you do that in RB code. Valentina C++ code will be faster. And this will be feature of engine, so all benefit -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 17:56:14 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 09:56:38 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 5:08 PM, "Ed Kleban" wrote: > But my whole point is that when you want to find a string fast, you use a > hash, not comparisons embedded in some string package -- no matter who wrote > it or how well. In fact even the REALbaic "=" operator is one of the worst > possible things to use to do string comparisons. This is simply the wrong > approach from my perspective. I think it is good idea add Hash Index for STRING/VarChar fields which are marked as Primary Key. Just IMHO this is bad design make keys on strings. :-) -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 10:04:02 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:04:23 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 9:45 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 4:57 PM, "Ed Kleban" wrote: > > All available methods and operators are descibed in the ValentinaSQL.pdf > Check it > > sin, cos, .. > concat, ... Trim ... Yep, sure enough. I was just not looking at this section before in the right context. But note that NOT, AND, XOR etc are listed there as "Logical Operators of SQL [that] can return TRUE, FALSE or NULL (unknown) values." There do not appear to be any bitwise logic functions available at all. Thus my request that you add the functions noted in my email for use as additional Find functions to make this possible. > >> As far as boolean fields go, this is not really a solution. I'm not looking >> for a way to create 16 separate single-bit boolean fields that I write >> complex separate logic for in RB methods that operate on returned values or >> with calculated field methods with long involved code. I'm looking for a >> fast bitwise operation that can perform 32 parallel bit comparisons with a >> single "&" operator and move on to check the next record. >> >> That make sense? Or am I really missing something here? > > You should not create 16 calculated fields. > > You can change formula on the fly. > If this field is not indexed, than change is ZERO time. > Uh.. ok sure. But I get the feeling that either you're missing my point, or I'm missing yours. Is there currently a way to do a search that uses a multi-bit field such as a short or a long as a Mask with logical operations? From what I've seen the answer is "no", and the fact that you can have calculated boolean fields doesn't really offer an alternative that's either viable or valuable from my perspective. From Ed at Kleban.com Wed Nov 30 10:05:43 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:06:05 2005 Subject: VCursor.AddRecord In-Reply-To: Message-ID: On 11/30/05 9:41 AM, "Ruslan Zasukhin" wrote: > On 11/28/05 8:13 PM, "Ed Kleban" wrote: > >> ...then I'm presuming that... >> >> Vcursor.AddRecord() >> will add a new record to BOTH the cursor as well as to the original table, >> correct? > > Yes. > > Cursor must be ReadOnly = false > > In fact, such cursor do not have own records. It work over original > Ah, excellent. Very good to hear. >> And what is done for fields in the original table that are not included in >> the cursor? Presumably they are set to their blank value equivalents? > > BLANK Thanks! From Ed at Kleban.com Wed Nov 30 10:14:08 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:14:31 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 9:50 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 5:33 PM, "Ed Kleban" wrote: > >> I don't fully understand what forAdd does however. I presume that it >> prevents the SetBlank method from writing the memory buffer values back to >> the current table record, but: > >> 1) Why should SetBlank do this at all? Should the user just use >> UpdateRecord if he wants that done? > > I can wish > > t.SetBlank > t.UpdateRecord() > > > So on default SetBlank() MUST make copies of old values. Sorry. I don't get it. Why must SetBlank make copies of old values? It copies what values from where and places them where? >> 2) If no memory fields have been modified does use of forAdd provide any >> savings? > > SetBlank() itslef ALWAYS copy all and each old values BEFORE you start write > new > > So answer is: number of dirty fields DOES NOT affect There is no need to copy the values of the current record in the table to the memory buffer, because SetBlank is going to immediately overwrite every memory buffer field with the blank value. There should be no need to copy the values of the in memory buffer fields to the current table record because the user should have used "UpdateRecord", and you're saying it is not affected by the number of dirty fields and (below) that this even happens when there are no allocated records in the table. So I don't get it. What's being copied from where to where and why? >> 3) If no records have yet been allocated does use of forAdd provide any >> savings? > > But you have some current record... From Ed at Kleban.com Wed Nov 30 10:17:05 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:17:32 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 9:52 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 5:08 PM, "Ed Kleban" wrote: > >>> You can add this as request into Mantis >> >> Hmm, I'm not sure what you're suggesting I submit to Mantis. I'm able to do >> hashed string searches just fine right now by simply storing a parallel hash >> field in a record that has string, text, or varchar field. FindSingle et. >> al. do a perfectly fine, fast job of doing lookups on them. > > But you do that in RB code. > > > Valentina C++ code will be faster. > And this will be feature of engine, so all benefit Yes, that makes good sense. Ok, I'll give it some thought on how to recommend doing this as an indexed mechanism that can implemented more efficient internally. The problem is you always have to be able to handle the hash collisions, and the manner of doing that is application dependent. So I'm not sure there is necessarily anything to be gained here. But I'll give it some thought and flag this for my growing "Add a request for this into Mantis" list. --Ed From sunshine at public.kherson.ua Wed Nov 30 18:21:43 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 10:22:05 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 6:04 PM, "Ed Kleban" wrote: >> On 11/30/05 4:57 PM, "Ed Kleban" wrote: >> >> All available methods and operators are descibed in the ValentinaSQL.pdf >> Check it >> >> sin, cos, .. >> concat, ... Trim ... > > Yep, sure enough. I was just not looking at this section before in the > right context. > > But note that NOT, AND, XOR etc are listed there as "Logical Operators of > SQL [that] can return TRUE, FALSE or NULL (unknown) values." > > There do not appear to be any bitwise logic functions available at all. > > Thus my request that you add the functions noted in my email for use as > additional Find functions to make this possible. > I have found that we have them in kernel, but they are not plugged into SQL Ops. Please add request into Mantis. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 18:22:49 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 10:23:10 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 6:04 PM, "Ed Kleban" wrote: > Is there currently a way to do a search that uses a multi-bit field such as > a short or a long as a Mask with logical operations? From what I've seen > the answer is "no", and the fact that you can have calculated boolean fields > doesn't really offer an alternative that's either viable or valuable from my > perspective. We need add WHERE fld & mask Is this what you ask for ? -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 18:34:58 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 10:35:22 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 6:14 PM, "Ed Kleban" wrote: >> I can wish >> >> t.SetBlank >> t.UpdateRecord() >> >> >> So on default SetBlank() MUST make copies of old values. > > Sorry. I don't get it. Why must SetBlank make copies of old values? It > copies what values from where and places them where? All in RAM. It must copy them to be able execute UpdateRecord() It must because we need OLD values to update index. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 10:38:35 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:39:03 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 9:56 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 5:08 PM, "Ed Kleban" wrote: > >> But my whole point is that when you want to find a string fast, you use a >> hash, not comparisons embedded in some string package -- no matter who wrote >> it or how well. In fact even the REALbaic "=" operator is one of the worst >> possible things to use to do string comparisons. This is simply the wrong >> approach from my perspective. > > I think it is good idea add Hash Index for STRING/VarChar fields which are > marked as Primary Key. Don't for get "Text" flavor fields stored as Blobs as well. There's a case where you REALLY want to use hashes because it saves you from having to page in or search many very large strings. > > Just IMHO this is bad design make keys on strings. :-) Because if the indexing mechanism such as hashing is valuable, then the database facilities should support it so that the user doesn't have to -- that being the whole point of a database? Yeah, sure I can appreciate and even support that view. The problem is, that off the top of my head, I don't know what "add[ing] HashIndex for STRING/VarChar fields" really means ... or why "[for] fields which are marked as Primary Key" is relevant since I might well have multiple string fields in a record that I want to access with hashes -- not likely, but possible. hmm... Yeah, I guess you could do this easy enough. It raises a few questions though.. 1) What syntax do you use. I suppose you could use EVFlags.fIsHashed on a VarChar, String, or Text field. As a result a separate parallel hash field would be created in the magic secret index table land that the user never sees. Then FindSingle and the other Find commands just work transparently... Yep. Sure. That would work fine. The only dicey question is whether there is value in allowing the internal hash encodings to be exposed or usable as arguments. I guess I'd say no, and if it really mattered you could simply implement hashes yourself manually the way I am now. There is one likely hitch however. This will only work for certain collation attributes unless you always cast a string into some normalized form before performing the hash. So in the case of kStrength, kPrimary for example, you'd have to cast the string to a version with no accents and all lowercase before calculating the hash. But yeah, I guess that's readily doable as well. Ok you've convinced me. Or we've convinced me. This should indeed be a feature added to V2, and can offer dramatic speed improvements to all. I'll (eventually) submit a feature request to Manits. From Ed at Kleban.com Wed Nov 30 10:40:12 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:40:34 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 10:21 AM, "Ruslan Zasukhin" wrote: >> There do not appear to be any bitwise logic functions available at all. >> >> Thus my request that you add the functions noted in my email for use as >> additional Find functions to make this possible. >> > > I have found that we have them in kernel, but they are not plugged into SQL > Ops. > > Please add request into Mantis. > > Excellent! I will gladly do so! Thanks --Ed From sunshine at public.kherson.ua Wed Nov 30 18:47:51 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 10:48:18 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 6:38 PM, "Ed Kleban" wrote: >> Just IMHO this is bad design make keys on strings. :-) > > Because if the indexing mechanism such as hashing is valuable, then the > database facilities should support it so that the user doesn't have to -- > that being the whole point of a database? Yeah, sure I can appreciate and > even support that view. > > The problem is, that off the top of my head, I don't know what "add[ing] > HashIndex for STRING/VarChar fields" really means ... or why "[for] fields > which are marked as Primary Key" is relevant since I might well have > multiple string fields in a record that I want to access with hashes -- not > likely, but possible. > > hmm... > > Yeah, I guess you could do this easy enough. It raises a few questions > though.. > > 1) What syntax do you use. I suppose you could use EVFlags.fIsHashed on a > VarChar, String, or Text field. As a result a separate parallel hash field > would be created in the magic secret index table land that the user never > sees. > > Then FindSingle and the other Find commands just work transparently... Yep. > Sure. That would work fine. right > The only dicey question is whether there is value in allowing the internal > hash encodings to be exposed or usable as arguments. I guess I'd say no, > and if it really mattered you could simply implement hashes yourself > manually the way I am now. Also right. Or only C++ developers can get to hashing. > There is one likely hitch however. This will only work for certain > collation attributes unless you always cast a string into some normalized > form before performing the hash. So in the case of kStrength, kPrimary for > example, you'd have to cast the string to a version with no accents and all > lowercase before calculating the hash. But yeah, I guess that's readily > doable as well. :-) It needs check, may be IBM guys have hash methods. Something they have exactly. > Ok you've convinced me. Or we've convinced me. This should indeed be a > feature added to V2, and can offer dramatic speed improvements to all. > > I'll (eventually) submit a feature request to Manits. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 10:51:11 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 10:51:35 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 10:22 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 6:04 PM, "Ed Kleban" wrote: > >> Is there currently a way to do a search that uses a multi-bit field such as >> a short or a long as a Mask with logical operations? From what I've seen >> the answer is "no", and the fact that you can have calculated boolean fields >> doesn't really offer an alternative that's either viable or valuable from my >> perspective. > > We need add > > WHERE fld & mask > > Is this what you ask for ? > What I asked for (and would like to see some day eventually) was: >> VField.FindMasked( >> inMask as Integer, inKeep as Integer, inSelection as VSet ) >> as VBitSet >> >> VField.FindMaskedAsArraySet( >> inMask as Integer, inKeep as Integer, inSelection as VSet ) >> as VArraySet >> >> VField.FindFirstMasked( >> inMask as Integer, inKeep as Integer, inSelection as VSet ) >> as Integer // Returns RecID >> >> High-order bits of the inMask argument would be truncated to the size of the >> VField. The inKeep argument would be one of: >> >> EVKeep.includeExactMatches >> EVKeep.excludeExactMatches >> EVKeep.includePartialMatches >> EVKeep.excludePartialMatches But if we can do this much sooner in SQL, that's great. Perhaps this translates to: WHERE "(fld & mask) = mask" // EVKeep.includeExactMatches WHERE "(fld & mask) <> 0" // EVKeep.includePartialMatches WHERE "not ((fld & mask) = mask" // EVKeep.excludeExactMatches WHERE "not ((fld & mask) <> 0)" // EVKeep.excludePartialMatches I would also request that if they are some magic symbols like "|", "^", "-", and "~" for OR, XOR, DIFFERENCE, and [1's complement negate] that are also already in the kernel and can be enabled along with "&" for AND that it would be great to have those as well.... But I'll happily take what I can get and be a very happy camper if I can use "&" in the manner noted above. From maxprog at mac.com Wed Nov 30 17:55:14 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 10:55:39 2005 Subject: [V4RB] Convert_v1_v2 Message-ID: <7B56E793-5733-4A82-9C8C-5B9366E3FC46@mac.com> Hi, What is the origin encoding used when converting text field? The conversion right now is screwing all accentuated characters. Is there a way to tell the Convert_v1_v2 I was using a given encoding for the database so it converts properly to URF-8? Stan From Ed at Kleban.com Wed Nov 30 11:05:09 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 11:06:13 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 9:48 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 5:33 PM, "Ed Kleban" wrote: > >>> Have you specify >>> Table.SetBlank( forAdd ) >>> ^^^^^^^ >>> This can give 20% speedup >>> >> >> Yes, I recently went through and put in "forAdd" in all my SetBlank calls. >> But since I typically do a SetBlank just once before entering a loop to set >> memory buffer fields and do an AddRecord for hundreds of records, It doesn't >> really have much affect. > > In fact this is wrong. > > It can kick you IF you have BLOB fields. > For BLOB fields, it is VERY important that you SetBlank() on each iteration. Ah, good to know. Ok, so noted. This will be very easy since I only intend to have a single VarChar and a single Blob field in the entire database, and those are both accessible by only a common, well-encapsulated function. In essence I have a "BDataStore" class that has a single call allowing you to intern an arbitrary RB string and returns a signed recId. If the recId is positive, the string was already found, or was just added, at the specified recId in a table having a VarChar and Vlong hash fields. If the recId is negative then the string was already found, or was just added, at the absolute value of the recId in a table having a Blob Text and a Vlong hash field. Strings are represented by one of these Vlong BDataId values in all other database fields that require a use of a string. Thus each string gets stored only once no matter how many times it appears in the application source, all string lookups are hashed, all string references require only a Vlong for storage, all string comparisons are as fast as integer comparisons, and constant BDataID values can be used to make instant string comparisons for commonly compared strings. Less filling, works great, runs very very fast. From Ed at Kleban.com Wed Nov 30 11:12:21 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 11:12:44 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 10:47 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 6:38 PM, "Ed Kleban" wrote: > >> There is one likely hitch however. This will only work for certain >> collation attributes unless you always cast a string into some normalized >> form before performing the hash. So in the case of kStrength, kPrimary for >> example, you'd have to cast the string to a version with no accents and all >> lowercase before calculating the hash. But yeah, I guess that's readily >> doable as well. > > :-) > > It needs check, may be IBM guys have hash methods. > Something they have exactly. > Be very careful. You don't want or need a hash with fantastic polynomial distribution characteristics like CRC, you don't need a secure hash like MD5. You simply need a very very "good enough" fast hash, like "rotate by 5 bits and xor". I'd look to see if the IBM guys have a normailzation method for converting a string to a standard form and then use your own hand-coded hash algorithm on that. I'd be very wary of using another "hash" algorithm without understanding exactly what it's doing. It's the primary bottleneck in making this run fast. From Ed at Kleban.com Wed Nov 30 11:16:08 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 11:16:31 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 10:34 AM, "Ruslan Zasukhin" wrote: > On 11/30/05 6:14 PM, "Ed Kleban" wrote: > >>> I can wish >>> >>> t.SetBlank >>> t.UpdateRecord() >>> >>> >>> So on default SetBlank() MUST make copies of old values. >> >> Sorry. I don't get it. Why must SetBlank make copies of old values? It >> copies what values from where and places them where? > > All in RAM. Yes, but from where in RAM (serving what purpose) to where else in RAM (serving what other purpose)? > > It must copy them to be able execute UpdateRecord() > > It must because we need OLD values to update index. I guess I'm just having a brain freeze right now. I'll think about it more later and see if I can make sense of this. From sunshine at public.kherson.ua Wed Nov 30 19:46:16 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 11:46:38 2005 Subject: [V4RB] Convert_v1_v2 In-Reply-To: <7B56E793-5733-4A82-9C8C-5B9366E3FC46@mac.com> Message-ID: On 11/30/05 6:55 PM, "Stan Busk" wrote: > Hi, > > What is the origin encoding used when converting text field? The > conversion right now is screwing all accentuated characters. Is there > a way to tell the Convert_v1_v2 I was using a given encoding for the > database so it converts properly to URF-8? Yes this report is in Mantis. We need check this issue ASAP -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 21:00:44 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 13:01:13 2005 Subject: Genetic string searches Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 6:51 PM, "Ed Kleban" wrote: >> WHERE fld & mask >> >> Is this what you ask for ? >> > > What I asked for (and would like to see some day eventually) was: > >>> VField.FindMasked( >>> inMask as Integer, inKeep as Integer, inSelection as VSet ) >>> as VBitSet >>> >>> VField.FindMaskedAsArraySet( >>> inMask as Integer, inKeep as Integer, inSelection as VSet ) >>> as VArraySet >>> >>> VField.FindFirstMasked( >>> inMask as Integer, inKeep as Integer, inSelection as VSet ) >>> as Integer // Returns RecID >>> >>> High-order bits of the inMask argument would be truncated to the size of the >>> VField. The inKeep argument would be one of: >>> >>> EVKeep.includeExactMatches >>> EVKeep.excludeExactMatches >>> EVKeep.includePartialMatches >>> EVKeep.excludePartialMatches > > But if we can do this much sooner in SQL, that's great. I first of all this should go to SQL > Perhaps this translates to: > > WHERE "(fld & mask) = mask" // EVKeep.includeExactMatches > WHERE "(fld & mask) <> 0" // EVKeep.includePartialMatches > WHERE "not ((fld & mask) = mask" // EVKeep.excludeExactMatches > WHERE "not ((fld & mask) <> 0)" // EVKeep.excludePartialMatches > > I would also request that if they are some magic symbols like "|", "^", "-", > and "~" for OR, XOR, DIFFERENCE, and [1's complement negate] that are also > already in the kernel and can be enabled along with "&" for AND that it > would be great to have those as well.... > But I'll happily take what I can get and be a very happy camper if I can use > "&" in the manner noted above. This will by C++ rules: & bit and | bit or ~ bit xor ^ bit no -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 21:03:08 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 13:03:31 2005 Subject: Optimal Performance String Searching with Hashes Re: VChar vs VText In-Reply-To: Message-ID: On 11/30/05 7:12 PM, "Ed Kleban" wrote: > Be very careful. You don't want or need a hash with fantastic polynomial > distribution characteristics like CRC, you don't need a secure hash like > MD5. You simply need a very very "good enough" fast hash, like "rotate by 5 > bits and xor". I'd look to see if the IBM guys have a normailzation method > for converting a string to a standard form and then use your own hand-coded > hash algorithm on that. I'd be very wary of using another "hash" algorithm > without understanding exactly what it's doing. It's the primary bottleneck > in making this run fast. You know you are right. I remember that have some canonic form of strings. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 21:03:50 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 13:04:14 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 7:16 PM, "Ed Kleban" wrote: >>> Sorry. I don't get it. Why must SetBlank make copies of old values? It >>> copies what values from where and places them where? >> >> All in RAM. > > Yes, but from where in RAM (serving what purpose) to where else in RAM > (serving what other purpose)? Ed, Very easy !!! Each Table have record buffer. And each Table have second record buffer for old values. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From lfredricks at proactive-intl.com Wed Nov 30 11:26:38 2005 From: lfredricks at proactive-intl.com (Lynn Fredricks) Date: Wed Nov 30 13:27:09 2005 Subject: Valentina 2 Windows Foundation Database Suite Announced Message-ID: <20051130192642.114E07F065@spatula.dreamhost.com> Valentina 2 Windows Foundation Database Suite Announced COMPLETE TOOLS FOR DEVELOPMENT USING ALL MAJOR WINDOWS ENVIRONMENTS November 30, 2005. Beaverton, Oregon. Leading multi-platform database vendor Paradigma Software, Inc, makers of the ultra-fast Valentina 2 Office Server, announce next-generation suite of components and a complete suite of tools for creating and deploying desktop and network based database solutions on the Windows operating system. These tools are all based on the Valentina 2 database kernel. Valentina 2's next generation, object-relational database model builds on the traditional relational database model much like C++ builds on top of the C language. All that you know from working with traditional database systems from IBM, Oracle and Microsoft also apply to Valentina-based development. This robust technology excels at ultra fast management of millions of records. Valentina 2 utilizes a revolutionary new abstraction data model: Links. Links are highly sophisticated structures that manage high speed, low overhead handling of 1:1, 1:M, M:M relationships. Valentina Links include ObjectPtrs and Binary Links, new reinterpretations of Foreign Keys. A Valentina 2 kernel document on this system is available by free download on the Paradigma Software website. Valentina 2 Windows Foundation is a special version of Valentina Developer's Network. With VDN (Windows Foundation), developers can develop and deploy complete client-server database solutions to their customers, royalty free, based on Valentina Embedded Server. Valentina 2 technology also supports XML data/schema management, best practices Unicode for international applications, Valentina SQL, 448-bit encryption, transparent file format support across operating systems and many more features. Valentina 2 Windows Foundation includes Valentina 2 C++ ADK Standard Edition for Windows (available separately for $199), and Valentina Studio Admin Edition for setting up, server start up and monitoring and other database creation and management utility functions. Also included in Windows Foundation (and available separately) are: Valentina 2 COM ADK. High performance COM component for building desktop database applications with COM-compatible tools such as Borland Delphi and Microsoft Visual BASIC 6 (and earlier). Valentina 2 COM means you don't have to use yesterday's database to ensure compatibility with your existing infrastructure. Valentina 2 COM is also available separately for $199. Valentina 2.NET ADK. Valentina 2.NET ADK empowers developers to deploy enterprise-ready, database enabled desktop solutions with any language compliant to the Microsoft .NET CLR. The current beta works with Framework 1.1, while the final shipping release will support Framework 2.0. Valentina 2.NET ADK is also available separately for $199. Both ADKs include Data Client components for simultaneously querying both local databases and remote database servers, including Valentina Office Server. VDN (Windows Foundation) also allows developers access to two years of software maintenance, test copies of Valentina Office Server, reseller pricing on Valentina Office Server and special promotion opportunities. VDN (Windows Foundation) pricing starts at $599, with support options in multiple languages. For more information, visit the Paradigma Software website at http://www.paradigmasoft.com. About Paradigma Software, Inc Founded in 1998, Beaverton, Oregon-based Paradigma Software, Inc is the leading provider of incredibly fast and robust database solutions for business and development. Valentina 2 technology powers solutions as diverse as graphics applications from major Japanese electronics companies to solutions supporting US public schools. Paradigma Software solutions are available for every major development environment on the Windows and Macintosh platforms. Contact Paradigma Software Ph. (503) 574-2776 http://www.paradigmasoft.com From Ed at Kleban.com Wed Nov 30 13:32:36 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 13:33:03 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 1:03 PM, "Ruslan Zasukhin" wrote: > On 11/30/05 7:16 PM, "Ed Kleban" wrote: > >>>> Sorry. I don't get it. Why must SetBlank make copies of old values? It >>>> copies what values from where and places them where? >>> >>> All in RAM. >> >> Yes, but from where in RAM (serving what purpose) to where else in RAM >> (serving what other purpose)? > > Ed, > > Very easy !!! > > Each Table have record buffer. > And each Table have second record buffer for old values. > Ah ha! The good old secret second record buffer we don't document trick. How silly of me. I should have known. Ok, that helps a bit. We now have more cards on the table for us all to see. I still have the same question: >> Yes, but from where in RAM (serving what purpose) to where else in RAM >> (serving what other purpose)? > I'm afraid I still need more help connecting the dots here because I still don't understand why the following are true: > > It must copy them to be able execute UpdateRecord() > > It must because we need OLD values to update index. Presumably if I'm going to change the values in the memory buffer with SetBlank, I've already stored any new values I care about by having already performed a UpdateRecord.... or stated another way as a question: Why would I ever want to do SetBlanks without first having executed UpdateRecord() if I care about the current memory buffer values? Wouldn't it be foolish for me to do this? Or is the (undocumented) case that by definition and intent SetBlank always first does an UpdateRecord before blanking the memory buffer unless you specify forAdd so that you don't accidentily screw yourself by being foolish? From maxprog at mac.com Wed Nov 30 21:03:35 2005 From: maxprog at mac.com (Stan Busk) Date: Wed Nov 30 14:03:59 2005 Subject: [V4RB] Convert_v1_v2 In-Reply-To: References: Message-ID: Hi, At least we should be able to choose to not convert encoding, I mean, just place the text as is to the database. Also could it be possible to get some feeback when this function is applied? Right now it locks the application and display the beach ball... Stan > On 11/30/05 6:55 PM, "Stan Busk" wrote: > >> Hi, >> >> What is the origin encoding used when converting text field? The >> conversion right now is screwing all accentuated characters. Is there >> a way to tell the Convert_v1_v2 I was using a given encoding for the >> database so it converts properly to URF-8? > > Yes this report is in Mantis. > > We need check this issue ASAP > > -- > Best regards, > > Ruslan Zasukhin > VP Engineering and New Technology > Paradigma Software, Inc > > Valentina - Joining Worlds of Information > http://www.paradigmasoft.com > > [I feel the need: the need for speed] > > > _______________________________________________ > Valentina mailing list > Valentina@lists.macserve.net > http://lists.macserve.net/mailman/listinfo/valentina From sunshine at public.kherson.ua Wed Nov 30 23:21:19 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 15:21:41 2005 Subject: [V4RB] Convert_v1_v2 In-Reply-To: Message-ID: On 11/30/05 10:03 PM, "Stan Busk" wrote: > Hi, > > At least we should be able to choose to not convert encoding, I mean, > just place the text as is to the database. v2 need convert to UTF16 > Also could it be possible to get some feeback when this function is > applied? Right now it locks the application and display the beach > ball... You can try run it from separate thread. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From sunshine at public.kherson.ua Wed Nov 30 23:29:13 2005 From: sunshine at public.kherson.ua (Ruslan Zasukhin) Date: Wed Nov 30 15:29:36 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 9:32 PM, "Ed Kleban" wrote: > I'm afraid I still need more help connecting the dots here because I still > don't understand why the following are true: > >> >> It must copy them to be able execute UpdateRecord() >> >> It must because we need OLD values to update index. > > Presumably if I'm going to change the values in the memory buffer with > SetBlank, I've already stored any new values I care about by having already > performed a UpdateRecord.... or stated another way as a question: > > Why would I ever want to do SetBlanks without first having executed > UpdateRecord() if I care about the current memory buffer values? Wouldn't > it be foolish for me to do this? Or is the (undocumented) case that by > definition and intent SetBlank always first does an UpdateRecord before > blanking the memory buffer unless you specify forAdd so that you don't > accidentily screw yourself by being foolish? I have understand almost nothing. Let me explain what I think you need: * you are on some CURRENT record. ALWAYS. * this means that values of THIS record are LOADED into Table record buffer. * Now you want update this current record. what you do. * Case 1 -- you just assign few fields -- and do tbl.UpdateRecord. this is 99% cases when you assign field new values, Valentina copy old one into second buffer. This old value is required to be able update index if it exists. * Case 2 somebody still can wish just blank this record. so -- tbl.SetBlank() -- tbl.UpdateRecord() the same: Valentina must copy old values. This is very rare case. But it have right to live. much more often, you do tbl.SetBlank() before you going add record. -- Best regards, Ruslan Zasukhin VP Engineering and New Technology Paradigma Software, Inc Valentina - Joining Worlds of Information http://www.paradigmasoft.com [I feel the need: the need for speed] From Ed at Kleban.com Wed Nov 30 15:52:39 2005 From: Ed at Kleban.com (Ed Kleban) Date: Wed Nov 30 15:53:04 2005 Subject: Performance metrics - some early conclusions In-Reply-To: Message-ID: On 11/30/05 3:29 PM, "Ruslan Zasukhin" wrote: > On 11/30/05 9:32 PM, "Ed Kleban" wrote: > >> I'm afraid I still need more help connecting the dots here because I still >> don't understand why the following are true: >> > I have understand almost nothing. > > Let me explain what I think you need: > > when you assign field new values, Valentina copy old one into > second buffer. This old value is required to be able update index > if it exists. > Ok, I get it now. It took me a while to understand why you need it to update the index, but it makes sense. So V2 copies the entire record every time you call SetBlank unless you use "forAdd" even if only one field, or none of the fields, is indexed. Sounds like an opportunity for optimization. Thanks! --Ed