Handling duplicate records

[Beatrix Willius]

In my email archiving application it's typical to archive emails a few times where duplicate handling is done in the application. And this I wanted to speed up.

First I tried to get the message id faster. But I just made the archival slower or not really faster.

Then I had a look at archival with Instruments. To my astonishment I found that the following code takes quite a while:

  dim UniqueCursor as VCursor = theArchive.SqlSelect("select RecID from message where InternalMessageID = '" + Valentina.EscapeString(Left(InternalMessageID, 100)) + "'", EVCursorLocation.kServerSide, EVLockType.kReadOnly, EVCursorDirection.kForwardOnly)
  if UniqueCursor <> Nil and UniqueCursor.RecordCount > 0 then
    Return False
  Else
    Return True
  End If

I had selected not to make the field InternalMessageID a unique field because in development I tend to archive mails again and again.

So now I made the field in the database unique and let the code that adds a record take care of handling the duplicates. But even this isn't much faster than before:

  dim theTable as VTable = theArchive.Table("Message")
  theTable.SetBlank
  theTable.Field("ContentDescription").setString(currentMailFields.ContentDescription)
  theTable.Field("ContentDisposition").setString(currentMailFields.ContentDisposition)
  'and so on
  #pragma BreakOnExceptions Off
  try
    RecID = theTable.AddRecord
  catch theError as VException
    if VException(theError).ErrorNumber = 144643 then 
      Globals.theErrorLog.AddToAppLog "Skipped xxx duplicate Mails." , 1
    else
      Globals.theErrorLog.logitem "Write Message to Valentina: " + str(VException(theError).ErrorNumber), ""
    end if
  end try
  #pragma BreakOnExceptions Default

In this code the throwValentineException is what takes so long. And this reaches into rbframework.dylib.

Does anyone have an idea how to speed this up? Do I really need the SetBlank here?

Mit freundlichen Grüßen/Regards

Trixi Willius

http://www.mothsoftware.com
Mail Archiver X: The email archiving solution for professionals