IBM®
Skip to main content
    Country/region select      Terms of use
 
 
   
     Home      Products      Services & solutions      Support & downloads      My account     

developerWorks  >  Lotus  >  Forums & community  >  Best Practice Makes Perfect

Best Practice Makes Perfect

A collaboration with Domino developers about how to do it and how to get it right in Domino

I recently dealt with a question from someone who wanted a way using LotusScript to take a document selection and export each document's data into one of several files, grouped based on the value of the "AssignedTo" field -- so, a separate file for each person documents are assigned to. Having too many files open at a time would be a blocker, so they needed to sort the documents first to write all the data for one file, then move on to the next file.

When faced with such a problem, a lot of people's first instinct seems to be to create a folder that's sorted the way they want, put the documents into it, then read them back out. While this works, it's not going to give the best performance, because there's a lot of overhead involved in removing the old set of documents from the folder, adding new ones, indexing the view, and reading them back out. It's a lot of disk activity and network activity, slowing the server (assuming it's a server database), and it makes the database bigger than it was before, because folder membership is tracked by little records associated with the folder giving the document ID, timestamp, and whether the document was added or removed at that time. This is necessary so that folder membership can replicate. Besides which, we have a problem if there are multiple users working with the same folder, that they may interfere with each other.

It's much more efficient to sort documents in memory, but people don't like having to write extra code to do it.  I'm working on a sample LotusScript library that should make this a lot easier, but in the meantime, here's the approach I suggested. The specific application didn't really require the documents to be sorted -- just grouped into collections that go in to the same export file. So I suggested this approach:

       Dim session As New NotesSession

        Dim ByAssigned List As NotesDocumentCollection
        Dim selected As NotesDocumentCollection
        Dim db As NotesDatabase
        Dim docCur As NotesDocument
        Dim strAssignedTo$
       
        Set db = session.Currentdatabase
        Set selected = db.UnprocessedDocuments
        Set docCur = selected.GetFirstDocument()
        Do Until docCur Is Nothing
                strAssignedTo = docCur.GetItemValue("AssignedTo")(0)
                If Not IsElement(ByAssigned(strAssignedTo)) Then
                        Set ByAssigned(strAssignedTo) = db.CreateDocumentCollection
                End If
                Call ByAssigned(strAssignedTo).AddDocument(docCur)
                Set docCur = selected.Getnextdocument(docCur)
        Loop
       
        ForAll coll In ByAssigned
                strAssignedTo = ListTag(coll)
                ' generate report for person strAssignedTo from documents in coll.
        End ForAll



This isn't perfect; there's probably a limit to the number of documents you can have in a collection, so you maybe could handle larger data sets by defining your own classes (which, as I mentioned, I'm also working on making simpler). But for a limited set of applications, this gets the job done without many lines of code, and without the overhead of working with folders.

Andre Guirard | 22 June 2009 08:51:48 AM ET | Home, Plymouth, MN, USA | Comments (7)


 Comments

1) Variation of this method
Brent Henry | 6/22/2009 11:18:28 AM

Andre,

I attempted this approach once and eventually hit the wall with resources (as you predicted). In my case I was dealing with a large volume of documents (10s of thousands).

The workaround was to track the Notes URL in a string list rather than use document collections. You can then get a handle to the document using NotesSession.Resolve(NotesURL) in your reporting loop.

A nice side effect was that the performance went up by an order of magnitude.

2) C handle limit
Karsten Lehmann | 6/22/2009 11:20:46 AM

Isn't there a limit of 10.000 C handles for Notes objects? I think I read something about thing in the API a few years ago. So storing a lot of Notes documents in memory should reach that limit. We already hit it in Java code.

3) Limits
Erik Brooks | 6/22/2009 2:31:53 PM

There's definitely a limit. When I use this approach I use lists of UNIDs.

And working with NotesDocumentCollections can be slow. E.g. you wouldn't think it, but using NotesDocument.GetDocument() to check for a doc's presence in a collection is MUCH slower than building an in-memory list of all UNIDs in the collection and checking against that.

4) Java Collections class
Mort | 6/22/2009 6:16:22 PM

Sorting datasets like this is nice and easy using the Collections class in Java. Create an object class with the few data values you plan to sort by or report on as fields of that class. Also include a field for the document UNID so you can retrieve other document values if you need to later. Loop through the Notes DocumentCollection and build a Collection of these objects setting each of the Objects values from the Notes document. This Collection will be far smaller in memory than a collection of entire documents. You then have a list of objects on which you can use the Collections.sort function to sort. You must pass a Comparator to the sort function to tell it how the objects are to be sorted. You just need to write a class implementing Comparator for each way you wish to sort the collection by. Eg by subject, by authorname etc.

5) Nota Bene
Colin Macdonald | 6/29/2009 2:58:53 PM

Don't forget your JavaScript Arrays have a sort method, so you could do the sort dynamically on the client or in SSJS (I think, haven't tried - yet)

6) Lotusscript solution
Rob Goudvis | 7/1/2009 6:39:31 AM

My solution looks like @4, but is done in Lotusscript. So define an array with a sort key (this key could be build from several document items) and a NoteID. Sort your table and process it by getting each entry sequentially.

Of course you could hit some limit here too.

7) PS
Colin Macdonald | 10/6/2009 12:45:54 PM

So finally got round to it, and resorting SSJS Arrays work nicely in XPages.

If you store the viewEntry in the array then you won't need to change any of your other components, just passing in a sort function will do it for you.

function sortExample(a,b){

return a.getPosition(".")-b.getPosition(".");

}

 Add a Comment
Subject:
   
Name:
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?     Cancel

Search this blog 

Disclaimer 

    About IBM Privacy Contact