IBM®
Skip to main content
    Country/region select      Terms of use
 
 
   
     Home      Products      Services & solutions      Support & downloads      My account     

developerWorks  >  Lotus  >  Forums & community  >  Best Practice Makes Perfect

Best Practice Makes Perfect

A collaboration with Domino developers about how to do it and how to get it right in Domino

A performance question came up after my presentation today which I thought I should address with a blog entry. This has to do with the database option "Don't support specialized response hierarchy" and the associated functions @AllChildren and @AllDescendants.

"Specialized response hierarchy" refers to an internal table stored in the NSF that lets Notes quickly locate all the responses to a specified parent document. Without this table, functions that require this ability -- such as the above-mentioned functions and the NotesDocument.Responses property -- simply do not work. Instead, they act as if the document had no responses. Of course, it's extra work for Notes to update this table every time a note is modified, so it's better to turn the option off if it's not needed. But how do you tell when it's needed?

Let's take a sample application containing one main form, and one response form. You want a view that shows the main documents whose status is "Unfinished", and also their responses. Without using the responses table, you could write the following selection formula:

SELECT (Form = "Main" & Status = "Unfinished") | Form = "Response"

I'm assuming that the response documents do not contain any field value that would let you tell whether their parent is in the view.

To do the same selection assuming the specialized response hierarchy enabled, you might write:

SELECT (Form = "Main" & Status = "Unfinished") | @AllResponses

On the face of it, there seems to be no differences between these two approaches. In either case, only those responses whose parents are in the view, are visible.

However, it's not quite so simple. Suppose there are 100 main documents and each of them has one response, and suppose 20% of the main documents are at "Unfinished" status. In the first case, all 100 response documents are also in the view, even though most of them are not visible. They're in the stored view index, and you can find them with a full-text search. So the index contains column values for 120 rows of data.

In the second case, the view index contains only those responses whose parents are also in the view. That's 20 main documents and 20 response documents, a total of 40. So the stored view index is smaller, and probably takes a lot less time to calculate just because fewer documents must be inserted into it.

Incidentally, you can see the size of the view indexes using Domino Administrator, if you want to try a test and compare for yourself.

Also consider what happens while the first view is being indexed -- imagine yourself in the situation of the view indexer. You're handed documents in random order, and you have to insert them into the index at the appropriate position. Every time you find a response document, you have to search the view for a matching main document (one whose UNID matches the response's $REF). Locating a document in a view by UNID is pretty fast, but it's not instantaneous.  Also, if you find a response document and don't find the corresponding main document, you have to insert it somewhere in the index anyway to keep track of it until the matching main document does turn up, if that ever happens.

When using @AllResponses, on the other hand, it's not necessary to search the view for a main document matching a given response, or vice versa. That's because when you find a main document that should be in the view, you can immediately grab all the responses from the internal "response hierarchy" table (a quick search because it's sorted by UNID), and they can be inserted into the index immediately following the main document you just added. If the first part of the selection formula matched any response documents, you would still have to insert them provisionally until their parents turned up, but in this example that's not going to happen (much -- if there's a replication conflict document it could end up that way).

So that sounds like less work. I haven't tested it, but it seems reasonable -- don't you think?

Andre Guirard | 23 April 2009 06:00:00 AM ET | DanNotes Conference, Denmark | Comments (5)


 Comments

1) Untitled
Erik Brooks | 4/23/2009 7:33:17 AM

Makes sense to me... sort of. @AllChildren and @AllDescendants were always a bit weird to me, seeing as how they only have that meaning within the context of a view.

An @Formula used for view selection generally being a boolean formula, you're expecting "| @AllChildren" to return a TRUE or FALSE accordingly. But it doesn't -- it kicks off a recursive-style forced inclusion of other docs.

What's always interested me about "specialized response hierarchy" is the fact that this is the one area of NSF that is actually supporting a JOIN operation of sorts. It's an index of the $Ref field, with a couple of @Functions and LS/Java hooks to allow you to access that index from the context of one doc.

Imagine if that functionality was expandable to allow developer-specified field indexes at the NSF-level. I.E. on every doc update in the NSF Domino would update your custom field index.

This is similar to "optimize document table map" (which indexes the Form field) and "specialized response hierarchy" (which indexes the $Ref field). Throw in some programmatic hooks for us to poll said index and/or make the @Formula/LS compiler be aware of the index, and you've really opened up NSF's real-time-querying capabilities to approach that of a relational database.

2) Post dannotes performance question
Klaus Terman | 4/24/2009 6:13:38 AM

At dannotes we spoke of view performance tuning and I started as soon as I returned home :)

Now I have a few export views that I use in some agents. These views are only used by the agent. The question is, if the index is dropped and the agent runs. Will the agent act as a user and just make the indexer create and update the index for it to use without any problems for the agent?

3) Yes and no
Erik Brooks | 4/24/2009 8:04:34 AM

@2 - There is only ever one view index for a view.

If the index is dropped, and a process needs it, it will be recreated. The index has all of the docs. Then the process that called for it (agent, web ?OpenView command, etc.) *then* the index gets filtered down real-time to what that user is supposed to see.

4) from the help
Klaus Terman | 4/24/2009 8:53:30 AM

Wouldn't the best configuration of export view indexes (run once a month) only used by agents or backend processes be manuel indexing and then to let the agent start of with a notesview.refresh method?

Or maybe automatic view indexing with a discard time of 1 day?

Is one better than the other ?

5) re: from the help
Andre Guirard | 4/26/2009 5:27:43 PM

@Klaus (slightly off topic), yes, accessing the view from an agent will cause the view to update the same as access by a user would (which is to say, depending on the indexing settings). I don't see a great deal of difference between setting a short index discard time vs. indexing the view "manually" in the agent (the first way needs less disk space, the second is faster if the documents don't change very often, since it takes advantage of the old index). Bear in mind that servers often have limits on agent execution time, so if the view takes particularly long to index it might not be an option to calculate it on the fly. You could have two agents in that case, one to start the index building, and another to do the export.

Alternatively, if there's no special need for it, why even have a view? NotesDatabase.Search works and lets the agent not require a supporting design element. The data wouldn't be sorted, but that's not always necessary.

 Add a Comment
Subject:
   
Name:
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?     Cancel

Search this blog 

Disclaimer 

    About IBM Privacy Contact