Best Practice Makes Perfect

I’m a famous photographer!

Andre Guirard — Fri, 16 May 2008 06:00:00 -0500

Without telling me, my wife submitted a picture I took of her, for Kodak's "picture of the day," and guess what! They selected it for display on their website and (every few minutes) on their big electronic billboard in Times Square, on May 18th -- that's this Sunday. If you don't happen to be able to get to Times Square on that day, you can see it, on kodak.com on or after the 18th (link on bottom left of their home page).

This agent contains an illegally added ’On behalf’ attribute. To make the agent valid, please remove it.

Andre Guirard — Mon, 12 May 2008 04:03:00 -0500

Keith Smillie had a question about the above error message, which occurred when he tried to add an "on behalf of" name to an agent by just creating a $OnBehalfOf text item.

The "On Behalf Of" value is stored in the agent design note item $OnBehalfOf, and is a text value. However, it is not valid to store a value in this item unless the "ODS version" of the design element is such as to support this feature. If you refer to the C API guide, the relevant structure is ODS_ASSISTSTRUCT, whose wVersion element must be 2 or higher. This binary structure is stored in the $AssistInfo item of the agent design note. At this time, there are no differences between the ODS formats besides this version number, but consult the latest API documentation to make certain.

There are a few different ways to fix this problem:

Edit the agent in Domino Designer.
Use LotusScript or Java to delete the $OnBehalfOf item from the agent note (You will need a NotesNoteCollection to get the note ID of the agent so that you can open it as a NotesDocument).
Use the C API for Notes to adjust the version number in the $AssistInfo item.
Export the agent to DXL, modify the DXL, and re-import it to update the agent.

Note: The latest version of the C API toolkit and documentation, is available in the Downloads section of the Lotus developerWorks website, under Downloads, Toolkits.

Performance Basics for Developers

Andre Guirard — Fri, 9 May 2008 04:56:00 -0500

Everybody please read the new whitepaper on how to write well-performing Notes/Domino apps. I've pulled together information from a lot of sources, interviewed people in Iris, and generally tried to hit the most important recommendations and get the answers right. I hope this will become an important resource for developers at all levels of expertise -- I certainly learned a few things putting it together.

The basics of messing around with design elements

Andre Guirard — Wed, 7 May 2008 15:35:00 -0500

Since I've had to answer this question three times recently, I thought I would blog about it both because it seems to be something people are needing to do, and because that way I have something to send a link to, in future.

There are basically three ways to programatically make changes to design elements in LotusScript:

Use the methods and properties that are provided, e.g. NotesForm.FormUsers.
If the change you want to make is not supported that way, you can use DXL to export the design element, manipulate the DXL, then re-import it. As yet, this does not work with 100% reliability for all design elements, however, and still not all options are represented.
For some properties, you can access the design note with a NotesDocument object and change or delete items that store those properties.

It's the third technique I'm focusing on here.

The first step is to obtain the NotesDocument object representing the design element. If it's a folder or a view, this is how:

Dim db As NotesDatabase
Dim vu As NotesView
Set vu = db.GetView("viewname")
Dim docDesElem As NotesDocument
Set docDesElem = db.GetDocumentByUNID(vu.UniversalID)

If the design element isn't a Notes view, use a NotesNoteCollection to get the note ID. For instance, to locate the design note for an outline you might write:

Function GetOutlineNote(db As NotesDatabase, Byval outlineName$) As NotesDocument
Dim nnc As NotesNoteCollection
Set nnc = db.CreateNoteCollection(False)
nnc.SelectOutlines = True
nnc.SelectionFormula = {@lowercase($TITLE) = "} & Lcase(Replace(outlineName, Split({\,"}, {,}), Split({\\,\"}, {,}))) & {"}
nnc.BuildCollection
If nnc.Count <> 1 Then
Error 31203, "Outline name " & outlineName & " matched " & nnc.Count & " outlines."
Else
Set GetOutlineNote db.GetDocumentByID(nnc.GetFirstNoteId)
End If
End Function

Obviously, similar functions could be written for other types of design elements.

Once you have the NotesDocument representing the design element, you can mess with its items. For instance, the setting for whether a design element is protected from design refresh is the character "P" in the item $Flags. So if you wanted to unprotect design elements from refresh, you could do something like this:

flags = docDesElem.GetItemValue("$Flags")(0)
If Instr(flags, "P") Then
docDesElem.ReplaceItemValue("$Flags", Replace(flags, "P", ""))
docDesElem.Save True, False, True
End If

"How do we know to do that?" you might be wondering. One way is to download the C API toolkit from this website -- it includes a file stdnames.h, important parts of which are also copied here. This describes the purposes of all the $Flags values and some $FlagsExt values.

Another way is to make the change you want manually, and compare the item values before and after to see what changed. From a view of design elements in Domino Designer press Alt+Enter or use menu File / Properties. Click the second tab to see a list of items in the design note.

Some items are of simple types -- number or text -- and you can replace their values as we did with $Flags above. Others, such as $AssistQuery in an agent, contain binary data which you'll be unable to decode from LotusScript -- but you can still delete the item, or replace it with a copy of an item from another design element of the same type (using CopyItemToDocument).

The other thing you must pay attention to, is whether the item you're modifying is signed. If it is, your modification will invalidate the signature of the design element, and you should re-sign it using the NotesDocument.Sign method before you save it. Otherwise Notes/Domino will refuse to use it. If you're not sure whether you're modifying a signed item, you can use the IsSigned property to check, or you can just sign the document anyway just in case (provided you're running your code with an ID that your users' ECL lists and so on will recognize).

In many cases, the design changes you make aren't noticed by the client until you exit and re-enter the application. The exception is outlines in version 8.0 and later -- there's a method to tell the client to reload these.

Have fun!

More on managing temporary files in LotusScript

Andre Guirard — Mon, 5 May 2008 04:30:00 -0500

A while back I posted a class to provide a NotesStream attached to a temporary file. People responded with suggestions, and on thinking about it (and getting another question about it recently), it seems to me there's a need for a more general class for creating a temporary folder for your temporary files (so that you don't overwrite anything important by accident), and deleting it and the files when you're all done. If it has the ability to generate a unique filename at need, it's easy enough to attach a NotesStream to it if that's what you needed, plus it's useful for other things, such as detaching files from a document.

Accordingly, the attached LotusScript code, described below. The class will locate the Notes client's temporary folder (this requires a call to a C function but I'm handling multiple platforms -- if yours isn't there you can easily enough add it). The class creates a folder within the Notes temp folder and remembers to delete it when the object is deleted -- assuming that you have been good and either deleted all the files in the folder yourself, or told the class about them so it can delete them.

If would be possible to write a version of this that would use Dir$ to find any files left in the folder and delete them itself, but this would still fail if any of the files were still open. Also, Dir$ doesn't work so hot with some filename characters.

Please note: when constructing a filepath using the value of the Path method, use "/" as your directory delimiter. This works on all client operating systems, and I think all servers of recent vintage also.

Also note: as indicated above, it's up to you to close any open files before the TempFolderManager object goes out of scope, or they will prevent the folder from being cleaned up.

CLASS TempFolderManager

Properties

Path (String) returns the temporary folder path. This path is unique to this object; if you create a new TempFolderManager you get a new path. You can create your own filename path by appending a filename to this (following a directory separator character: "/" works for this on all OS).

Methods

New( )
Creates a new temporary folder for you. When the object is deleted it will attempt to delete the folder, but if the folder is not empty that will fail -- that's sort of up to you.
Function NewFilename(Byval strSuffix$, Byval bManage As Boolean) As String
Creates a new unique filename and returns its full path in your temp folder. The file is not created -- that's up to you -- just the name. If you specify a suffix, that suffix will be appended to the filename after a period. Don't include the period in the suffix yourself. If bManage is true, the file will be deleted automatically when this object is deleted.
Sub Manage(Byval strPath$)
If you create your own filename in the temp folder and you want to make sure the file gets deleted when you're done, call Manage with the full filepath of the file. Take care -- if you call this method you are asking to delete the file whose path you pass in -- eventually.
Sub Unmanage(Byval strPath$)
If the file is in the list of files to be deleted, remove it from that list. The file will not be deleted when you're done. The path must match exactly the path that appears in the list of managed files, or this won't work.
Function ClearFiles( ) As Boolean
Delete all the managed files immediately. This returns True if all managed files were successfully deleted. This lets you reuse the same folder for more files without worrying about name conflicts -- for instance, if you were processing attachments from a set of NotesDocuments and you wanted to extract them all into a temp folder, but clean up after each document without having to create a new folder.

Download the code here:

A tool to manage profile documents

Andre Guirard — Mon, 28 Apr 2008 04:15:00 -0500

I finally got tired of people whining that there was no way to make a view of profile documents. So, I created a way, which you can download from here -- no charge. It's a form which, when you open it, gets a list of all profile documents in the current database and shows them, letting you select one to edit, or multiple to delete. You can click the column headings to sort.

The edit function depends on having an appropriately named form or subform; if a subform, the thing opens in a dialog. Either way, you can save your changes. If you end up with duplicate-named profiles, you can still edit them all, but this tool will tell you if you're editing in the "normal document" way rather than via EditProfile (the latter will of course only work for one of the duplicates).

The delete has a confirmation, so that you don't permanently delete all your profiles by accident.

You can just paste this form into any application, and it should work in 6.0 on up (though 8.0 has a more efficient way to get the list of profile documents, which this code will take advantage of, if available). As usual, this is provided "as is," but if you run into any difficulties please let me know here.

The form is set to appear in the Create / Other dialog, so you can get to it without adding an action or outline entry, though of course it's a good idea to do that. I just wanted to have minimum effort to add it to an application.

I'm a big fan of profile documents, both for performance and because they make it simple to centralize configuration functions on a single screen. This tool should, I think, remove one more objection to their use. Along with the practice I mentioned earlier of checking for profile document existence in the database Postopen code before trying to use it, I see no reason anyone would want to use a dumb old lookup document for keyword lists, default rich text values, and so on, anymore.

P.S. Yes, Jamie, I know NoteMan will do the same thing with more features for free, but I wanted something that would be part of the application so that people wouldn't complain about all the developers having to install it. Huh.

My novel moves one step closer to reality...

Andre Guirard — Fri, 25 Apr 2008 07:38:11 -0500

I've written this book, see. It's about a little girl and her mole man buddies (and other friends) and the evil elves who threaten her and kidnap her parents, whom they then try to rescue. I sent letters of inquiry to a couple of agents, and one of them has responded asking to see the first 50 pages. I would of course have preferred she request the whole book, but this is progress!

Once I find someone to represent me, they have to then try to find a publisher, and then there's an involved process of getting it into print. And I am impatient.

And by the way, my wife just had an article accepted by Fine Gardening magazine. Yeah team!

Lists and Option Compare incompatibility

Andre Guirard — Thu, 24 Apr 2008 05:45:00 -0500

A message in one of the dW forums recently discussed a problem they were having with the List datatype. Lists behave differently depending on whether you've selected Option Compare Nocase -- which is nice, because sometimes you like to have one that's not case sensitive. The problem is that if you use that same list in a different module with a different Option Compare setting, LotusScript doesn't remember whether the list is supposed to be case-sensitive, and just treats it according to its own settings. This can result in the data getting snarled up and being no use to any of the modules.

To avoid this problem, you have to make sure that all operations on the same list are carried out in modules with compatible Compare settings. If your settings vary between modules, you can arrange for all operations on that list to be carried out in the same module by creating a wrapper class that contains the list. Sample code appears below.

Class ListWrapper

' If you have to share a list among different LotusScript modules with different Option Compare
' settings, use this class to avoid the list getting snarled up and not working correctly for anyone.
' It will operate based on the Option Compare settings of the module where it is defined.
Public BaseList List As Variant ' make it public so it can be used in Forall statements

Function IsElement(tag As String) As Boolean
Me.IsElement Iselement(BaseList(tag))
End Function

' if the list elements are not objects, set and read their values using the Item property.
Public Property Get Item(tag As String) As Variant
Item = BaseList(tag)
End Property

Public Property Set Item(tag As String) As Variant
BaseList(tag) Item
End Property

' if the list elements are objects, set and read their values using the Object property.
Public Property Get Object(tag As String) As Variant
' same as Get Item, but assume the item is an object (more efficient if you know this to be the case)
Set Me.Object BaseList(tag)
End Property

Public Property Set Object(tag As String) As Variant
Set BaseList(tag) = Me.Object
End Property
End Class

So now if you want to keep a list of documents keyed by their note IDs, you might use statements such as:

Dim doclist As New ListWrapper
...
Set docList.Object(doc.NoteID) = doc
...
Set someDoc = docList.Object(keyNoteID$)
...
Forall aDoc In docList.BaseList

The get and set properties could be written to be more flexible, setting and returning either object or non-object values, but that would be less efficient. I'm assuming that the code that uses the list, knows what kind of data are in it -- and if not, they can do their own error trapping to deal with their peculiar mix of types.
By putting this in a script library with Option Compare Nocase, you can now have a list that's case insensitive, without having to have all your comparisons be case insensitive (in which case you might want to change the class name to reflect this).

View columns do not have a datatype

Andre Guirard — Wed, 23 Apr 2008 06:30:00 -0500

The image at the right looks deceptively like a property of the column -- but it is not. The type of data stored in a column is a non-concept, because data are not stored in columns in the view index. Data are stored in rows. The type of data stored for a column for a particular row, is determined by the formula of the column. It can be different for different rows.

It therefore makes sense to specify formatting for numbers (in case the value on a some rows happens to be a number) and dates (in case some rows contain dates) and names (because it might be a username string on some rows, or might pick up a name from another column). Because of the design of your application, you might know (or hope!) that all the values will be numbers, but that is not a property of the view design, but a statistical attribute of your data. Think of the Style field not as a property, but as a navigational control allowing you access to these different sets of formatting options. It is not the "datatype of the column," because no such thing can exist. Imagine instead of this field, a second row of tabs for number options, date/time options, and name options. It would've prevented misunderstanding if we'd done it that way at the start, but c'est la vie.

To get a correct picture of what's going on, one must think of the view's contents, and its formatting for display, as two completely separate things. If a column formula returns a number value for a particular document, then that exact number is stored in the view index; if a date/time value, that exact date/time is stored. For instance, suppose the formula for the Amount (K) column is TotalAmount/1000. If TotalAmount contains (the number value) -14506, the view index will store -14.506, expressed in binary floating point, to within the limit of accuracy of that representation.

Now suppose the settings on the "Advanced Format" tab of the column properties, specify two decimal places and parenthesis around negative numbers. How does this affect the value stored in the row entry? Answer: not at all. The sole purpose of these settings is to determine how the value will be displayed on someone's screen. -14.506 remains -14.506, even if when you open the view, you see "(14.51)" (or for some users, "(14,51)"). The column value retains whatever precision it had from the formula that calculated it (or the exact value from the original field, if it was a field column instead of a formula). It doesn't contain the character "(", or ".", or any characters. A number is the same no matter how you choose to display it.

You may be thinking, "Why does it matter, you picky Andre person?" Apart from an understanding of how things work being generally important to being able to predict what your design will do, there are three common situations where the difference between storage and presentation has important effects.

1. When users with different formatting preferences use the view, the values are displayed using their local settings. As mentioned before, different users of the same view may see (14,51) or (14.51) depending what they have chosen to use as a decimal point character. Similarly with date formatting; the order of the year, month and day can be different for different users. This is highly desirable. It really is. Please don't try to defeat this useful feature by using @Text in the column formulas to convert the values to strings. If you do, you will confuse users, and make your data sort incorrectly (because 9 < 10 but "9" > "10").

2. When you read the view data using code, for instance with an @DbColumn formula, or Columnvalues in LotusScript. These functions ignore formatting, going straight for the data. This is important both when reading the data, to realize that the value you read might not be exactly what's displayed in the view, and when specifying a lookup key, since it will only find documents that match the key exactly -- not that just look like they ought to match the key.

3. Likewise, if you sort or categorize the view, you may see unexpected results if your column values have more accuracy than they display. For instance, suppose you sort ascending first by a column containing date/time values, then by a Name column. You might expect to see 04/23/2008 Alvin before 04/23/2008 Sam, but if Alvin's date/time is really 04/23/2008 4:15 PM and Sam's is 04/23/2008 7:40 AM, then Sam will list first despite your having used the formatting options to hide the time value. Similarly, suppose you categorize a view by date values that also contain a time (which is hidden). The date (the displayed value) may be the same, but the exact date/time is likely to be different for each document. This results in a view containing 50 categories for 04/23/2008, each containing one document, because while the category headings may look the same, they are not in fact the same. If you really need the entries to have limited precision, use the column formula to return an entry of limited precision (E.g. by using @Date(fieldname) instead of just fieldname ).

From A to Z (%$#@!!)

Andre Guirard — Fri, 18 Apr 2008 09:59:59 -0500

One thing that comes as an unpleasant surprise to folks who've grown up thinking in ASCII, is the default collation sequence in LotusScript. Because, you see, "A" < "b" < "C". This is nice when you're sorting a list of strings and like the de Veres to be somewhere in the neighborhood of the Dexters, but it has side effects you might not be aware of. For example, you might expect these expressions:

Case "A" To "Z"

If keyID Like "[A-Z][A-Z]###-## Then

to match only uppercase letters. In fact, they match all the lowercase letters also (except "a", which is less than "A").

There are probably a lot of applications out there that use expressions of this sort with the expectation that they're insuring the letters are the right case. It seems so obvious that it should that way, that people might not even think to test it. This is an example, if it were needed, of why one should test the obvious...

If you really want to check for just uppercase or just lowercase, you'll have to do one of two things:

List each separate character (keyID Like "[ABCDEFGHIJKLMNOPQRSTUVWXYZ][ABC...")
Use Option Compare Binary, which switches the collating sequence to one that groups all the uppercase letters in one bunch, and all the lowercase in another bunch.

Air travel - a modest proposal

Andre Guirard — Thu, 17 Apr 2008 07:46:49 -0500

This morning my wife showed me this article from the Star Tribune (no login needed) about the past days of air travel, and my attention was caught by the photo of an air stewardess entertaining the passengers (except, apparently, for one fellow), by playing music on an electronic organ on a 1959 flight.

How far we have fallen since those days! Live music on a flight! Why aren't they doing that anymore? And look how big the seats are! These days airlines seem to be focusing entirely on cutting costs, and so they take away and take away from their passengers. Pretty soon they'll rip out all the seats and have everyone stand -- they take up too much room sitting down. If you pack them in tight enough and put a huge long strap around the bunch of them, they won't fall down when you go through turbulence. I'm sure we could easily quadruple the number of people on a flight.

Someday soon there's got to be a rebound -- the customers will revolt. Really, musicians are not that expensive when you divide by the number of passengers on a flight (especially if you don't have seats), and wouldn't you prefer to take the jazz flight to New Orleans instead of the boring old economy flight? It's like starting your vacation early. Likewise the blues flight to Chicago (or they could do jazz too I suppose). Or the Broadway show tunes flight to New York? There might not be as many takers for the polka flight to St. Paul, though.

"Write out number" function, and a challenge

Andre Guirard — Mon, 14 Apr 2008 07:34:13 -0500

Here's a little function to take an integer and write it out in words (e.g. input = 34, output = "thirty-four") -- see attached file. This is also another example of recursion. This one's a little harder to convert to iterative since it calls itself at multiple points (and there's less incentive to because the deepest level of recursion it can reach is fairly shallow -- 2 levels). Exercise for the reader: produce an iterative implementation.

It hasn't been optimized for performance either.

Recursive Mkdir (vs. Iteration)

Andre Guirard — Fri, 11 Apr 2008 04:12:00 -0500

Bob Balaban recently posted about recursion and how to program so that it is not needed. The main problem is that the size of the LotusScript stack is limited, and bad things happen if you exceed it. This is generally not a problem if you're not using recursion, because the stack only goes as deep as you have explicitly designed it to go.

I agree that it's better to try for an iterative implementation in many cases. I just thought Bob's example was rather long and complex, and I happened to have a simpler one that makes the point more clearly, and is useful. The problem was to write a function like the Mkdir command, but that would work even if the containing directory didn't already exist. So for instance, you would use the call:

MakeDir "C:\movies\stars\carmen-miranda\pics"

If you do this with the built-in Mkdir function, it will fail if the containing folder (C:\movies\stars\carmen-miranda) doesn't already exist. I wanted it to be able to create the whole series of folders, if necessary.

The simplest way to find out whether the containing folder is already there, is to try a Mkdir of the whole path and trap the error that happens if it is not so. Yes, you could parse out the filepath and explicitly check for the existence of the parent and whether it's a directory, but why do all that work when the Mkdir command will do it for you? So here's a recursive and iterative implementation of the task using that technique:

*Recursive MakeDirR*	*Iterative MakeDir*
Sub MakeDirR(Byval strWhere$) On Error 76 Goto parentDoesNotExist Mkdir strWhere Exit Sub parentDoesNotExist: ' or other problem, but let's assume that for now. On Error 76 Goto fullpatherr Dim fname$, fpath$ SplitFilepath strWhere, fpath, fname MakeDirR fpath Mkdir strWhere Exit Sub End Sub	Sub MakeDir(Byval strWhere$) On Error 76 Goto parentDoesNotExist Dim stack$ Const NL = { } Do Mkdir strWhere On Error Goto 0 ' first success, stop trapping errors; avoid infinite loop. strWhere = Strleft(stack, NL) ' "pop" a path for next iteration stack = Mid$(stack, Len(strWhere)+2) failed: Loop Until strWhere = "" Exit Sub parentDoesNotExist: ' This error code can indicate other problems, but assume missing parent. ' If not, we get a different error (75) later when trying to create the parent. Dim fpath$, fname$ SplitFilepath strWhere, fpath, fname If fpath = "" Then Error 76, "Invalid path: '" & strWhere & "'" stack = strWhere & NL & stack ' "push" onto stack to retry later. strWhere = fpath ' try a path one step shorter. Resume failed End Sub
Code shared by both approaches:
Sub SplitFilepath(Byval fullpath$, dirpath$, filename$) Const DELIMS = {/\:} While Instr(DELIMS, Right$(fullPath, 1)) ' discard final delimiter character... fullpath = Left$(fullpath, Len(fullpath)-1) Wend Dim candidate$, i% filename = Strtoken(fullpath, Left$(DELIMS, 1), -1) For i = 2 To Len(DELIMS) candidate = Strtoken(fullpath, Mid$(DELIMS, i, 1), -1) If Len(candidate) < Len(filename) Then filename = candidate End If Next Dim fplen% fplen = Len(fullpath)-Len(filename) If fplen > 0 Then fplen = fplen - 1 dirpath = Left$(fullpath, fplen) End Sub

Look at the recursive routine, MakeDirR, first. The role of the stack here is to keep track of the remaining work to be done, and where in the middle of a suspended task we were (on which line). This routine tries to create a folder, and if this fails, tries to create the parent folder, and if that fails, tries to create its parent folder, and so on until either the create succeeds, or there's an error that's not consistent with a missing parent folder. Then it backtracks to create the folders that it put off creating earlier (this time not trapping errors, to avoid infinite recursion). The variable strWhere has a different value at each level of the stack, and that's how we remember what folder we were trying to create at that level.

Converting any recursive implementation to iterative, requires a replacement for the role of the stack in tracking work to be done. In Bob's example, he uses a Queue class. This is a good way to do it, and it's nice to have such a class in your personal library (I have a version that's not specific to DOM nodes). But it doesn't need to be that complex; in many cases an array or string can be used to stuff everything you want to keep track of. The nice thing about a string or a "redimmed" array, is that it doesn't take up a lot of room on the stack -- just a few bytes -- but you can store lots of data in it. This is also true of an object variable.

In my example, I use a string (stack) to keep a list of the directories I must try to create. This technique depends on the existence of a delimiter character (newline in this case) that will not appear in the data. Each time creation of a folder fails, the failed path is added to the beginning of stack, and the last part of the path is dropped. Again, once successful in creating a folder, it turns off error trapping to avoid an infinite loop. This is the same sequence of operations as the recursive function, but without the stack, we have to add logic to explicitly store and retrieve information and to control position in the code. This makes the iterative implementation a little longer, as usual (though in this case some of the extra length is caused by better comments).

So the iterative implementation has a little more code, and lacks the elegant simplicity of recursion, but is safer (and often more efficient). In this case, the chance of overrunning the stack limit is probably fairly small, since a path won't have so very many levels of folder names. We'd probably be just fine with the recursive approach, unless we're already pushing the limit before we call this routine. So, do what you think best -- but I'm putting the iterative version in my library.

Please write XSL for Design Synopsis

Andre Guirard — Wed, 9 Apr 2008 04:35:00 -0500

You know how you can use Tools / DXL Utilities / Transform in Designer, and run a DXL description of all (or some) design elements through an XSL transformer to produce an HTML report about the design elements? You didn't know that? Well you can. The problem is that there are only a couple of stylesheets supplied with the product, none of which provide a comprehensive description of the design. There's one to list out all the LotusScript code, for instance. That's useful, but fairly limited.

If someone would write a stylesheet to translate the whole of DXL into something along the lines of the Design Synopsis, only more complete and better formatted, and share it with all of us, I would see to it that they get a nice umbrella or something, besides the fame and fortune that would naturally accrue. I'm sure I would be able to get a group together to put up a bust of the person at Lotusphere.

Why I’m glad I no longer live in Louisiana

Andre Guirard — Mon, 7 Apr 2008 05:05:00 -0500

Several months ago, a Shreveport, Louisiana police officer was caught on surveillance camera using a taser -- an occasionally deadly weapon -- on a citizen who was just standing there. So far from acting in any sort of threatening manner, the victim of this assault was talking on his cell phone, apparently not aware of the presence of the officer. See the details here. One might expect the response of the Shreveport police department to this blatant and unprovoked assault might be to promptly dismiss the officer and institute criminal proceedings for assault. Not in Louisiana. The officer was suspended for 45 days without pay and as best I can determine, is still working for the Shreveport police.

In case you want to communicate with the city officials of Shreveport, this page contains link to the various city departments, and this page, contact links for the mayor and city council.

Faster load times on script libraries - coming

Andre Guirard — Thu, 3 Apr 2008 10:46:44 -0500

I have word from one of our developers at Iris that the problem that causes scripts to load extremely slowly when they use many script libraries, has been found and corrected. Rather than being "ridiculous," performance in such a situation should be merely "bad" from now on -- a considerable improvement. For security reasons, the extra work that caused the slowdown in the first place still has to be done, but at least it can be done more efficiently.

Of course, it will be a while before a release that has this fix is in your hands, but it's something to look forward to.

We have a program in IBM where we can give other employees "thanks awards." Nothing too expensive -- the recipient can choose among hats and jackets and ice chests and secret agent USB drive pens and so on -- but a nice token. I have so awarded the developer who corrected the problem -- in gratitude on behalf of the Notes developer community.

Incidentally, if you're having a problem with this now, the recommended solution is to reduce the number of script libraries by merging some. A main factor causing long load times is the depth of your hierarchy of script libraries -- not just the total number of libraries.

The skinny on user-defined view columns

Andre Guirard — Fri, 28 Mar 2008 03:31:00 -0500

You know those columns in your inbox that let you choose to display a different background color on a row based on who the email is from, and that show the "to-ness" icons that clue you in whether the email is to you, or you're just copied, and so on?

It's kind of confusing how this stuff works, and I've been looking into it recently. I decided to write it down here so I can look it up when I forget. Here are some relevant facts:

User-defined columns can use any of the display options. If you read the Designer help you might come away thinking that you can only use them for color columns. In fact, they can also be icons, or just display characters like a "normal" column. They use a formula which is stored in text item (not formula datatype) in a shared profile document. Because of this, it is not possible to have different preferences for different users. It's user-defined in the sense that it's easy for an authorized end-user to edit the profile document and thus change the behavior for everyone -- not in the sense that one can potentially see a personalized value.
..............	The name of the profile document is a column property. All the user-defined columns in a view have to use the same profile document. If you enter different profile document names in different columns, you don't get an error message warning you of an incorrect configuration -- it just doesn't work. The name of the item in the profile document is the same as the column programmatic name (in the Advanced tab of the column properties). You don't have to accept the default column name (which is of the form "$" followed by a number), but if you change the name, the new name also has to start with "$". This is not a general rule about column names -- it only applies to user-defined columns. If you enter a name that doesn't start with $, you don't get a warning -- it just doesn't work.
The formula for the column must be non-constant, because the indexer ignores columns with constant values. IBM uses @UserName as the formula for such columns, and that's misleading because it implies some personalization is happening, which as we have seen is not the case. There's no reason you shouldn't use @UserName -- it's non-constant, which is all we require. If you do put in a formula that returns a constant value, you are not warned -- it just doesn't work. Some functions you would think would be considered variable, like @Unique, are treated as constant for this purpose. Bottom line -- use @UserName, it's safe, but remember that it's just a placeholder -- not actually used for anything. The value of the column is calculated by the server -- not by the user's workstation -- as part of the normal view indexing task (unless of course it's a local replica). Therefore, you can't do anything in the formula from the profile document, that you can't also do in a regular column formula. It's just more customizable -- not more capable in any way.
	In particular, @UserName and other user-based functions, don't work any better than they would in a "normal" column formula. Nor can you use @DbLookup or @GetProfileField or any of those other things people often want to do in a column formula and can't. You can use @Now and @Today in these functions without causing the view to rebuild every time you use it, but this is just like using @ToTime("Today") -- you are fooling the indexer so that the view also doesn't rebuild when the row values of previously indexed documents become obsolete. Result: your view contains out-of-date information. User-defined views do not help you do anything date-related that you couldn't do with normal columns.
The server (or your workstation if it's a local replica) does notice when the profile document has changed, and rebuilds the view index. So your changes get applied right away to documents that were previously indexed. I believe this also means that view indexes may be rebuilt unnecessarily if you modify the profile document for some other reason. I'm not sure whether the indexer pays attention to the item modification time. My guess is not. User-defined columns are slower to index than regular columns, but I think not very much slower. There's overhead in the setup for indexing, where the indexer has to read two notes (the view and the profile) instead of just the view note. But once it's got the formula in memory, I don't see why it should be slower to execute this formula on each document, than would a normal formula.

The third thing you might not have known about @DbFunctions

Andre Guirard — Thu, 27 Mar 2008 03:40:00 -0500

You've decided to follow best practices, and create a separate view to use in a @DbColumn operation. You accept that this will slow the server somewhat and take up more room in your database, as compared to a single view used for both purposes. Can you at least optimize performance and space usage?

My first suggestion might seem a little obvious, but I mention it because it surprised someone I thought would find it "old hat." If you use @Unique to remove duplicate values from a list, that's always a danger signal. @Unique(@DbColumn...) in particular, can be improved on. Executing @Unique on a long list is expensive because it has to compare each element with each other element, with an execution time probably proportional to the square of the number of elements (O(n*log(n)) is possible in theory). Add to that the amount of time it takes @DbColumn to scan all the entries in a view, compile a list of values from a column, and (if using a server database) transmit information over the network to the client, and you'll see there's a lot of wasted time creating a list most of whose elements you're going to throw away anyway.

Far better if you can get a list whose elements are already unique. One way to do this is with a categorized view. Assuming there are far fewer categories than there are documents, it's a lot faster for @DbColumn to just scan through the category headings, compared to a regular view where it has to scan every row. Use @DbColumn to read a categorized column, and you don't need to use @Unique on it after.

One problem, though; categorized views are more expensive for the server to maintain, and their stored index is larger, than the same view without categories. We're only interested in the list of 100 unique values. Isn't there any way to get that without all the overhead of listing 30,000 documents in the index?

In fact, there is a way, or I probably wouldn't've brought it up. In the Advanced tab of the view properties, find the option "Generate unique keys in index". If you tick this box, Notes will show only one document per unique key value (the key consists of all the sorted columns). When a document is created or modified, if a document with the same values in the sorted columns is already present in the index, the document will not be added to the view. If there are many duplications, this saves a lot of space (though it's maybe only a little faster to index than a regular sorted view).

Using Domino Administrator, you can see the sizes of view indexes. A view with "normal" sorting, had a size of 1021KB. The same view with the sort column categorized instead, was 1297KB -- a little larger because it has to store the category rows and the document rows. But if instead of categorizing, I set the "unique keys" option, I can get the index size down to 118KB. Your mileage may vary, depending how many documents use the same key.

There are a few things to watch out for with this feature:

You can't predict which document will end up in the view, among the ones with the same key. It's not necessarily the oldest, or the newest, or the most recently modified, or any such thing. If you're just reading the unique values off the sorted column, of course, it doesn't matter.
If you have any replication/save conflicts, you must exclude them from the view using a selection formula such as SELECT Form = "Something" & @IsUnavailable($CONFLICT). Otherwise, if any of the conflict documents end up in the index, they will not be visible because they are response documents whose parents are not in the view. Setting the view to not display hierarchically should also do the trick.
If a document is deleted, you might expect another document with the same key value to take its place in the view. Alas, this does not occur. The view indexer has already considered those existing documents for inclusion in the view, and rejected them because the key was already there. Now that the key is no longer there, Notes doesn't hunt through the already-rejected documents for a replacement. Documents are considered for inclusion only when they are modified or created. So, if in your application the documents you're looking up might be deleted or their keys changed, this isn't an appropriate solution for you -- unless you can arrange to have the index rebuilt each time this occurs. For instance, if the documents are deleted by an archiving agent, have that same agent issue a NotesView.Rebuild against the view, or schedule the index rebuild as a periodic server task timed for after the agent runs. A refresh isn't good enough -- it has to be a rebuild.

Thing Two of Three about @Dbfunctions

Andre Guirard — Wed, 26 Mar 2008 03:45:00 -0500

Yesterday I talked about reasons to use a separate, hidden view for @DbColumn and @DbLookup. But if you choose to risk the wrath of harkpabst, and use a single view (without re-sorts!) for both users and lookups, can you at least mitigate the adverse effect on maintainability?

There is one thing you can do. For @DbLookup, for the argument that specifies what column to get the results from, you can code either a column number, or a name. For instance, if column 5 displays the field DateDue, you could write either:

@DbLookup(""; ""; "lkByKey"; Key; 5)

@DbLookup(""; ""; "lkByKey"; Key; "DateDue")

and the result is the same. (For @DbColumn, you must use a column number. Too bad.)

You might have been taught that it's more efficient to use the column number because the value is read directly from the view index, whereas using a fieldname requires accessing the document, which is slower. This is only approximately true. The Designer help actually says, "Lookups based on view columns are more efficient than those based on fields not included in the view." Because DueDate is included in the view -- there's a column that displays that exact value -- the two formulas above are equally efficient. There may be some tiny difference between them, but very small compared to the extra time it takes to "crack open" the document note to read an item value that's not in a column, and I'm not actually even sure which is faster.

So, all right; that's nice. If the above two formulas are equally fast, there are two good reasons to use the second one.

It's more readable.
It's more maintainable because it won't break if you edit the view design and rearrange the columns. Even if you delete the column, the lookup will still work. It just won't be as fast.

But wait! There's more! By successive approximations, we come ever closer to the truth. In fact, when you specify a name argument to @DbLookup, it's not a fieldname. It's really the column name. Only if there are no columns with the specified name, does the lookup code open the document note to look for items with that name.

"What!" (you might be saying to yourself) "Columns have names? Does he mean the column title?"

No. I'm referring to the column "programmatic name" which appears on the Advanced tab of the column properties. If the column just refers to a field, it's automatically assigned a programmatic name which is the name of the field. That's why "DueDate" works in the above formula. You're not referring to the field named DueDate in that formula; you're referring to the column by that name. If you write a formula in the column instead of selecting a field, the column is assigned a unique programmatic name of the form $n where n is a number, but you can change it if you like.

Now here's a key point: you can use the column name to specify the data column for your lookup. So if you know a column is called "$4", you can write:

@DbLookup(""; ""; "lkByKey"; Key; "$4")

instead of using the column number. Once again it's just as efficient, but less likely to break when someone edits the view design.

Of course, $4 is not a very descriptive name to appear in your formula. If you want to use the column for lookups, I suggest entering a better name in the column properties.

One other fun thing you can do with column names (if you share my ideas of fun), is use them in the formulas of other columns. This is occasionally useful in avoiding a repetition of some complex calculation.

Three things you may not have known about @Dbfunctions - thing 1

Andre Guirard — Tue, 25 Mar 2008 04:07:00 -0500

@DbLookup and @DbColumn have a little quirk that's not made really clear by the documentation. If you use an @Dbfunction against a view that the user has re-sorted by clicking a column heading, the @Dbfunction will return a result based on the user's selected sort order of that view, rather than using the default sort column (also known as the "primary collation"). This is true even if the view is not open at the time. I refer to this euphemistically as a "surprise feature." There are other names for it, one containing three letters and others containing four.

As you might imagine, problems caused by this disconnect between developer expectation and actual functionality are hard to track down, because when a developer tries to reproduce it, the view is probably not sorted the same way on their workstation as it is for the user having a problem. The user may not even have used the view in weeks -- it doesn't matter how long ago, their workstation remembers their custom sorting.

So this is one reason to use a separate, hidden view for your lookups -- even though you get better performance with fewer views. Another reason is maintainability -- user views are more likely to change than purpose-built lookup views. If someone adds or moves columns and you have a hardcoded column number in a lookup to that view, there'll be a problem.

Another little-known tip about lookups tomorrow.