Best Practice Makes Perfect

The question before us today: should the DXL Exporter give an error and refuse to export design elements if (a) the user has less than Designer access to the application, and/or (b) the design of the application is hidden?

I'm going to argue that the answer is "no" in both cases. At the moment, there are restrictions -- you can't export any design elements if the database design is hidden, and (last I checked) you can't export an entire design at once if you don't have Designer access, but you can export individual design notes.

Some of you might regard these as security features, e.g. if you're using hide design to prevent users from seeing information in "hidden" fields, you don't want them to export the form design and see what the field names are. Others, including myself, regard these limits as a nuisance that does far more harm by preventing useful applications, than any good that may come from making it easier to hide information.

This is an example of "security through obscurity;" it only protects your information insofar as the people trying to get to it don't know how the technique for bypassing the restriction. Security through obscurity is still security; for instance, I know that it's not all that hard to open any combination padlock if you know the trick, but I'm not overly concerned that someone will use that knowledge to break into my gym locker (though I always give the dial a couple of turns to make it harder, should anyone try). But security of any sort is only valuable if there's not an obvious workaround.

There are several APIs that let users access information about Notes databases and design elements. None of these APIs are restricted by lack of Designer access or by design hiding (except that you can't get formulas that are hidden without going to the C/C++ APIs).

For instance, here's a very simple way to see the values of hidden fields in a database with hidden design. Any user can do this even without the Designer client, and without using DXL.

There are also various tools, such as NotesMan and the Teamstudio suite, that let users with zero application development skill get all sorts of detailed information about nsf files. The NotesPeek program, which is publically available, also will give you basically all the information you would need to know about an application. DXL adds very little to that.

Even without such tools, a user who can program in LotusScript or Java can use those APIs to get basic information about the names of design elements and their contents -- a list of all the fields on a form, for instance. The DXL restrictions I'm discussing seem to be aimed at users who can program, who have a Designer client, and are looking for information that they couldn't get from these other APIs or from publicly available tools. To me, this sounds like an empty set.

But let's just suppose for a moment that there are such users. We have to also assume that they aren't bright enough to use the NotesNoteCollection class to get all the design elements in the database and copy them to a local database without a hidden design, in which they are Manager. Then they can use DXL to their hearts' content. This is really pretty obvious, and takes maybe 10 lines of code.

The Hide Design feature, in particular, isn't really appropriate to use for security. Read/write access to documents and design elements, database ACL lists, and the like -- those are security. There are too many ways for an even somewhat clever person to get access to any information that they technically do have access to, for that to be effective as a security measure. Hide design's real purpose is to protect proprietary source code.

So in brief, these DXL limitations do very, very little to protect information, but they do make it more difficult to write efficient applications that use DXL to do useful tasks, such as scanning databases on a server for fields without field help or images without alt text.

Please comment. If you support leaving these restrictions in place (or adding to them!), could you give examples of a way in which they increase security or some other benefit?

Andre Guirard | 1 July 2009 01:32:56 PM ET | Caribou Coffee, Plymouth, MN, USA | Comments (14) | Permanent Link

I recently dealt with a question from someone who wanted a way using LotusScript to take a document selection and export each document's data into one of several files, grouped based on the value of the "AssignedTo" field -- so, a separate file for each person documents are assigned to. Having too many files open at a time would be a blocker, so they needed to sort the documents first to write all the data for one file, then move on to the next file.

When faced with such a problem, a lot of people's first instinct seems to be to create a folder that's sorted the way they want, put the documents into it, then read them back out. While this works, it's not going to give the best performance, because there's a lot of overhead involved in removing the old set of documents from the folder, adding new ones, indexing the view, and reading them back out. It's a lot of disk activity and network activity, slowing the server (assuming it's a server database), and it makes the database bigger than it was before, because folder membership is tracked by little records associated with the folder giving the document ID, timestamp, and whether the document was added or removed at that time. This is necessary so that folder membership can replicate. Besides which, we have a problem if there are multiple users working with the same folder, that they may interfere with each other.

It's much more efficient to sort documents in memory, but people don't like having to write extra code to do it.  I'm working on a sample LotusScript library that should make this a lot easier, but in the meantime, here's the approach I suggested. The specific application didn't really require the documents to be sorted -- just grouped into collections that go in to the same export file. So I suggested this approach:

Andre Guirard | 22 June 2009 08:51:48 AM ET | Home, Plymouth, MN, USA | Comments (6) | Permanent Link

Something I happened on recently in the course of helping the template developers, that I thought might not be generally known.

When you create a shared action in LotusScript, if it uses unknown names, you are allowed to save that shared action, even though the syntax would be considered incorrect for code used in any other context. For instance, if you write this line in an agent:

the compiler will complain if the GetExecutionContext function isn't defined elsewhere in the code. A shared action will tolerate this, however, with the expectation that the reference will be resolved at runtime as a function in the form, view, or whatever that includes the shared action.

What's important to realize about this, is that the compiler still can't produce object code with this unresolved reference. So for this shared action, Designer saves only the source code. When the action is used, the source must be compiled at runtime in the context of the design element containing it.

Of course, this is only possible in cases where the source code is available at runtime. So, shared actions containing unresolved references are not compatible with being able to hide the design of your application.

If this is a problem, define a script library that includes the globals and/or functions that you want your shared action and form (or whatever) to share.  Both the form and the shared action would "use" that library. When code is loaded, script libraries referenced in multiple places in the code are only loaded once, and their global variables only allocated once. So if a form event assigns a global that's defined in the script library, and a shared action reads that same global, it will see the value assigned by the form code. Since this can be successfully compiled, the object code is stored, and no runtime compilation is needed.

Andre Guirard | 16 June 2009 01:36:07 PM ET | Home, Plymouth, MN, USA | Comments (5) | Permanent Link

I noticed this morning, while writing a utility that uses DXL to reverse the order of left and right frames in a frameset (for BIDI) that the representation for complex framesets, though it "round trips" successfully, is not structured as one might expect.  For instance, consider the frameset shown on the right. We would expect the DXL to have a tree structure such as:

However, the actual structure is:

A subtle but important difference -- frame D is a child of the wrong node.

The importer doesn't actually pay attention to the XML hierarchy in this case. Instead, it looks at the 'rows' or 'columns' attribute of a frameset element to determine how many children it has. That's why the round-trip works, and of course, if you wanted to manipulate the frameset in a way that doesn't depend on the structure (replacing a replica ID in links, for instance), it's not a problem. But if you're doing something where it does matter (as it does for my application), the workaround is to fix up the DOM tree using a routine such as the following:

Andre Guirard | 1 June 2009 01:03:37 PM ET | Home, Plymouth, MN, USA | Comments (1) | Permanent Link

Revision: Never mind, looks like this survey is only open to Design Partners, who will have gotten email about it.

For everyone else, please note we're working with partners to make sure we do the highest-value improvements to DXL, first, as we work our way toward full fidelity.

Previous surveys of our business partners for DXL priorities have identified views, agents, and script libraries as high value areas for improvements. We've been working on those, and we believe you'll be pleased by the improvements in 8.5.1 CD6/Beta 1. We've greatly improved stability also, which was identified as a top priority.

We're about to plan the next phase of DXL work, and we'd like your further feedback to help us identify which improvements will be of the greatest immediate use to you. We're focusing on getting as much as we can into 8.5.1 to do the most good.

Please help us prioritize our upcoming activities by thinking about what you need most to do the things you want to do with DXL; then go to Crowded Wisdom on bleedyellow.com and weigh in. We'd like feedback from more people and thanks to this tool, are equipped to measure with more precision the relative "weight" of potential improvements.

Andre Guirard | 26 May 2009 10:47:35 AM ET | | Comments (6) | Permanent Link

Earlier, I blogged about the problem of Queryclose code being unable to prevent closure of documents in the Eclipse-based Notes client. This is SPR # AGUD7DSU3Q. At that time I described a "fix," actually a workaround since in addition to installing a new version, it requires modifying the form.

We're working on a fix for 8.5.1 that would simply fix the problem (note: as always, no information about fixes in future versions can be considered final until the version is actually released).

If you feel this is an important issue, and particularly if you feel it will block deployment of 8.5 clients in your company (or your customers), please contact Lotus Support and open a PMR to request inclusion of the fix in an 8.5 FP. This will help me make the case for this internally, in case there's any push-back on doing the FP.

Equally, if you feel this is not an important issue and you'd prefer that developer work on something else, please feel free to respond by commenting here.

Thanks

Andre Guirard | 20 May 2009 03:11:12 PM ET | Caribou Coffee, Plymouth, MN, US | Comments (8) | Permanent Link

I had reason to look recently at the FAQ we link to from the new topic form in the forums. It seems to me this is not the right format for our main purpose (heading off the same two dozen questions that get asked over and over). I'd like to have a FAQ document that directly addresses those questions, then tells them where to search for more answers. Here's my proposed list of questions; I've probably left off some important ones, so please chime in with any you think should be added or removed. My goal is to have up to 50 of the commonest questions.

• Where else to look for answers.
• Why doesn't anyone answer my question on the forum? Doesn't IBM monitor these?
• @DbLookup, @DbColumn don't work on the web.
• Other problems with @DbLookup, @DbColumn, GetDocumentByKey, etc.
• Comparing date fields.
• Documents in a view show up under different dates for different users.
• My full text search isn't working.
• How do I add fields to a form while the user is editing it?
• Change the value of rich text in a document that's open in edit mode (add text, insert attachment, add a picture).
• My application is slow.
• @UserName doesn't work in view selection formula.
• @DbLookup, other functions not working in view selection or column formula.
• View selection to include only documents that have/do not have responses of a particular kind.
• Sort view categories by total.
• Sort view based on information in, or number of, response documents
• Users can't edit documents they're supposed to be able to edit.
• Users can't see documents they're supposed to have read access to.
• Documents go missing.
• Old documents suddenly reappear long after they have been deleted.
• My hide formula isn't working.
• How to assign a sequential number to each document.
• Schedule an agent to run at time intervals that the agent properties do not support (e.g. the 1st and 3rd Sunday of every month).
• Notes client or Domino server crashed; how do I find out why?

Andre Guirard | 18 May 2009 12:52:58 PM ET | Home, Plymouth, MN, USA | Comments (12) | Permanent Link

So there's someone with a PhD who says time travel is possible without the planet-sized exotic objects previously thought necessary -- he thinks he can build something human-scale.

The first application that springs to mind, of course, is to travel back in time and do something to stop Pauly Shore from making any movies.

Unfortunately, it's not possible to go back to before the machine is first constructed. Darn.

Andre Guirard | 5 May 2009 09:24:13 AM ET | | Comments (4) | Permanent Link

Just posted a new article in the Domino Designer wiki about working with dates and times in Lotus Notes. As usual, you are invited to comment if you see anything wrong or incomplete.

Link

Andre Guirard | 2 May 2009 11:10:42 PM ET | Plymouth, MN, USA | Comments (3) | Permanent Link

A feature added in the Notes version 8.0 client but not much publicized, is the ability to perform a full-text search in a view using a Notes URL. This makes it simple to automate the creation of complex full-text searches for the user, perhaps based on information they enter in a dialog.

The URL syntax to perform the search is:

Notes://server/db/view?SearchView&query=yourQueryHere

There are no arguments to control stemming and sort order; these go to the defaults that the user last used. The server, db, and view components can be as usual for Notes URLs, and the query parameter must be escaped into valid URL characters. If the view is already open, the search occurs in the open view window.

The code below demonstrates how to create a search in this way based on information the user entered in a dialog:

Of course, you could read the user's field values in the agent and calculate the query there, but I like an approach that lets you put everything that needs to know what fields are in the user assistant, into a single design element. If you prefer to use LotusScript to calculate the query, you could do it in the Queryclose event of the dialog form.

Note: the expression wksp.CurrentView.View.UniversalID doesn't work in desktop private views. You might want to modify this code to see whether wksp.CurrentView.View returns Nothing, and in that case use the view name or alias instead. In case the UNID is available, though, it's more reliable, since there may be duplicate view names or aliases.

Andre Guirard | 30 April 2009 06:00:00 AM ET | Caribou Coffee, Plymouth, MN, US | Comments (8) | Permanent Link