IBM®
Skip to main content
    Country/region select      Terms of use
 
 
   
     Home      Products      Services & solutions      Support & downloads      My account     

developerWorks  >  Lotus  >  Forums & community  >  Best Practice Makes Perfect

Best Practice Makes Perfect

A collaboration with Domino developers about how to do it and how to get it right in Domino

Now and again, I come upon an application that has, say, 380 views (and that's just the ones shown in the outline) and a ton of documents. It's slow as molasses. The people who maintain the application know that probably only fifty views are in use, but how to tell which ones? Most of these views were probably created for a specific purpose, and the purpose has gone away, but of course nobody thinks to tell the developers that. So views just accumulate over time like snow falling, falling, until it's just too much and the roof breaks in (you'll note I'm writing from Minnesota).

One could, of course, take a guess which views were no longer in use and delete. The problem is, that unless you guess accurately, it'll seem like a major disaster to any users who are still using those views.

One could use Science. Look to see how long since each view has been re-indexed (you should be able to tell by the existence and age of a $Collation field in the design note). If a view has been unused for, say, thirty days, chuck it. Keep a copy so if someone complains you can put it back. This is fine, provided you keep track of these saved views, and provided users have a way to complain and know who to complain to, and provided you're responsive when someone needs a view back.

But that's a lot of provideds. There might be views needed for some task that's only done anually. Users may bump around in the application, wasting hours trying to figure out what happened to the view they need, without thinking to ask the support/development staff ("I know it must be here somewhere but I can't remember what it was called!"). The person with access to do something about it might be on vacation for the next three weeks (after all, they deleted those views eight months ago. They can't be on alert every minute). The user might have left it until the last minute, it's 9:30 PM and they need it now.

What's needed is an automatic view revalidation procedure. It could start with a logging function so you can tell who is using which views and how often. But since there might be views that are only used annually, you don't want to have to wait a whole year collecting statistics before you can conclude that a view is really unused. We want better performance right away.

The tool should delete unused views (based on the age of their $Collation or on the log stats). You provide it a list of views not to touch. Leave all entries in the outline (we might dim some of them), and automatically restore the view if someone needs to use it. It would initially be replaced with, perhaps, a folder of the same name, which would be a placeholder. This folder would contain Queryopen code advising the user that the view has been deleted because of apparent disuse, but if they want it back they can have it restored immediately by clicking the button -- and by the way, please use this text field to explain what the view is used for, for our records, and select how many days the view needs to sit unused before I can delete it again. The view can be stored anywhere in the database, in a format where it could be replaced on demand but doesn't take up CPUs in the app. The actual replacement would be done by a runonserver agent.

And of course, if the replacement folder sits long enough without anyone having the view restored, you can delete it  for good, and the outline entry -- but meanwhile, those folders don't hurt performance much.

Then there's also the problem that often, essentially the same view will be re-created time after time because nobody realizes that a view already exists that would serve the purpose. This is especially likely when there are already a lot of views, because who has the time to go through them all looking for the one that suits their task? We need code that'll go through all the views in an application, note those that are similar (logically the same selection formula, same sort columns) or that could possibly be merged with the use of a re-sort column, and make recommendations to the developer. The outline could look the same but change to have multiple entries pointing to the same view, so that people would find their view at the same place in the outline that they're used to.

Someone go write that tool, OK, or else tell me where it already exists. Otherwise I might have to do it. Sounds entertaining but I don't want to waste my time reproducing something that exists.

Andre Guirard | 13 December 2007 10:05:51 PM ET | Plymouth, MN, USA | Comments (22)


 Comments

1) Go for It
Theo Heselmans | 12/14/2007 2:12:24 AM

I've spent so much time already figuring out which views are used regularly (e.g. by looking at the logs).

A decent logging/monitoring/synopsis tool would be most welcome.

I would be hesitant about the 'automatic' part of your suggestion though.

2) Needed
Ursus Schneider | 12/14/2007 3:14:56 AM

I wrote a bit of code that created statistic documents when a view is accessed. I then used this info to see which views are no longer needed. It was a bit of a boring but worked for us. Your ideas sound great (especially the automation stuff) - I would be willing to help you realising it if you like. I am currently on holiday but will be available around feb 08 - drop me a line if I can help

ursus

3) How to use $Collation
Dan | 12/14/2007 10:42:49 AM

So, how do you use the $collation field to tell when a view was last used?

4) View Indices
Peter Presnell | 12/14/2007 11:10:18 AM

In many cases it can be as simple as looking at the view indices. If a view has not been accessed in 45 days its view index will be discarded.

5) Great Idea
Carlos L Rivera | 12/14/2007 11:27:13 AM

This is a great idea. We have many clients that go "view crazy" and then call us when app performance drops. I would also be willing to help.

6) Private view
Jonatha | 12/14/2007 1:39:38 PM

I would go to replace possibly unused view by private views on first use. By restricting access to user to create view in the acl, they will be created in their workspace instead inside the app. Simple, isn't it?

7) Talk to me at ’sphere, Andre
Nathan T. Freeman | 12/14/2007 5:37:07 PM

Got this one in the can. I'll talk to the crew about how best to share it.

8) $Collation
Jamie Magee | 12/15/2007 9:39:33 AM

Good post on an important topic. The $Collation field is updated each time the view index is updated. If the view's index trigger is the "Auto, after first use" or "Auto", then this happens any time any doc matching that view's selection formula is modified, even if the user wasn't using that view.

Peter Presnell has the right idea. And if you want to make sure, you can remove the index of a suspected unused view either by a) cutting and pasting back the view or b) use the "Manage Views..." right click action on the db in Admin Client. The latter lets you remove a view index, and then return later to see if the view's index has been created.

9) Private views, stored in workspace
Fabian Robok | 12/19/2007 2:08:05 AM

I'm not a friend of private views stored in the workspace at all. they are a support nightmare, if you ask me. Nobody (including db managers and developers) can see, what views a certain user has and they tend to be either not deleted, when the designer expects them to, or deleted when the user does not expect them to.

10) Team Studio & communication
Andrew Broxholme | 12/19/2007 4:53:34 AM

I'm afraid there is no substitute for communication here. If the users are complaining about poor performance then do a scan with something like Team Studio Configurator to identify view that arn't accessed by system processes then identify a bunch of 'big' views or views that look very similar that might be deletable. Take a copy of the design and email the users, if you get no response then take out a few at a time. I think automation is very dangerous excpet to identify views that can be deleted, they should never be removed except by manual action, authorised by development manager. If you do anything else then it just makes the IT department more hated than they currently are.

p.s - I am building a new application that replaces one with 470 views!

11) how to determine age of $Collation field
Joseph LeMay | 12/20/2007 10:30:44 AM

here's a $Collation field. how do you determine its age?

Field Name: $Collation

Data Type: Index Collation Specification

Data Length: 32 bytes

Seq Num: 12

Dup Item ID: 0

Field Flags: SUMMARY

20 00 02 00 00 ....

44 00 66 07 00 D.f..

00 02 00 00 66 ....f

00 02 00 0A 00 .....

24 32 56 65 6E $2Ven

64 6F 72 4E 61 dorNa

6D 65 me

12) re: how to determine age of $Collation field
Andre Guirard | 12/20/2007 6:45:14 PM

Items have a date/time modified which you can access through the NotesItem property, but actually, the $Collation item probably isn't what we need for this. Perhaps Nathan, who has got it covered, will be willing to tell us his technique.

13) I think it is $Collection, not $Collation...
Eric Houvenaghel | 12/21/2007 8:49:15 PM

This is what we learned when we built the dev tool viewEZ...

The view index is stored in an object, "linked" to the view through a $Collection item. So, when you refresh a view, nothing in the Note is modified (no date changed), just the object that the view is linked to changed, avoiding like this any replication/design refresh of the view just because the index has been refreshed.

The $Collation item is a special item (of type "Collation"...) which contains all the basic information regarding the categorization and the (re-)sorting of the view, used in correlation with the $Formula item to build the one or several view indexes.

Unfortunately, the date of the item $Collection does not seem to change, even if you dump the view index using the Administrator.

But what about the LastAccessed property of a Note? Maybe this one may give you the date you are looking for?

14) re: I think it is $Collection, not $Collation...
Andre Guirard | 12/22/2007 1:08:17 PM

It's not exactly clear when LastAccessed is updated; probably when the view is opened on-screen. The problem is that we also want to know whether a view is used without being opened. You can analyze a design to tell whether there are lookup formulas that access the view in the same application, but how to know whether deleting a view would break a lookup in another application?

The other drawback to using the LastAccessed property, is that it requires that the "Maintain LastAccessed" option in the database properties has been turned on during the last few months -- and this is one of the first things we advise people to turn off if their app is slow.

15) re: I think it is $Collection, not $Collation...
Eric Houvenaghel | 12/24/2007 10:49:27 AM

I agree with you, LastAccessed may not be the best property in our case.

The answer to your needs seems to be in the object linked to the item $Collection. Unfortunately, the structure of this object is not published in the API, so only someone who has access to the source will be able to learn more about it.

When you look at it, the object itself does not seem to have the index itself (anyway: the size of the object does not match with the size of of the index as shown in the Administrator client) but more a description of it.

The system must keep a date somewhere in this description, at least to know when to discard the index if the view index has been set to be discarded after a given number of days.

16) Many, many thanks...
Mael | 1/23/2008 1:11:28 PM

...for bringing this subject to life.

This tool is really needed.

Any news on Nathan's approach?

17) re: Many, many thanks...
Andre Guirard | 1/24/2008 7:04:30 AM

I'll have to swing by the Lotus911 booth today and ask about that.

18) nathan’s approach?
Usha | 1/29/2008 12:39:59 PM

We could really use a tool to help us validate view usage. Any word from Nathan?

19) re: nathan’s approach?
Andre Guirard | 1/29/2008 2:36:55 PM

It's not clear whether the folks at Lotus911 have come up with an approach that works.

20) re: Nathan’s approach
Mael | 1/30/2008 1:47:00 PM

Maybe there are some recyclable ideas in Lotus911's approach? Some snippets, perhaps, that could point in the right direction for developing this tool?

Many thanks, Nathan and Andre, and everybody involved in this thread for your input.

21) Solution
Dan Smith | 8/18/2011 9:16:00 PM

I researched this and got a response that I used. Code in a QueryOpen creates a document with the username, date and time and saves it in the application. Views show these documents in a format that is useful. The feature is turned on or off by a database profile document value accessed by the manager of the application. Send email to dansmith@catscollaborative.com if you need this code. I'd have to go find it, but I know where it is.

22) View Revalidation
Andre Guirard | 11/2/2011 2:27:26 PM

@Dan, your response misses one point, which is that users sometimes will open a view without specific intention to use that view, because they're just poking around looking for a view that will suit their purpose. So the fact that a user has opened a view, doesn't mean that the view is really needed. If they can't be bothered to send an email or click a button to say they want to keep this view, it can go away.

Also, I dislike creating or editing documents for "read" events like opening documents or views. This gives you a lot of extra documents, and causes "churn" in all the view indexes because documents keep getting modified while users are just browsing. For best performance you want to avoid modifying documents.

 Add a Comment
Subject:
   
Name:
Comment:  (No HTML - Links will be converted if prefixed http://)
 
Remember Me?     Cancel

Search this blog 

Disclaimer 

    About IBM Privacy Contact