Darren Duke Blog Zone

Random rambling about technology, BlackBerry, Lotus Domino, accents and the pursuit of happiness.
Darren Duke is co-founder and technical lead with Simplified Technology Solutions, Inc. Why not join the growing hordes of satisfied Lotus, BlackBerry and VMware customers and hire me! New customers ARE NEVER disappointed. No crisis is too big, no issue is too small. Take a look at the latest latest customer satisfaction survey and see what we mean. Proud to work at Tags 4.5 6.5 802 851 8.0.2 8.5 8.5.1 9530 About me admin atlug backupexec basic bbdevcon bes blackberry bold books centos certification CHUG conference connections daos domino email exams exchange file size fix pack free training GLUG google ibm ideajam intern internet explorer Job titles journal LCTY licensing linux lotus lotus knows lotus notes lotus tng lotusknows lotusphere ls10 LUG mail mds migration misc musing notes outlook performance pot presentations quickr r8 rbl relaying review rim sametime san security sep smsdom snow leopard spam sprint SQL ssd SSRS storage symphony tip Tips Tips In Two transaction logging traveler vmware web dev web services windows 7 xp xpages Admin	Darren Duke Blog Zone 8.5 daos domino Production DAOS - How does 40-45% free disk space sound? Darren Duke January 6 2009 07:26:20 PM Initial DAOS with Beta 2 testing looked like a winner from IBM. Well now that IBM have released 8.5 as gold I thought I'd scare the crap out of the STS worker bees and upgrade the production mail server and turn on DAOS for mail and mail archive files. For those just back from Mars, see the official IBM DAOS (Domino Attachment and Object Services) blurb. (they should have named it "Store" not "Services") To quote another great tester (that is me for those wondering).....HOLY FRICKEN COW!!! 40% reduction in file size at the OS level! Read the line again...... OK, so you got that. And that was with the 8.0.1 compression already enabled on most of the files. You may see north of 45% if you jump straight from R6, R7 or 8.0.0 Domino servers. So what is the catch? Well there is one. Transaction logging is needed for this baby to work. So all of you that were (incorrectly) told by your previous services provider (you'll be using STS now right?) that you didn't need transaction logging turned on will need to revaluate that strategy and find separate volumes to store those on. But other than that, you should be good to go. I have tried this on an 8.5 Client and 8.0.2CH1 Client and it was completely transparent to Notes. My guess is that R6 and R7 clients will work too as this is all server side magic but you will have to verify that for yourselves. Take a look at the html file below for the full breakdown. Oh, and do you mail.boxes too! DAOS file sizes.html Comments [10] Comments 1Sean Cull 1/7/2009 7:50:56 AM what about backups Hi Darren, how are you backing up your data with DAOS ? tks, sean 2Darren Duke 1/8/2009 6:48:16 AM Production DAOS - How does 40-45% free disk space sound? Per the IBM link above: < snip > 1. If you are using deferred deletion with DAOS, set the interval to longer than the interval between your backups. For example, if you back up weekly, specify eight days for the setting "Defer deletion of DAOS objects n days" in the server document. 2. Back up NSF files on the server using a backup utility that is compatible with NSF files. The utility must be able to use the backup and recovery methods of the Lotus Domino C API Toolkit. 3. Back up the DAOSCAT.NSF and DAOS.CFG files. These files are located in the data directory. 4. Back up all NLO files in your DAOS repository. You can use any flat file backup utility of your choice (such as Tivoli® Storage Manager). If DAOS has created subdirectories, maintain the directory hierarchy in your backup. 5. After the first backup of the DAOS repository, perform incremental backups as desired of both NSF and NLO files. 6. (Optional) It is highly recommended that you archive any transaction logs so that changes that occurred after the last backup can be replayed for the most complete restoration of data. < / snip > We have a cluster server that is still 8.0.2 and that is what we backup until I verify that Backup Exec 12.5 can indeed backup and restore DAOS. Note, NLO files are encrypted by the server they are stored on via the server ID file. This means you can only restore the NLO files to a server running the same ID file. 3vlad 1/8/2009 10:50:40 AM clustering, changing back... Hi Darren Does DAOS work in a clustering environment? Since the files are encrypted with the server id do you need to back up all servers in a cluster? If I want to copy over a database on a server that does not have DAOS, does it work? If I would like to change back a DAOS database to a non DAOS how easy is it? thanks vlad 4Darren Duke 1/8/2009 11:07:04 AM No affect on clustering vlad - each member of the cluster will have DAOS enabled (or not) independently of other cluster members. As each server had it's own DAOS storage folder (which is file level, and therefore not clustered) and as each server uses it's own ID file for encryption it can be said to be a per server setting. The non-DAOS cluster members don't know or care about this as to Domino, Notes, C-API and anything else it "looks" like the attachments are still in the Notes DBs. As for turning it off, from the help it looks like it cannot really be turned off, but can be "disabled": < snip > Read Only - DAOS is currently disabled on this server. Because DAOS was previously enabled, file attachments exist in the DAOS repository, and are read when documents containing them are retrieved. New copies of any attachments, whether or not the originals are already stored in DAOS, are stored in documents instead of in the DAOS repository. < /snip > So it looks like when it is disabled, attachments are now left in the document but all documents prior to disablement are left in DAOS. You could "revert" by creating a new server (with DAOS disabled) and replicating all the dbs across. Note, I HAVE NOT TESTED ANY OF THIS. PROCEED AT YOUR OWN RISK. 5Darren Duke 1/8/2009 11:09:27 AM Another thought - a bit of a down side Sometimes when creating new servers (and this is not the "preferred" way) you can copy the NFS files across via a file copy. With DAOS enabled you CAN NO LONGER DO THIS. You will have to use Domino replication to pull the NSF files to the new server (and this is the "preferred" way anyway). 6vlad 1/9/2009 10:31:20 AM Other thoughts... Hi Darren It makes sense what you says... After re-reading how DAOS works it seems that: -it is mostly designed for emails and occurs when an email is broadcasted... What would happen with the existing attachments (before you enable the DAOS)? Are they saved once or multiple times? -would a regular application benefit from DAOS? -personally I do not see any advantage from encrypting the file with the server id. Is it possible to choose to not use the encryption? Any idea? Thanks vlad 7Darren Duke 1/9/2009 11:48:45 AM Production DAOS - How does 40-45% free disk space sound? Lets see... 1) Prior to enabling DAOS the attachment would be stored once per mail file. After DAOS the attachment is saved only once. Note, that this also looks to be true for replies to the original email that have the exact same attachment but I can't vouch for this yet. Do the math and this is powerful stuff. 1.5MB attachment to 10 people = 15MB of attachment data without DAOS. Post DAOS the server only stores 1.5MB attachment. Now, (and if replies work like I think they do) someone does a "reply to all with history and attachments"....... 2) Probably not. Unless you have the same attachment in multiple documents. 3) I don't think encryption can be turned off. As to the reason why, I guess it is so that admins (or users) can't go snooping for personal, private or secret data in the NLO files. I'd rather it be encrypted than not. 8vlad 1/9/2009 12:35:14 PM NLO may be stored unencrypted... browsing through notes ini settings at { http://www.ibm.com/developerworks/lotus/documentation/notes-ini/ I have noticed "DAOS_Encrypt_NLO" setting which "Specifies whether encryption is enabled or disabled for subsequently created .NLO files in the Domino Attachment and Object Service (DAOS) repository"... } I have noticed "DAOS_Encrypt_NLO" setting which "Specifies whether encryption is enabled or disabled for subsequently created .NLO files in the Domino Attachment and Object Service (DAOS) repository"... It is really helpful for a clustered environment and not only (imagine that you would like to change the server's name). I would not worry about the admins. If a user email is encrypted, the attachment would be encrypted and protected when saved... still admins already have access to the databases and to the users' id as well. vlad 9Darren Duke 1/9/2009 12:46:37 PM Nice find Obviously as the default is enabled you'll want to add this ini setting before you convert any databases. 10Darren Duke 1/9/2009 12:50:09 PM Another reason to encrypt If the OS is compromised..... At least with Domino even if the OS is compromised Domino and the email file "should" still be safe depending on the setup and OS. But unencrypted NLO files are at risk. The big advantage I see from NOT enabling encryption is that you can now do (5) again. Subject: Name: Email: Website: Add Comment:	Popular Darren Presents Real World DAOS Results Tips In Two Feeds Content Feed Comment Feed Recent Entries Quickr Domino 8.2 Fix Pac Symantec adds support for Reminder, Chattanooga Lot The very un-scientific lo Like many others before m Recent Comments German fixpack via suppor German @6, you are wrong - i Should be cautious when e CHUG vs. GLUG Planet Lotus Twitter follow darrenduke at http://twitter.com Archives March 2010 (2) February 2010 (11) January 2010 (8) December 2009 (10) November 2009 (14) October 2009 (10) September 2009 (9) August 2009 (12) July 2009 (13) June 2009 (16) May 2009 (20) April 2009 (9) March 2009 (7) February 2009 (14) January 2009 (15) December 2008 (8) November 2008 (8) October 2008 (12) September 2008 (14) August 2008 (11) July 2008 (15)