One of the questions I get asked frequently is "how long will DAOS take to implement on my servers". Hopefully this will give you some insight and some ammunition to take to management about why you need to do DAOS.

Some background first. This STS client has two clustered 7.0.3FP1 Domino servers running on Windows 2003 32 bit. The servers are IBM HS21 Blades with local storage for the OS and transactions logs. Domino data is stored on an IBM SAN. The client was running low on SAN storage, and hence Domino storage too. In rides DAOS to the rescue ;)

The backup cluster server was upgraded this last weekend. The production server will be upgraded in 3-6 weeks.

OK, so now the facts, figures and timelines need to upgrade a SAN based Domino server. As usual YMMV based on a multitude of variables...

Pre DAOS
Mail folder : 594G
Archive folder : 41.1G
TOTAL 635.1GB

Pre upgrade tasks to make the NSF files "pristine":
fixup : 4 hours
compact 12 hours

Post DAOS
Folder Time Taken NSF size DAOS size DAOS file count Reduction
Mail 16 hours 41.5GB 157GB 512,441 66%
Archive < 1 hour 8.7GB 7GB 18,056 61%


As you can see, we were able to reduce SAN storage by an incredible 379GB. When we do this for both servers, well, the numbers speak for themselves.

For this specific client moving to 8.5 has a number of profound advantages:
  1. A new SAN can now be postponed for the foreseeable future. Given todays economic climate, that is a huge win for IT
  2. From a software license view point, this was free thanks to PPA.
  3. With this newly found space, the client is now able to do a Quickr implementation with minimal outlay costs for hardware (indeed this will only be a new blade).

Hopefully this will give you some ideas on how to address the need for DAOS to your organization.
Darren Duke   |   June 8 2009 07:42:16 AM   |    daos    |  
  |   Next Document   |   Previous Document

Discussion for this entry is now closed.

Comments (13)

Gravatar Image
1 - Jeff Stieglitz       06/08/2009 6:59:55 PM

I'll be following your efforts with great interest; thanks for the post.

Some time ago, I had an installation that used the Domino Single Copy Object store. Over time, and after innumerable Domino crashes, the store became unusable, and hundreds of people were calling me with incruitable errors like "entry in index not found".

It was just horrible. Lotus could not accept a file that large for examination via electronic means, and would not accept a file via the post office.

Since then, I have sworn off that type of technology, thinking that one mail file per customer offers lots of redundancy and simplifies backup/recovery.

Your results seem encouraging so I hope you post a status report in a few months.

Jeff

Gravatar Image
2 - Darren Duke    http://blog.darrenduke.net    06/09/2009 6:12:58 AM

@1, let me point out that Shared Mail never really worked and, personally, I never, ever enabled it a client. Storing that amount of attachments in a NSF is a plain bad idea. We've all heard the horror stories about Shared Mail and yours is, unfortunately, one of many out there.

However, DAOS is in no way related to Shared Mail. It is a completely different animal and works as advertised. It utilizes the underlying OS for attachment storage, which is after all sometimes called a "file server" ;)

We've implemented DAOS many, many times since January when 8.5 went gold and have not had an issue. You are correct about it being a non-simple recovery right now but that should change when the backup vendors catch up in 3-9 months. However, you can always cluster and/or replicate to a non-DAOS server for that purpose (but you will lose a 50-80% reduction in backup time if you do that). Backup using DAOS on the other hand is much, much smoother and faster. Hopefully one will be backing up more than recovering.

Gravatar Image
3 - Lucas       06/09/2009 7:06:20 AM

This is very impressive. Nearly 480GB with very little cost, who could argue with that.

Gravatar Image
4 - Lucas       06/09/2009 7:08:04 AM

I mistyped that, I meant to say 380GB, not 480GB, forgive my fat figures.

Gravatar Image
5 - Ryan       06/09/2009 12:51:45 PM

All in all this was a tremendous jump. This will help our company in the future to come.

Great company STS

Gravatar Image
6 - John O’Laughlin       06/09/2009 4:21:56 PM

We have a number of Domino Mail servers in the same datacenter, does this work across Domino mail servers too? Or is it limited to one DAOS store per mail server?

Gravatar Image
7 - Darren Duke    http://blog.darrenduke.net    06/10/2009 4:36:38 AM

@5, thanks Ryan.

@6, no, DAOS is a per-server technology. Hence there is a DAOS repository stored on each DAOS enabled server (remember, all server do not need to be DAOS enabled either and they will all replicate fine). There were some indications at LS09 that DAOS could have some "central storage" options in a future release. Even without this future option the savings can be staggering. It is mouthwatering to think what a central storage option could do :)

Gravatar Image
8 - Irv Schor       07/06/2009 7:13:19 AM

Hello,

I am reviewing DAOS as an option, however our organization is already moving forward with other Deduplication technology that runs on a per-server basis as the disk level. Apparently, an agent will sit on the operating system hosting the server and it will handle all deduplication operations. Is there any benefit to considering DAOS in this fashion? Is there a performance benefit in addition to the disk savings in using DAOS? Note: The decision to use a dedicated deduplication technology was an enterprise-wide decision (not just related to Domino).

Gravatar Image
9 - Irv Schor       07/06/2009 7:17:17 AM

One other question related to restores. If you are backing up the files, it seems that the all of the .nsf files and the entire DAOS file system containing all attachments. If lets say, 3 years from now I need to restore a single .nsf file, I would also need to retreive the daos files from that time period as well, and then if I needed to submit a single file for discovery, for example, then make a replica (local, or on a non-DAOS enabled server) to provide full standalone fidelity. Is this correct, or will there be a special backup API on the way?

Gravatar Image
10 - Darren Duke    http://blog.darrenduke.net    07/07/2009 4:52:26 AM

There are some other benefits other than straight de-dup. The main one being much, much smaller NSF files. This in turn leads to a much more streamlined server (happier server) as there are no longer any large binary files stored inside the NSF file. Compact, Updall, etc all run much, much faster. Additionally, with much smaller NSF files chances of database corruption should be reduced. All-in-all DAOS is more then just de-dup.

Gravatar Image
11 - Darren Duke    http://blog.darrenduke.net    07/07/2009 5:04:16 AM

The backup providers are about 6-9 months behind on DAOS so right now restoring is somewhat manual process. I would expect Symantec, IBM, CA, et al to eventually restore the DAOS NLO files automatically at some point in the future. In the meantime check out this post, http://blog.darrenduke.net/Darren/DDBZ.nsf/dx/daos-my-2-cents-and-a-possible-future-and-how-to-back-it-up.htm

Gravatar Image
12 - Irv Schor       07/07/2009 7:41:09 AM

Thanks for that update. I think if we go is route, we'll save disk on our mail servers and improve its 'happiness' as you put it (smaller .nsf files, etc.). We will, however, then to move our backups to a replication 'hub' type server containing the full .nsf files from which we can use our dedup solution (Avamar) to handle the day to day changes. Without APIs in place, the restoration of a DAOS enabled server (more so an individual file from a particular point in time) seems fairly challenging at present. Thanks for your quick response. This info is very helpful.

Gravatar Image
13 - Latheef       01/15/2011 11:42:15 PM

We have few shared mail boxes.

But in the server document, shared mail is disabled.

In this case, will it be a problem enabling DAOS in our Domino server?