September 19 2018 Wednesday

Domino logging to Syslog

First off....I know right, a *blog post*! Who knew.......

So you have Domino.

And you have a syslog server where everything except Domino is logged to.

You want Domino to play along with everything else. What can you do?


For starters there is this event handler type in events4.nsf:


Image:Domino logging to Syslog

But in typical IBM fashion the documentation for the above is practically non-existent on how this works . Nor does there seem to be a way to specify a remote syslog server. So I would presume (and dear reader feel free to leave a comment with links, etc if you know them) that this only works with a Domino server running on Linux and AIX. Now with Linux I could syslog locally and then punt my logs to another syslog server. So theoretically this could do what we want on Linux platforms.

But what about Windows I hear you ask? Well here we need to use a combination of things. Specifically this event handler:


Image:Domino logging to Syslog

That puts my Domino server logs into the Windows Event viewer like so:


Image:Domino logging to Syslog

First stage ignition completed! Now to punt this over to a syslog server. For this I use
NXLog Communty Edition installed on the Windows servers and configure this to throw Windows Events out to my syslog server (in my case Nagios Log Server which is free for <500MB/day although I have used GrayLog before as well that has an free version and an Enterprise with is free <5GB/day and also IBM's QRadar with also has a community edition).

With NXLog conifgured to send Windows Event Logs to my syslog server I now have my Domino logs (from Windows Domino servers at least) in an infinitely more usable format (any my security folks maybe happier too. Final stage ignition complete!


So from this (Domino console and log.nsf):


Image:Domino logging to Syslog

To this (Nagios Log Server):


Image:Domino logging to Syslog


And with the use a quick elastic search, all my Domino server logs in one place:


Image:Domino logging to Syslog

So there you have it. Domino logging to a syslog server.

Darren Duke   |   September 19 2018 05:42:29 AM   |    domino    |   Comments [3]

Here's my CollabSphere 2018 presentation from the conference in Ann Arbor, MI. Again, a huge thank you to Richard and Leann Moy for organizing another spectacular conference.

,
Darren Duke   |   July 26 2018 10:28:18 AM   |    domino  mwlug  collabsphere  presentation  security    |   Comments [1]

June 12 2018 Tuesday

10 Year Blogaversary

I really wasn't keeping track of this, but I've had some issues with the blog lately and went down the archive section and low and behold the first post was on July 2nd 2008. 10 fricken years. An entire decade of ranting and raving and hopefully occasionally useful stuff......That's not to say I've really kept up the pace from the early years (in fact the frequency has plummeted), nor has it been all smelling of roses (more on that later).

So what have the last 1,685 posts taught me? Well, first off I've never moved the blog off of Domino and a true testament to the genius that is Steve Castledine (I do find the occasional bug but I can usually work around that). The UI was changed back in 2013 (thanks to Sedar's work with bootstrap) and the location move to Prominic but it's still the same NSF and same data. I wonder how many different blog platforms Stuart has tried in the last decade? It is behind NginX these days and the blog was SSL'd a while back.  

So what else has happened here since that first posting? Well....
  • 1,685 blog posts.
  • 1,226 comments.
  • 2 podcasts have been born.
  • 1 podcast has been put to rest.
  • 8 end of year (snark) reviews have been published.
  • At least 13 conference presentations have been given (11 are linked on the blog), but that number is probably closer to 17.
  • There are 13 drafts that never saw the light of day. One or two are pretty scathing.

Now for some interesting product things....

First BlackBerry post - July 8, 2008
Last BlackBerry post - March 9, 2011

First iPhone post - March 9, 2011 (the exact same post as the last BlackBerry one!)

First Quickr post - August 21, 2008
Last Quickr post - August 25, 2017 (almost a year after IBM support ended)

First Symphony post - June 11, 2009
Last Symphony post - June 11, 2009 (only one post, a tad prescient)

First Let's Encrypt post - October 1, 2015

First Veeam post - June 15, 2009

I don't track any other stats any more, like page reads and that kind of stuff (if there is any Google Analytics stuff in here, it's just because I've not gotten around to taking it out...I'm egotistical and a tad narcissistic but not *that* egotistical and narcissistic), but you can tell the popular posts usually by either comments on the blog or off-line messages like email or phone calls. So what was the most active blog posts? There are two and they are both IBM related, and remember when I said above it was not all plain sailing....

The first was a lamenting, somewhat ranty yet absolutely correct post where IBM were all fired up about mail.next and I mentioned that mail.now should take precedence. This culminated in IBM threatening to revoke (my non-existent) IBM Champion status and I believe they also threatened to sue STS and boot us out of the business partner program. Why you may be wondering what caused all this kerfuffle? This sentence, “no one in my organization will tell me the truth because I’m a vindictive asshole, so don’t cross me or I will end your career" in reference to what an IBM executive may think to themselves.

Now, IBMers usually won't call me directly, so they'll go via Lisa. Call they did, and all they seemed to want to know was who possibly could this IBM executive be? I wasn't anyone I actually know personally so if you've ever had a conversation with me, it's not you. I also wasn't any of the names mentioned in the calls Lisa got involved in. Boy was she pissed at me, but I think IBM using threats pissed her off more. Buy her a drink at the next conference you see her at and get the full story.

The second was also IBM Champion related. It is also one of the funniest things I've ever done "form-submission-wise". I filled in an IBM Champion self nomination for myself and actually submitted it. You have to see it to really get it. I still chortle to myself on the odd occasion I go read that modern day classic.  

Honorable mentions also go to the posts about IBM killing 9.0.2 and it's slightly older sibling 9.0.2 Where for art thou?.

What other upsides are there to this decade long journey? Well, I've not written or updated a resume/CV for years. The hiring process for new customers goes something like this:

Step 1 - Customer Google's something. Google kindly returns one of my blog entries.
Step 2 - The customer contacts STS asking if we know this stuff. Lisa searches the blog, sends several articles on the topic to the customer.
Step 3 - Profit.

Seriously, that's pretty much how this works now, and while we'll never really know how much impact the blog has had to STS revenue, I think it's up there. And Lisa never wanted me to start a blog in the first place! True story. Also a true story, the search was broken in the blog (due to a security HTTP header) and that's how I realized it had been 10 years.

Finally there is nothing quite like Googling something and your own blog post has the answer to the question. It's equal parts effing cool and worrying that you used to know this and no longer even remember writing the answer.

Here's to another 10 years......
Darren Duke   |   June 12 2018 05:10:19 AM   |    misc    |   Comments [4]

It's a three day weekend in the States so I'm busy patching servers and installing fix pack and intermediate fixes to Domino servers. I came across an issue that a customer had mentioned to me a few weeks back while on-site that I see all the time but think nothing of. Basically they stop all the IBM Domino services and any backup agents and they still can't install a FP or IF. They called support and were told to boot into safe mode and do the install (wtf support?).

Here's there error (and we've all seen it);


Image:You’re installing a new fix pack or IF on Domino and Windows won’t let you

So how do you fix this without booting into safe mode (again, wtf support?)? Well, provided you've stopped the IBM Domino services (both of them) and any backup agent servers then the lightly culprit is this little bugger, the Windows Management Instrumentation Service:


Image:You’re installing a new fix pack or IF on Domino and Windows won’t let you

Stop this service (it may restart again, so if it does disable it until the upgrade completes, then re-enable and restart it).

You may see this, if you do click OK, it's not going to hurt anything to stop these services while upgrading:


Image:You’re installing a new fix pack or IF on Domino and Windows won’t let you

Once Windows Management Instrumentation is stopped, voila, Domino upgrades successfully:


Image:You’re installing a new fix pack or IF on Domino and Windows won’t let you

No need to boot into safe mode (again, again, wtf support?).


If you did disable the Windows Management Instrumentation service because it restarted, remember to go back and re-enable it and restart it.


Darren Duke   |   May 28 2018 04:38:43 AM   |    domino  upgrade    |   Comments [3]

If you are using Apache as either a HTTP server or a reverse proxy (like my free one) then it is relatively simple to disable TLS1.0 and TLS1.1. Basically this should do it:


SSLProtocol TLSv1.2



Except if you're using SNI
. With Apache you may see a strange phenomenon where no matter what you do to a virtual server config TLS1.0 and TLS1.1 still remain active. This has to do with an OpenSSL bug outlined here
https://bz.apache.org/bugzilla/show_bug.cgi?id=55707.

What this bug means is that if you have multiple virtual hosts (either in a single config or as multiple configs), Apache can only use the SSLProtocol setting from the first host it loads. Which one will load first? No idea. But if you change each and every virtual host to the above an reload Apache, voila, TLS1.0 and TLS1.1 are now disabled.
Darren Duke   |   March 12 2018 10:15:02 AM   |    android  security  ssl    |   Comments [0]

We've all heard the swirl around IBM's reversal on releasing a Domino 10 (a really good reversal of a really bad decision), but let's not forget until 10 ships it's still feature packs that distribute new features and fixes and feature pack 10 (FP10) is a biggie.....

FP10 completes IBM's promise to finally provide Java 8 in both the client and DDE (new in FP10) and on the server (release in FP8...I think....). So because of this quite massive change (and Eclipse has been updated to 4 dot something from 3 dot something) I would test this more than I generally would especially if you are doing anything Ecplisy or Javery stuff in your clients or apps.


Also if you are updating your Traveler server to Domino 9.0.1 FP10 be sure to install Traveler 9.0.1.20 first!. You have been warned.


Get your FP10 here

Domino :
https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=Collaboration%20Solutions&product=ibm/Lotus/Lotus+Domino&release=9.0.1.9&platform=All&function=all

Notes :

http://www.ibm.com/support/fixcentral/swg/quickorder?parent=Collaboration%20Solutions&product=ibm/Lotus/Lotus+Notes&release=9.0.1.9&platform=All&function=all&source=fc

It is also worth pointing out that with FP9 there were new templates released for mail9, pubnames and pernames. These are not installed when you upgrade to FP9 or FP10 and need to be installed separately. These add some new features like running mail rules on existing mail.
Darren Duke   |   January 31 2018 07:17:44 AM   |    901  domino  lotus  lotus notes  notes    |   Comments [4]

Last month, in December 2017 a new (well old, but new) vulnerability was discovered in TLS, the ROBOT attack (Return Of Bleichenbacher's Oracle Threat) and yes, your Domino servers are probably susceptible to it. To avoid re-posting everything from that article go read it them come back.

Your back? OK. So you need to disable any and all RSA encryption ciphers. So here goes (all tests were done on a 9.0.1 FP9 server).....


Before a custom SSLCipherSpec:


Image:How to prevent ROBOT (Return Of Bleichenbacher’s Oracle Threat) on Domino servers

If you look at the ciphers, sure enough there are RSA ciphers in there (TLS_RSA_xxxxxx) for both TLS 1.2 and TLS 1.0 respectively:


Image:How to prevent ROBOT (Return Of Bleichenbacher’s Oracle Threat) on Domino servers

Image:How to prevent ROBOT (Return Of Bleichenbacher’s Oracle Threat) on Domino servers

OK, Houston we have a problem. To rectify it use this SSLCipherSpec


C030009FC02F009EC028006BC0140039C0270067C013C0140039C013



Note, I added this via the Domino Console):



set config SSLCIPHERSPEC= C030009FC02F009EC028006BC0140039C0270067C013C0140039C013



Results of the SSL Test after restarting HTTP:


Image:How to prevent ROBOT (Return Of Bleichenbacher’s Oracle Threat) on Domino servers

And the errant ciphers are gone:


Image:How to prevent ROBOT (Return Of Bleichenbacher’s Oracle Threat) on Domino servers
Darren Duke   |   January 15 2018 01:51:20 PM   |    domino  domino security  security    |   Comments [3]

December 21 2017 Thursday

2017 Snark Review

Holy crap, this snark-laced review thing has been going since 2010......

Firefox started at 50, ended at 57.

Chrome started at 55, ended at 63.

IE...ho-hum.

Edge...keeps informing me it's more secure than Firefox. Show me the source code and I'll decide that.

Switched back to Firefox as of Quantum (57) as my main browser. Seems (at least to me) to be much faster than Chrome.

Was there a Connect 17? If there was I wasn't there. Pretty certain I won't be at Think 18 so I'm starting a streak of some type.

Still never seen a live (or otherwise) CCM installation. This is now a very long streak, as long as CCM has been a thing. CCM still is thing right?

Rob Novak and a collaborator issued a fix for Chrome and Firefox users still using Quickr. IBM really screwed up killing Quickr and foisting CCM on the world. I see Quickr every now and then, for CCM see above,

Speaking of IBM, they claim they are in no way exiting the Domino market, nor selling it all to HCL. Confused? Yeah.....

At MWLUG in August IBM pretty much 'fessed up to screwing the whole "we're not dead yet" message and mentioned there "could" be a Domino 10.

And there will be a Domino 10. They've sort of pinky promised that in "Jam" sessions. This is a much needed and somewhat surprising admission that the whole "lets just do fix packs and rename them to feature packs and no one will notice" message that, while completely insane, was IBM's "message" for quite a while. It seems to me that the "only feature packs" insanity is a microcosm of the whole IBM 2015 plan. Remember that train wreck? Everyone knew it was bad but carry on they did, doing irreparable damage. Yeah, it's a lot like that now I recollect on it.

I mentioned HCL and the confusing things around that right?

Speaking of "jams" I attended the virtual one. IBM keep doing the same thing over and over again and expecting different results. For now I'll be reserving my jam participation to the one I use on toast.

The new pod has stalled a bit. Not sure why. Shame as I actually have some useful tips piling up.

Alien technology apparently exists in some building in Vegas. Isn't Think 18 in Vegas? Maybe I will go to Think 18.....oh wait, IBM and alien technology? Maybe I should leave the country during Think.

Many companies screwed up their UI designs, including but not limited to Sonos and Skype (on Windows 10).

Confusion around HCL and IBM....I mentioned that already but it's stuck in my head.

Speaking of Windows 10, I'm now running it on my main work PC's except the laptop I travel with. It's irritating as f#*@ but you get used to it being irritating as f@#$. Still, it's not all bad. Just mostly bad. And irritating as f*%!.

Speaking of "mostly bad" things Trump is *still* in office and Brexit is *still* happening.

The 0.29 of a bitcoin I mined back in 2013-14 is now worth $5,000+. If it keeps up this growth rate for 3 more years it'll be worth over $5,000,000. Cross your fingers dear reader, cross your fingers.

HCL? Kinda weird right?

Got an Apple Pencil. I actually like it. A lot. Glad I didn't have to shell out $75 on a Apple Pencil Sharpener. If you do get an Apple Pencil for $99 make sure to get the $10 app "Notes Plus". It makes the Pencil. Truly it does.

Tried to use Verse several times. The install was surprisingly easy. More Domino and less Websphere of an install process (at least a 1.0.2 where I came into this song). But it's still missing *so* much that I go back to iNotes/Notes almost immediately. Then I forget the lack of features is as irritating as f**# and try it again, only to be mightily disappointed by it again. The search though?....absolutely wonderful. Get that in Notes and iNotes you'd have a winner. I may have a 2014 blog post to that effect somewhere......if only IBM would hold Jams to get excellent feedback from their customers and partners.

So this IBM/HCL thing......WTF?

Software/hardware that made 2017 Keepass, Wink Smarthome Hub, Firefox Quantum, Apple Pencil and Notes Plus for iPad.







Darren Duke   |   December 21 2017 02:59:00 PM   |    misc    |   Comments [5]

August 25 2017 Friday

Quickr Fix for Chrome 60+

Yeah, Quickr right? Long time since I've posted about Quickr. Still a great (if somewhat complicated and unsupported) product, Still being used by me and several clients despite IBM having several products claiming to replace it.. Despite IBM ceasing support many moons ago it chugged along very nicely  so long as a reverse proxy is in front of it to allow TLS 1.2 and SHA2. Except Chrome 60 broke it. Google (and Safari before it) changed the way XHR works and basically buggered Quickr in the process (and Connections, but IBM will give you a fix for that).

There is a thread in the Quickr forum about this Chrome issue, The always friendly (and usually beer laden on the way to present a session) Rob Novak indicated he had collaborated with some other folks and has came out with a fix. Even better he offered it to anyone who asks. If you still use Quickr, ask away. Oh, the fix also addresses Safari too (apparently)......

Image:Quickr Fix for Chrome 60+
Darren Duke   |   August 25 2017 03:53:43 PM   |    quickr    |   Comments [0]

As promised, here is the presentation from my MWLUG 2017 session on SSO. If you need (for hire) help with any of this see the STS contact page.
Darren Duke   |   August 10 2017 03:44:31 PM   |    domino  notes  saml  mwlug  notes federated login  nfl    |   Comments [0]