July 15 2015 Wednesday
Domino Directory Assistance to Active Directory when using SSL breaks with 9.0.1 FP4
UPDATE - Apparently LDAPSearch is broken, not TLS1.2 to AD. Go read this post instead and see how to do it with SHA2 certs : SOLUTION - Domino Directory Assistance to Active Directory when using SSL DOES NOT break with 9.0.1 FP4
DA and AD's....how could this not get confusing?
Over the past few days I've been working to figure out why 9.0.1 FP4 can no longer connect to Active Directory when using a SSL connection for the LDAP connection from Domino. Specifically this is AD 2012 but I would guess the same issues hit 2012 R2. Not sure about 2008. Like this:
Anyway, what worked in 9.0.1 FP3 no longer worked after an upgrade to 9.0.1 FP4. After much testing it appears that Windows 2012 servers really doesn't like TLS1.2 when using Domino. A quick Google will show you than OpenSSL seems to have the same issue so it would seem this is a Windows Server issue as opposed to a Domino issue but that is pure speculation on my part.
Here's the testing I did to figure this out. I knew AD SSL was working thanks the LDP.exe tool on my PC. The connection to the domain controller over port 636 (the SSL) port came back fine.
On a server not running 9.0.1 FP3 (not FP4) I turned up SSL logging using these notes.ini settings:
DEBUG_SSL_CIPHERS=1
DEBUG_SSL_ALL=1
I then fired up LDAPSearch.exe on the server, piped the query out to a text file and ran this query:
On the working server (FP3) this was returned in the text file:
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> A certificate has been requested
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> An X509 certificate has been requested
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> Empty Server DN List
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLEncodeCertificate> Generating a certificate message with 0 certs
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> After handshake state= 13 Status= -5000
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Exit Status = -5000
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Enter
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Current Cipher 0x002F (RSA_WITH_AES_128_CBC_SHA)
OK so FP3 is using TLS1.0 and the RSA_WITH_AES_128_CBC_SHA (the 2F) cipher. Lets see what the same query from a FP4 server does:
...snip...
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 0 Available cipherspec: 0x009F
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 1 Available cipherspec: 0x009E
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 2 Available cipherspec: 0x006B
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 3 Available cipherspec: 0x0039
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 4 Available cipherspec: 0x0067
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 5 Available cipherspec: 0x009D
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 6 Available cipherspec: 0x009C
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 7 Available cipherspec: 0x003D
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 8 Available cipherspec: 0x0035
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 9 Available cipherspec: 0x003C
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 10 Available cipherspec: 0x002F
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 11 Available cipherspec: 0x0033
... snip ....
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSLSendAlert> Sending an alert of 0x0 (close_notify) level 0x2 (fatal)
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Changing SSL status from -6989 to -5000 to flush write queue
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> After handshake state= 2 Status= -5000
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Exit Status = -5000
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Enter
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM S_Write> Enter len = 7
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Xmt> 00000000: 15 03 03 00 02 02 00
.... snip ....
ldap_bind_s( dn=cn=darren duke,cn=users,dc=blah,dc=local, pw=password, method=128 ) failed, error
: Not an LDAP errno 7289
SSL invalid certificate, may need to cross-certify.
Hum. So the 2F cipher is there but this is a TLS1.2 transaction. Also that last line threw me for a curve. Why would FP3 not care about a cross cert by FP4 does? I even added the root signing certificate from AD to Domino as an Internet Cross Certificate. No joy.
Next swing at this, I wondered if it was a TLS1.2 issue so I Googled the shit out of changing the ciphers in AD to test it. Argh, registry hacks,,,,more searching and I found this fantastic free tool to view/adjust Windows ciphers used by Schannel https://www.nartac.com/Products/IISCrypto/Download.
With this tool I adjusted the domain controller to the following (basically I selected "Best Practice, then disabled TLS1.1 and TLS1.2......oh, Domino does not like seeing TLS1.1 from the DC either):
Once I did that and rebooted the DC, voila, Domino 9.0.1 FP4 could now connect.
Again, I am not 100% certain here but this smells like a AD issue, but it is Domino that doesn't work. A Sonicwall VPN using LDAPS to the AD controller worked fine with TLS1.2, so maybe it is Domino. I did PMR this and after I'd already figured out my work around support suggested I add this to Domino server notes.ini:
So that is basically the same workaround but from the Domino side. This suggests one side or the other is either not handshaking correctly or cannot decide on a cipher when using TLS1.2.
It sure would be nice to have TLS1.2 everywhere. Some day perhaps. Someday.
.
DA and AD's....how could this not get confusing?
Over the past few days I've been working to figure out why 9.0.1 FP4 can no longer connect to Active Directory when using a SSL connection for the LDAP connection from Domino. Specifically this is AD 2012 but I would guess the same issues hit 2012 R2. Not sure about 2008. Like this:
Anyway, what worked in 9.0.1 FP3 no longer worked after an upgrade to 9.0.1 FP4. After much testing it appears that Windows 2012 servers really doesn't like TLS1.2 when using Domino. A quick Google will show you than OpenSSL seems to have the same issue so it would seem this is a Windows Server issue as opposed to a Domino issue but that is pure speculation on my part.
Here's the testing I did to figure this out. I knew AD SSL was working thanks the LDP.exe tool on my PC. The connection to the domain controller over port 636 (the SSL) port came back fine.
On a server not running 9.0.1 FP3 (not FP4) I turned up SSL logging using these notes.ini settings:
DEBUG_SSL_HANDSHAKE=1
DEBUG_SSL_CIPHERS=1
DEBUG_SSL_ALL=1
I then fired up LDAPSearch.exe on the server, piped the query out to a text file and ran this query:
ldapsearch -h dc2-pw.blah.local -p 636 -D "cn=darren duke,cn=users,dc=blah,dc=local" -w password -b "dc=blah,dc=local" "objectClass=*"
On the working server (FP3) this was returned in the text file:
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLProcessServerHello> Server chose SSL/TLS version TLS1.0 (0x0301)
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> A certificate has been requested
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> An X509 certificate has been requested
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLAdvanceHandshake> Empty Server DN List
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSLEncodeCertificate> Generating a certificate message with 0 certs
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> After handshake state= 13 Status= -5000
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Exit Status = -5000
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Enter
[1BD44:0002-1C534] 07/13/2015 04:39:14.66 PM SSL_Handshake> Current Cipher 0x002F (RSA_WITH_AES_128_CBC_SHA)
OK so FP3 is using TLS1.0 and the RSA_WITH_AES_128_CBC_SHA (the 2F) cipher. Lets see what the same query from a FP4 server does:
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLEncodeClientHello> We offered SSL/TLS version TLS1.2 (0x0303)
...snip...
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 0 Available cipherspec: 0x009F
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 1 Available cipherspec: 0x009E
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 2 Available cipherspec: 0x006B
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 3 Available cipherspec: 0x0039
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 4 Available cipherspec: 0x0067
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 5 Available cipherspec: 0x009D
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 6 Available cipherspec: 0x009C
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 7 Available cipherspec: 0x003D
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 8 Available cipherspec: 0x0035
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 9 Available cipherspec: 0x003C
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 10 Available cipherspec: 0x002F
[0824:0002-0C34] 07/15/2015 07:42:56.36 AM SSLInitContext> 11 Available cipherspec: 0x0033
... snip ....
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSLSendAlert> Sending an alert of 0x0 (close_notify) level 0x2 (fatal)
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Changing SSL status from -6989 to -5000 to flush write queue
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> After handshake state= 2 Status= -5000
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Exit Status = -5000
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Enter
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM S_Write> Enter len = 7
[0824:0002-0C34] 07/15/2015 07:42:56.38 AM SSL_Xmt> 00000000: 15 03 03 00 02 02 00
.... snip ....
ldap_bind_s( dn=cn=darren duke,cn=users,dc=blah,dc=local, pw=password, method=128 ) failed, error
: Not an LDAP errno 7289
SSL invalid certificate, may need to cross-certify.
Hum. So the 2F cipher is there but this is a TLS1.2 transaction. Also that last line threw me for a curve. Why would FP3 not care about a cross cert by FP4 does? I even added the root signing certificate from AD to Domino as an Internet Cross Certificate. No joy.
Next swing at this, I wondered if it was a TLS1.2 issue so I Googled the shit out of changing the ciphers in AD to test it. Argh, registry hacks,,,,more searching and I found this fantastic free tool to view/adjust Windows ciphers used by Schannel https://www.nartac.com/Products/IISCrypto/Download.
With this tool I adjusted the domain controller to the following (basically I selected "Best Practice, then disabled TLS1.1 and TLS1.2......oh, Domino does not like seeing TLS1.1 from the DC either):
Once I did that and rebooted the DC, voila, Domino 9.0.1 FP4 could now connect.
Again, I am not 100% certain here but this smells like a AD issue, but it is Domino that doesn't work. A Sonicwall VPN using LDAPS to the AD controller worked fine with TLS1.2, so maybe it is Domino. I did PMR this and after I'd already figured out my work around support suggested I add this to Domino server notes.ini:
SSL_DISABLE_TLS_12=1
So that is basically the same workaround but from the Domino side. This suggests one side or the other is either not handshaking correctly or cannot decide on a cipher when using TLS1.2.
It sure would be nice to have TLS1.2 everywhere. Some day perhaps. Someday.
.
Darren Duke
|
July 15 2015 06:19:59 AM
|
domino activedirectory ldap ssl
|
Discussion for this entry is now closed.
Comments (4)
Great find and valuable content as always... With this issue in FP4 { http://www-01.ibm.com/support/docview.wss?uid=swg21961701 we should double check FPs... } we should double check FPs...
IBM after starred to give new features in FPs and point releases also introduced lack of regression testing...
Thanks for sharing Darren, AD and DA and Crypto stuff is always a big pain. The tools will probably ease the pain some. ;-)
Is that setting SSL_DISABLE_TLS_12=1 ?