This forum is closed to new posts and responses. New discussions are now taking place in the IBM Developer Answers forum.


Nov 4, 2014, 1:14 PM
24 Posts

Released - Domino server Interim Fixes that implement TLS 1.0 with TLS_FALLBACK_SCSV for HTTP to protect against the POODLE attack.

  • Category: Security
  • Platform: All Platforms
  • Release: 9.0.1,9.0
  • Role:
  • Tags:
  • Replies: 60
See the following Technote for the latest information:


Title: How is IBM Domino impacted by the POODLE attack?
Doc #: 1687167
URL:
http://www.ibm.com/support/docview.wss?uid=swg21687167
Please refer to the WIKI article cited in the Technote for more information:
IBM Domino Interim Fixes to support TLS 1.0 which can be used to prevent the POODLE attack
http://www-10.lotus.com/ldd/dominowiki.nsf/dx/IBM_Domino_TLS_1.0
Nov 5, 2014, 5:08 AM
108 Posts
Thanks for the heads-up, and...
I'm glad this took less than two weeks (10/23 - 11/3), instead of "several weeks" as originally announced. Good job!
Nov 5, 2014, 10:46 AM
26 Posts
Problem with TLS1.0 on SMTP

Today I have installed the 901FP2HF353 Hotfix. Not for poodle, but we need TLS1.0 Support for securing SMTP.

This is working quite fine, Encypted connection using TLS1.0 is established. (SSL_Handshake> Protocol Version = TLS1.0 (0x301))

But the disadvantage is that the domino server doesn't seem to support SSLV3 anymore.

Other Servers that only Support SSLV3 get an Error when they try to connect. ( 4.7.0 TLS handshake failed.)

Is it possible to additionally activate SSLV3 Support?

Nov 5, 2014, 12:06 PM
26 Posts
Thaks for your support but...

.. it does not work as expected.

Still the same error.

Nov 5, 2014, 12:39 PM
94 Posts
Please set DEBUG_SSL_HANDSHAKE=2 and DEBUG_SSL_CIPHERS=2 in your notes.ini and post the ou...
Nov 6, 2014, 11:37 AM
26 Posts
Debug Output

Hello,

it is not much but I hope anyone can read something out of it:

[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Enter
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> After handshake state= 3 Status= -6996
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Exit Status = -6996
[0318:0008-1374] 11/06/2014 17:33:47.61 int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]

Nov 6, 2014, 4:40 PM
94 Posts
That log appears truncated - is there anything else showing up on the server console? <>
Nov 6, 2014, 11:37 AM
26 Posts
Debug Output

Hello,

it is not much but I hope anyone can read something out of it:

[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Enter
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> After handshake state= 3 Status= -6996
[0318:0008-1374] 11/06/2014 17:33:47.61 SSL_Handshake> Exit Status = -6996
[0318:0008-1374] 11/06/2014 17:33:47.61 int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]

Nov 10, 2014, 3:48 PM
94 Posts
Try configuring those other servers to not use SSLv2 or SSLv2 backwards compatibility mode...
Nov 7, 2014, 8:55 AM
2 Posts
I see the same error

I have the exact same issue with a Java application that needs to send mail through Domino using secure SMTP. After upgrading to 901FP2HF353 the SSL connection fails.

I've turned on DEBUG_SSL_HANDSHAKE=2 and DEBUG_SSL_CIPHERS=2 and this is the only output I get:

[31711:00008-1607153408] 11/07/2014 01:47:49.29 PM SMTP CIServ Listen> Connection Accepted on Port 465 for Session 097639AD
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM SSLInitContext> User is forcing 3079 cipher spec bitmask
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM SSL_Handshake> Enter
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM SSL_Handshake> After handshake state= 3 Status= -6996
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM SSL_Handshake> Exit Status = -6996
[31711:00027-1570965248] 11/07/2014 01:47:49.30 PM int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]

Does anyone know what error -6996 means?

 

Nov 10, 2014, 3:47 PM
94 Posts
Try configuring your java application to explicitly specify SSLv3 or TLS instead of SSLv2
All support for SSLv2 was removed as part of this hotfix, including SSLv2 backwards compatibility mode.


Unable to connect to patched Domino servers using SSLv2 backwards compatibility mode
http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2
Nov 7, 2014, 12:04 PM
2 Posts
APAR/SPR from IBM

I submitted this error to IBM and after some back and forth this is what I got:

This is to inform you that we have confirmed thru internal testing that there are functionalities that were broken by the new hotfix. I created a new APAR#LO82706 and SPR # MJTM9QMLDC set to severity 1 for this issue. Thank you for reporting this problem. I will keep you posted for updates from our development team. 

Nov 7, 2014, 5:25 PM
2 Posts
Interim Fix also breaks directory assistance connection over LDAPS

We installed this hot fix to today.  TLS worked fine when connecting to our Domino server.  However, our Domino server uses directory assistance to connect to an external LDAP server and this connection no longer worked.  IBM has said this issue will be fixed in 9.0.1 Fix Pack 3 which is due out in the first quarter of 2015.

Nov 10, 2014, 3:41 PM
94 Posts
Did you receive an SPR?
I'm not aware of any issues in this space, beyond the fact that all support for SSLv2 was dropped.

Unable to connect to patched Domino servers using SSLv2 backwards compatibility mode
http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2

This issue could be present for outbound connections as well if your LDAP server only supports SSLv2, as the patched Domino server will no longer use SSLv2 regardless of how it is configured.  
Nov 7, 2014, 5:25 PM
2 Posts
Interim Fix also breaks directory assistance connection over LDAPS

We installed this hot fix to today.  TLS worked fine when connecting to our Domino server.  However, our Domino server uses directory assistance to connect to an external LDAP server and this connection no longer worked.  IBM has said this issue will be fixed in 9.0.1 Fix Pack 3 which is due out in the first quarter of 2015.

Nov 10, 2014, 7:54 PM
57 Posts
SMTP inbound SMTP problems after Domino 9.0.1 FP2 IF1

I too am seeing inbound connection problems from some sending servers after updating to 9.0.1 FP2 IF1.  Senders who could send successfully before update immediately could not send after the update.

Enabling DEBUG_SSL_HANDSHAKE=2 and DEBUG_SSL_CIPHERS=2 resulted in log items such as Ronald Hoppe noted.:

[1990:0008-1A68] 11/10/2014 03:38:15 PM  SMTP Server: Remote host spring-chicken-ba.twitter.com (199.16.156.166) found in whitelist at PrivateWhitelist
[1990:0008-1A68] 11/10/2014 03:38:15 PM  SMTP Server: spring-chicken-ba.twitter.com (199.16.156.166) connected
[1990:0008-1A68] 11/10/2014 03:38:15.85 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[1990:0008-1A68] 11/10/2014 03:38:15.85 PM SSL_Handshake> Enter
[1990:0008-1A68] 11/10/2014 03:38:15.85 PM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[1990:0008-1A68] 11/10/2014 03:38:15.92 PM SSL_Handshake> After handshake state= 3 Status= -6996
[1990:0008-1A68] 11/10/2014 03:38:15.92 PM SSL_Handshake> Exit Status = -6996
[1990:0008-1A68] 11/10/2014 03:38:15.92 PM int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]
[1990:0008-1A68] 11/10/2014 03:38:16 PM  SMTP Server: spring-chicken-ba.twitter.com (199.16.156.166) disconnected. 0 message[s] received

This sequence occurs over and over as the sending server tries to repeatedly send email until it ultimately exceeds the fail time limit on their end.

If I disable the requirement for inbound SSL connections by changing the Server Configuration Setting "SSL negotiated over TCP/IP port" to "Disabled", then all inbound mail arrives successfully (albeit, unencrypted).  I've had to leave inbound SSL for SMTP disabled for now to keep mail flowing and customers and my employees from screaming.

BTW, the Domino server passes both inbound and outbound TLS SMTP tests from CheckTLS.com here: http://www.checktls.com/index.html as well as the HTTP server tests from Qualys (different protocol, but just to be thorough).

My cert is from GoDaddy, is still SHA-1, and is functioning and unchanged for the past 18 months.  I'm going to update to SHA-2, but lets address one thing at a time -- unless that is known to be the root problem.

I've also opened a Service Request from IBM, but have not yet received more than the initial auto-responder.

Nov 11, 2014, 1:36 PM
94 Posts
The IF removed all support for SSLv2
The IF removed all support for SSLv2, as described in the Notes/Domino wiki:

http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2

If setting DEBUG_SSL_ALL=1 on your Domino server results in a line like the following...

[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_RCV> 00000000: 80 65 01 03 01                                    '.e...'
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM S_Read> Exit, read 5 bytes
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_Handshake> After handshake state= 3 Status= -6996
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_Handshake> Exit Status = -6996
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]


... then the incoming connection is attempting to use SSLv2 compatibility mode and being rejected.

A "good" incoming SSLv3 or TLS connection will look like this instead:

[10846:00012-930277120] 11/11/2014 12:34:26.18 PM SSL_RCV> 00000000: 16 03 01 00 53                                    '....S'
[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Exit, read 5 bytes
[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Enter len = 83
[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Switching Endpoint to sync
[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Posting a nti_rcv for 83 bytes

If this is what you are seeing, then I would recommend upgrading or reconfiguring the clients to stop using SSLv2. If you're concerned about SSLv3, then you should be horrified by SSLv2.
Nov 11, 2014, 5:03 AM
7 Posts
SMTP inbound SMTP problems after Domino 9.0.1 FP2 IF1

Hi Mark

We have the same problem and also circumstances:
-same error logs
-same Domino Version: 9.0.1FP2 IF1 w64
-also a SHA1 certificate
-passed tests on CheckTLS vor in/outbound
-same configuration and "workaround" for Server Configuration Setting "SSL negotiated over TCP/IP port": Before problem =enabled, now, after IF1 =disabled.

We opened also a PMR last week and for the moment it seems that untrusted (self signed) sender certificates are the problem and responsible for this behavior (see below for last response from IBM):


********************************************************Begin of last PMR response********************************************************
Hi Cristian,

It seems that the issue connecting via port 465 secured SSL only happens to users with self signed certificates. Tests wherein the client application has  trusted certificate were successfully connected via secured SSL port-- sending and receiving email, proving that the hotfix is working fine.
Problems occur when the application (client) does not have all certs imported and trusted in the client certificate store.
So I would recommend checking which clients are affected by the issue and ensure that all the certs are imported and trusted.

If even after trying the above the issue persists I would recommend:
  1. enable Debug_SSL_All=1 on the gateway Domino server
   2. enable console logging on the same server
  3. Attempt a TLS smtp connection to reproduce the issue.

This issue is specifically TLS over SMTP so we will need a new PMR as it will have to go to the Messaging Team (there is a different APAR for this issue from the one in 91643,211,848).
So if the issue persists, I would recommend opening a new PMR mentioning that this is SPR # MJTM9QMLDC - APAR LO82706  and attaching the console logs from the gateway to the PMR.
In this way we should save time in data collection etc..
I hope this helps, let me know if there are any questions.
Thank you,
********************************************************End of last PMR response********************************************************

Now a few thing are not really clear to me, which why we posed this questions to IBM.....

1. Why should the fix working fine? I mean before of the fix (in my understanding the introduction of TLS 1.0 and the prevention to a "SSLv3 fallback")  untrusted/ self signed certificates worked very well! Is there a parameter we could set to accept untrusted/ self signed certificates?

2. In the configuration setting document for the gateway servers we set initially  "SSL negotiated over TCP/IP port = enabled" . In my understanding this means that SSL is not mandatory and if a SSL communication will not be possible/desired from the sender or accepted by us there will be a fallback to a nonSSL communication. Why is this not the case when the sender uses a selfsigned certifcate? It would not be an ideal solution, because self signed certificates would fallback to a nonSL communication, but it would be far than a rejection.

3. IBM wrote "It seems that the issue connecting via port 465 secured SSL only happens to users with self signed certificates."
We don't use the 465 port. We consulted our network team and they have confirmed that only port 25 is permitted by the firewall. In my opinion this is a valid configuration (see https://www-304.ibm.com/support/docview.wss?uid=swg21108352 / "....Also, Port 465 is no longer registered as SMTP-SSL. It has been deprecated in favor of TLS/SSL over port 25. For more information see: http://www.iana.org/assignments/port-numbers " . Is this fact relevant to IBMs explanation?

 

...will make a new post if some news are available and useful...

Nov 11, 2014, 1:44 PM
94 Posts
The IF removed all support for SSLv2
The IF removed all support for SSLv2, as described in the Notes/Domino wiki:

http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2

If setting DEBUG_SSL_ALL=1 on your Domino server results in a line like the following...


[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_RCV> 00000000:
80 65 01 03 01                                    '.e...'
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM S_Read> Exit, read 5 bytes
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_Handshake> After handshake state= 3 Status= -6996
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM SSL_Handshake> Exit Status = -6996
[30406:00011-233826048] 11/05/2014 06:36:49.27 PM int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]


... then the incoming connection is attempting to use SSLv2 compatibility mode and being rejected.


A "good" incoming SSLv3 or TLS connection will look like this instead:


[10846:00012-930277120] 11/11/2014 12:34:26.18 PM SSL_RCV> 00000000:
16 03 01 00 53                                    '....S'
[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Exit, read 5 bytes

[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Enter len = 83

[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Switching Endpoint to sync

[10846:00012-930277120] 11/11/2014 12:34:26.18 PM S_Read> Posting a nti_rcv for 83 bytes


If this is what you are seeing, then I would recommend upgrading or reconfiguring the clients to stop using SSLv2. If you're concerned about SSLv3, then you should be horrified by SSLv2.


Domino's support for self-signed certificates has not changed as part of this IF. You may see some web browsers react slightly differently to "untrusted" server certificates over TLS 1.0 than over SSLv3, but that will vary from client to client.
Nov 11, 2014, 8:27 AM
46 Posts
I was thinking the same as Cristian ...

... at point 3. 

 

I have a server using TLS/SSL over port 25 as stated on that document Cristian posted before. We are passing all checks on checktls.com, but we're using a GeoTrust certificate, not a self-signed one.

 

I didn't install the new HotFix yet, I would like to see more information about this. I don't want users screaming on me!

Nov 11, 2014, 10:49 AM
7 Posts
SMTP inbound SMTP problems after Domino 9.0.1 FP2 IF1

Hi Andres

We (recipient) also have a trusted certificate! The problems seems to be the self signed/untrusted certificate of the SENDER.

And btw....IBM remarked correctly it's a IF not a hotfix...my bad.

 

Nov 11, 2014, 1:43 PM
57 Posts
Re: SMTP inbound SMTP problems after Domino 9.0.1 FP2 IF1

Christian, thanks for posting your experience.   Regarding IBM's response to you:

It seems that the issue connecting via port 465 secured SSL only happens to users with self signed certificates. Tests wherein the client application has  trusted certificate were successfully connected via secured SSL port-- sending and receiving email, proving that the hotfix is working fine.
Problems occur when the application (client) does not have all certs imported and trusted in the client certificate store.
So I would recommend checking which clients are affected by the issue and ensure that all the certs are imported and trusted.

Okay, maybe.  But if you look at the log snippet I posted you'll see that the inbound sender is Twitter.  Obviously, it's possible that Twitter uses self signed certificates, but something makes me think the $27-billion company probably has real certs with a certificate authority on their SMTP servers.  Maybe not.  Regardless, Domino should accept self-signed certs to encrypt the stream anyway, since that is perfectly allowable.  IBM's own instructions for setting up a certificate for Domino servers even includes a section on using a self-signed cert.

With regards to your comment #2 about falling back to non-encrypted: there is a difference between being the sender and being the receiver.  The setting "SSL negotiated over TCP/IP port = enabled" tells Domino to try to negotiate inbound SMTP TLS/SSL, or will allow Domino to accept non-encrypted SMTP connections (setting it to "Required" disallows inbound non-encrypted connections, as I'm sure you know).  However, it is up to the sending SMTP server to initiate the retry using a non-encrypted connection if the SSL one fails.  It appears to me that some of the sending systems are not configured to do this, so they beat on the Domino server over and over, failing each time to negotiate an encrypted connection and not ever trying to send it unencrypted after failing.  And yet, if we set "SSL negotiated over TCP/IP" to "disabled" they are perfectly happy sending mail unencrypted and never even trying to negotiate a TLs connection.

Note: Domino has a NOTES.INI setting "RouterFallbackNonTLS=1" which applies to outbound mail connections and causes Domino to try sending non-encrypted if it is unable to send mail over encrypted connection.  If this is set to "0" then Domino will be one of those systems that won't retry a non-encrypted connection should the TLS connection fail or not be supported. IBM description for this setting here: http://www-10.lotus.com/ldd/dominowiki.nsf/dx/routerfallbacknontls

I agree with you that the recent fix is not "working fine".   There is no internet requirement that the certs be from an authority and not self-signed to send encrypted SMTP (the process is not being used to verify identity, only to create an encrypted SMTP stream).  If that is something IBM broke via the recent IF1 update, then they need to add a notes.ini setting to turn the requirement off that it won't accept self-signed certs.

Nov 11, 2014, 3:03 PM
57 Posts
Re: There were no changes to self-signed certs in this IF

Dave,

We've had SSLv2 not enabled in Domino for many years (i.e. on Server Doc, Internet Ports tab "Enable SSL V2" has been unchecked for as long as I recall it being available).  Does the change in the current IF remove functionality beyond this setting with regards to SSL v2?

Also, you refer to client browsers in some of your responses, but the problem that drove some of us to post SSL log snippets here is in regards to inbound SMTP only, not HTTP(S).

If this is what you are seeing, then I would recommend upgrading or reconfiguring the clients to stop using SSLv2. If you're concerned about SSLv3, then you should be horrified by SSLv2.


If we consider the "client" to be inbound SMTP server connections, I have zero ability to get Twitter (as a real example) to change how it sends mail.  I also have very limited leverage with our own clients who could no longer send us mail after we updated to 9.0.1 FP2 IF1.  Especially when I'm not 100% sure what is going on.  These clients (who are far larger than my firm) have said "we're not having any issues delivering mail to anybody but you.  It is your problem".

Several of the clients who could no longer send us mail happen to use the same third-party service Messagelabs/messagescreen.com, so that these clients' IT folks have directed me to their vendor -- which has not led to anything productive so far.  Again, the implication is that we're misconfigured because they are not having issues with anybody else.

In order to keep my sales staff and our clients happy -- in lieu of any conclusive evidence that it isn't our problem -- my only option at the moment is to set incoming SSL to "disabled".  We've had clients in the past who refused to send mail that way, but at the moment it is keeping more critical mail flowing than setting SSL inbound to Enabled and failing to accept connections from many sources for a yet to be determined reason.

Nov 11, 2014, 4:07 PM
94 Posts
What do you see for one of these incoming connections with DEBUG_SSL_ALL=1 set in the note...
Nov 11, 2014, 7:44 PM
57 Posts
With DEBUG_SSL_ALL=1 I see...

Log example for failing connections.  These did not fail before 9.0.1 FP2 IF1 and also don't fail now if "SSL Negotiated over TCP/IP port" is set to "1".  SSL V2 has not been enabled on the server for years:

[167C:0008-0FB4] 11/11/2014 04:16:55 PM  SMTP Server: zoniac3.nmsrv.com (204.187.13.193) connected
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM SSL_Handshake> Enter
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM S_Read> Enter len = 5
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM S_Read> Switching Endpoint to sync
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM S_Read> Posting a nti_rcv for 5 bytes
[167C:0008-0BEC] 11/11/2014 04:16:55.80 PM SSL_RcvSetup> SSL not init exit
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM S_Read> Switching Endpoint to async
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM S_Read> nti_done return 5 bytes rc = 0
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM SSL_RCV> 00000000: 80 8F 01 03 01                                    '.....'

[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM S_Read> Exit, read 5 bytes
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM SSL_Handshake> After handshake state= 3 Status= -6996
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM SSL_Handshake> Exit Status = -6996
[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM int_MapSSLError> Mapping SSL error -6996 to 4166 [SSLProtocolErr]
[167C:0008-0BEC] 11/11/2014 04:16:55 PM  SMTP Server: zoniac3.nmsrv.com (204.187.13.193) disconnected. 0 message[s] received

IBM support escalated the PMR for this to severity 1 today based on prior logs sent previously, which look just like above.

Nov 11, 2014, 10:06 PM
94 Posts
That's an SSLv2 message
That's an SSLv2 handshake message offered up by the client for backwards compatibility with servers that only support SSLv2. You can tell by the first byte -- it's 0x80, not 0x16.

I've expanded the documentation to include the notes.ini and console messages to help people diagnose this problem.  This is not a bug -- use of SSLv2, including backwards compatibility mode, has been prohibited since 2011 by RFC 6176.

http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2

[167C:0008-0BEC] 11/11/2014 04:16:55.84 PM SSL_RCV> 00000000: 80 8F 01 03 01                  

Nov 11, 2014, 8:30 PM
57 Posts
(more) With DEBUG_SSL_ALL=1 I see...

Dave,

This is what I see for a "successful" inbound SSL/TLS SMTP connection.  At least I think it is a successful. I don't know enough about the log format to be sure it was encrypted or failed and fell back to unencrypted.  It is waaaay longer than the example you posted for a successful SSL connection.  I snipped the middle of it this to keep it half way reasonable.  This is an inbound SMTP connection from Hotmail.

[1AEC:000A-12E8] 11/11/2014 05:04:04 PM  SMTP Server: bay004-omc4s17.hotmail.com (65.54.190.219) connected
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> nti_done return 5 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000000: 16 03 01 01 06                                    '.....'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Exit, read 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Enter len = 262
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Posting a nti_rcv for 262 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> nti_done return 262 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000000: 10 00 01 02 01 00 BF 06 63 44 E7 A1 37 15 7C B3   '......?.cDg!7.|3'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000010: F3 C9 83 49 79 C1 05 83 17 BB 25 1B 1F 3B 1A D9   'sI.IyA...;%..;.Y'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000020: BD 66 73 EC 0F DC 1A F3 C1 D0 BF 2C 8A BF C5 DB   '=fsl.\.sAP?,.?E['
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000030: 07 19 42 DE 92 7A E2 A4 E8 2F 83 CA 26 3F 87 D1   '..B^.zb$h/.J&?.Q'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000040: DB 3C E8 46 8F 3C CB B4 01 14 F1 21 26 8B E2 D2   '[<hF.<K4..q!&.bR'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000050: CA 6D DA D1 9B DF 8F A9 8E F2 5B EE F2 A4 67 6F   'JmZQ._.).r[nr$go'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000060: 5D F7 BA 70 67 3A B4 8E A6 07 3D 68 46 B1 23 F5   ']w:pg:4.&.=hF1#u'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000070: DE EB 88 17 51 70 30 85 19 E6 48 ED B6 C3 D3 85   '^k..Qp0..fHm6CS.'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000080: 2F B6 4F E0 91 41 12 10 19 A8 43 37 37 66 E5 2F   '/6O`.A...(C77fe/'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000090: 3D 9D 2A 4C A1 79 2E F1 9C 45 95 64 5F 2D 7A AB   '=.*L!y.q.E.d_-z+'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000A0: 0D EA 50 B3 46 F8 40 98 D3 CA E2 A8 F2 9E 6E B3   '.jP3Fx@.SJb(r.n3'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000B0: AB DE 51 72 AB C3 B9 31 3B 83 0F 02 7D 83 B7 60   '+^Qr+C91;...}.7`'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000C0: 9E 7A 83 CC DC 05 AA EB 1F 96 6E F0 E7 3E D8 0A   '.z.L\.*k..npg>X.'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000D0: FF 3A 99 AE 06 FA 40 F0 F7 63 9B E0 B3 3E CE 7D   '.:...z@pwc.`3>N}'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000E0: FC AE 99 E4 CF 34 71 BA 80 EF 52 93 AA EA 9D 28   '|..dO4q:.oR.*j.('
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 000000F0: 55 FA C4 29 28 13 4F AF D2 A3 5D 3B B7 70 7C 7D   'UzD)(.O/R#];7p|}'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000100: 1D AB C8 20 9B D1                                 '.+H .Q'
[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM S_Read> Exit, read 262 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> After handshake2 state 13
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Exit Status = -5000
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Enter
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Current Cipher 0x0035 (RSA_WITH_AES_256_CBC_SHA)
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Enter len = 5
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Posting a nti_rcv for 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> nti_done return 5 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000000: 14 03 01 00 01                                    '.....'
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Exit, read 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Enter len = 1
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Posting a nti_rcv for 1 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> nti_done return 1 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000000: 01                                                '.'
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Exit, read 1 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> After handshake2 state 14
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Exit Status = -5000
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM int_MapSSLError> Mapping SSL error -5000 to 4176 [SSLHandshakeNoDone]
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Enter
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_Handshake> Current Cipher 0x0035 (RSA_WITH_AES_256_CBC_SHA)
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Enter len = 5
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Posting a nti_rcv for 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> nti_done return 5 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000000: 16 03 01 00 30                                    '....0'
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Exit, read 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Enter len = 48
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Posting a nti_rcv for 48 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM S_Read> nti_done return 48 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000000: DD C0 05 7E F8 F3 58 FD 6F 7F 43 21 4E 21 74 E3   ']@.~xsX}o.C!N!tc'
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000010: DE 98 BC AD 20 4A 3C 2A 2D AB 43 D5 94 DA 02 AC   '^.<- J<*-+CU.Z.,'
[1AEC:000A-1C08] 11/11/2014 05:04:04.30 PM SSL_RCV> 00000020: 1F 9E 41 2C 9E 3C 1F AB B4 07 6C 0F 91 FA 4F 69   '..A,.<.+4.l..zOi'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Read> Exit, read 48 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Enter len = 6
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Xmt> 00000000: 14 03 01 00 01 01                                 '......'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Posting a nti_snd for 6 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_EncryptData> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_EncryptDataCleanup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> nti_done return 6 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Exit, wrote 6 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Enter len = 53
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Xmt> 00000000: 16 03 01 00 30 77 12 94 B2 E3 02 10 24 4F 80 B1   '....0w..2c..$O.1'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Xmt> 00000010: 23 4E 73 35 AD 77 75 94 32 D8 22 B0 1E 95 9A F0   '#Ns5-wu.2X"0...p'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Xmt> 00000020: D8 AD E7 45 87 F5 26 59 A6 50 EB 54 7F 22 A0 41   'X-gE.u&Y&PkT." A'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Xmt> 00000030: DD C8 68 86 AB                                    ']Hh.+'
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Switching Endpoint to sync
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Posting a nti_snd for 53 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_EncryptData> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Switching Endpoint to async
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_EncryptDataCleanup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> nti_done return 53 bytes rc = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_Write> Exit, wrote 53 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> After handshake2 state 3
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Protocol Version = TLS1.0 (0x301)
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> KeySize = 256 bits
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Current Cipher = 0x0035 (RSA_WITH_AES_256_CBC_SHA)
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> SSLErr = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> TLS/SSL Handshake completed successfully
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Exit Status = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_ReadGetBytesNeeded> Need to 5 more bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_ReadAsync> Enter len = 5
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_ReadAsync> Posting a nti_rcv for 5 bytes
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_RcvSetup> SSL not init exit
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM S_ReadAsync> Exit
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_Poll> Enter
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_Poll> Rcv has completed, adding 5 to Rcv Ring
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_Poll> Rcv has completed, RingFreeSize 32766
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_Poll> Current rcv ring data size is: 5
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_ReadGetBytesNeeded> Removing 5 from the Rcv Ring
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_ReadGetBytesNeeded> Need to 64 more bytes
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM S_ReadAsync> Enter len = 64
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM S_ReadAsync> Posting a nti_rcv for 64 bytes
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM SSL_RcvSetup> SSL not init exit
[1AEC:0007-1C08] 11/11/2014 05:04:04.57 PM S_ReadAsync> Exit

<snip: this sort of thing repeated 11 more times.  If the whole thing is needed to make sense, let me know...>


[1AEC:000A-1C08] 11/11/2014 05:04:05.44 PM S_ReadAsync> Exit
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_Poll> Enter
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_Poll> Rcv has completed, adding 5 to Rcv Ring
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_Poll> Rcv has completed, RingFreeSize 32766
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_Poll> Current rcv ring data size is: 5
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_ReadGetBytesNeeded> Removing 5 from the Rcv Ring
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_ReadGetBytesNeeded> Need to 32 more bytes
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM S_ReadAsync> Enter len = 32
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM S_ReadAsync> Posting a nti_rcv for 32 bytes
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM SSL_RcvSetup> SSL not init exit
[1AEC:0007-1C08] 11/11/2014 05:04:05.46 PM S_ReadAsync> Exit
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM SSL_Poll> Enter
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM SSL_Poll> Rcv has completed, adding 32 to Rcv Ring
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM SSL_Poll> Rcv has completed, RingFreeSize 32766
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM SSL_Poll> Current rcv ring data size is: 32
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM SSL_ReadGetBytesNeeded> Removing 32 from the Rcv Ring
[1AEC:000A-12E8] 11/11/2014 05:04:05 PM  SMTP Server: Message 0005DE20 (MessageID: <BAY175-W512D25CA614D4A2F4778B0D98E0@phx.gbl>) received from bay004-omc4s17.hotmail.com (65.54.190.219) size 1673 bytes
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM CompleteNTIRequest> Enter
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM CompleteNTIRequest> Event = 4 Status = 0 Bytes = 6
[1AEC:0009-12E8] 11/11/2014 05:04:05.58 PM CompleteNTIRequest> Exit
[1AEC:000A-12E8] 11/11/2014 05:04:05.58 PM SSL_EncryptData> Asked to write 64 and wrote 101
[1AEC:000A-1C08] 11/11/2014 05:04:05 PM  SMTP Server: bay004-omc4s17.hotmail.com (65.54.190.219) disconnected. 1 message[s] received

Nov 11, 2014, 10:12 PM
94 Posts
Yes, that is a sucessful TLS 1.0 handshake
Snipping the relevant portions of the log:

That same initial message begins with 0x16 (type = handshake message), then 0x0301 (TLS 1.0), then 0x0106 (length of the TLS record)

[1AEC:000A-1C08] 11/11/2014 05:04:04.27 PM SSL_RCV> 00000000: 16 03 01 01 06                                    '.....'

Searching that log for "success" will find the end of the successful handshake, and tell you what version of SSL/TLS was used:

[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Protocol Version = TLS1.0 (0x301)
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> KeySize = 256 bits
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Current Cipher = 0x0035 (RSA_WITH_AES_256_CBC_SHA)
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> SSLErr = 0
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> TLS/SSL Handshake completed successfully
[1AEC:000A-1C08] 11/11/2014 05:04:04.32 PM SSL_Handshake> Exit Status = 0

Nov 12, 2014, 12:24 PM
2 Posts
Senders who won't "talk" to this HF

I am having trouble receiving mail from these senders. I spoke to one of them, and they say they use "opportunistic" TLS, but it won't connect nor fallback to clear text.

Here are two of these senders. Does anyone have others?

 

Symantec Messaging Gateway

Reflexion.net  TLS connect failed: connection reset; connected to ..... STARTTLS proto=TLSv1; cipher=(NONE).

Nov 12, 2014, 2:46 PM
57 Posts
Re: Yes, that is a sucessful TLS 1.0 handshake

Thanks, Dave.

 So, with IF1 and SSL enabled,  I do get successful, encrypted inbound SMTP connections from senders such as Hotmail.  But, there are others that now fail (such as inbound mail connections from Twitter).  The senders that are failing after IF1 had no problem sending prior to the update.  Others could not send us mail prior to the IF1 update -- presumably because they had configured their server to send using TLS 1.0+ and Domino only supported up to SSL 3.  These "others" worked fine after the IF1 update, but it broke different senders, such as Twitter.  THAT is the heart of the problem.

The only way for me to receive 99.9%+ of mail at the moment is to disable the requirement for encryption.  We've had high-security clients in the past who would not send without encryption, and if we encounter another one of those right now I'll be unable to set the Domino configuration in a way that satisfies the conflicting requirements.

Nov 24, 2014, 2:09 AM
57 Posts
same error here

Hi there,

I installed the fix on our 9.0.1FP2 server (with a self signed certificate), enabled the parameters SSL_ENABLE_INSECURE_RENEGOTIATE=1, DEBUG_SSL_HANDSHAKE=2 and DEBUG_SSL_CIPHERS=2

We let the server run for about an hour. The TLS was working but we got the error "Mapping SSL error -6996 to 4166 [SSLProtocolErr]" on some domains.

And some of these domains are important clients/partners.

So, like Matt, we have disabled TLS so that we can receive most of our emails.

 

Matt also wrote this: "However, it is up to the sending SMTP server to initiate the retry using a non-encrypted connection if the SSL one fails."

Is there any official documentation/standard specifying this?(Maybe an RFC)

And could someone please provide us with a link to it, so that we can send it to our clients/partners and encourage them to check their email setup.

 

I would also like to know what IBM intends to do about this?

Do you intend to fix it on the receiver side or is it not in your plans because it is the sender's responsibility to retry using an unencrypted connection?

 

Also, in my personal experience, only one of our clients requires TLS to send us emails while most do not.

Wouldn't it be possible to specify the domains or server IPs that require TLS somewhere (maybe in notes.ini) such that when they connect, domino uses TLS, but for remaining servers, Domino does not use any encryption?

 

Kind regards.

Nov 24, 2014, 2:01 PM
57 Posts
Update: Inbound SMTP TLS/SSL handshake problem

I've been working with IBM regarding this issue via a PMR for the past two weeks.  There is no conclusive resolution as of this moment.

I'm not going to give all the details, but I've gone back and forth many times, sent multiple log files after trying various suggested notes.ini/configuration changes, etc, etc, etc.  These have been analyzed, presumably, but no fix.

There was speculation that our certificate -- which was SHA-1, because until just this recent release SHA-2 certs were not supported -- might have been the issue.  So, I updated the certificate to SHA-2, and still no joy with certain inbound SSL handshakes failing.

NOTE:  for those of you who have GoDaddy for your SSL, switching to a SHA-2 cert was relatively painless and cost free.  I first updated our cert from SHA-1 to SHA-2, which is an option in the GoDaddy SSL management screen.  Then, realizing that this "new" cert would be based on the original CSR and old private keys, which were SHA-1, I used the option in GoDaddy to rekey the certificate as if the keys were lost.  I generated a new CSR using the arcane and obtuse command-line method outlined by IBM support (here: http://www-10.lotus.com/ldd/dominowiki.nsf/dx/3rd_Party_SHA-2_with_OpenSSL_and_kyrtool?open) and got a full SHA-2 certificate with new keys from GoDaddy.  Again, no additional cost; same expiration date for cert as the old one.  The cert works fine and tests fine using the Qualys SSL HTTP test AND the CheckTLS test for SMTP (http://www.checktls.com/index.html).  BTW, I have yet to find a way to get the SHA-2 cert into the "IBM HTTP Server" transparent proxy I have configured to sit in front of Domino, only into Domino itself.  This whole thing is a convoluted PITA.

As mentioned, this did not fix the SMTP handshake issue with certain inbound senders, which still error with:

Mapping SSL error -6996 to 4166 [SSLProtocolErr]

I think the issue might be related to this from Symantec: "Symantec Messaging Gateway may fail to connect to mail servers utilizing strict TLS version 1 transport encryption"
http://www.symantec.com/business/support/index?page=content&id=TECH215003

Regardless, IBM needs to fix this so that Domino can reliably accept mail, encrypted, from as many sources as possible.  If gmail, Yahoo mail, and hotmail accept connections from the sources in question, then so must our company.  If Domino can't do it, then it is Domino that needs fixing, regardless of any "strict" reading of RFCs or whatever.  I can't not accept mail from the sources in question (some of whom are clients), and I have no power to get these organizations to change what they are doing.  Their response is "we're not having any problems sending mail to anybody except to you; the problem is on your end."  And I agree with them.

As for "Nad Surf"'s question regarding why connections don't fall back to unencrypted after the encryption handshake fails, it is my understanding that if the sending server is configured to send unencrypted, it will typically only do so if the receiving server indicates it doesn't support SSL/TLS at all (i.e. it doesn't offer STARTTLS during the connection negotiation).  If the receiving server indicates it DOES support encryption and gives a STARTTLS, and if the two servers cannot agree on a way to encrypt the connection, then the failure does not trigger a plain text send as an alternative.  The connection simply fails, and the sender probably tries it over and over until it times out and returns the mail.

It would be nice if encryption handshake failures did fall back to plain (if configured on the server), because I've yet to find an inbound connection that fails the encryption handshake yet is unwilling to forgo encryption completely and deliver mail in the clear if I turn off inbound encryption on Domino.  No, I don't have an RFC reference.  This is my experience.

The root problem I'm assuming is that the "fix" given to us by IBM strictly supports only TLS 1.0, not 1.1 or 1.2.  This is not acceptable.  TLS 1.1 was defined by RFC in 2006, going on nine years ago.  TLS 1.2 was defined nearly seven years ago, in 2008. (https://en.wikipedia.org/wiki/Transport_Layer_Security#TLS_1.1).  These are NOT new standards.

Our choices are:

A) don't patch/update Domino and remain vulnerable to POODLE, etc,  You'll have increasing issues from visitors of all kinds due to supporting SSL 3.  This isn't a real option.
B) update to 9.0.1 FP2 IF1 and incompletely support the current TLS standards -- which have been available for going on a decade
C) update Domino but disable SMTP inbound SSL/TLS encryption so that you don't miss critical mail from certain senders (twitter is one off them, BTW)

My "solution" to reliably receiving inbound mail -- which is all users and management is really interested in -- is option "C".  Obviously, this isn't really a "solution".

 SMTP inbound TLS on Domino is incomplete/broken as currently offered.  It only satisfies a lawyer's interpretation of "we have given the clients a solution to the problem", and cannot be used for inbound SMTP by organizations in the real world without the risk of rejecting significant legitimate mail.

Nov 25, 2014, 1:16 PM
108 Posts
Option D
FWIW
We decided to go for yet another option. I.e., we put Domino behind a reverse proxy (nginx), which gives us state-of-the-art SSL/TLS support. Of course that approach has its caveats, too. Some have been discussed in this forum and elsewhere. Another big downside we stumbled across was the fact that SPAM control (previously Domino-based) was no longer functioning as desired. We had to set up Linux-based SPAM control from scratch -- a steep learning curve if you've never done that before. Bottom line: This whole mess (my apologies for the harsh word) cost us a LOT of time and resources.
Nov 25, 2014, 8:04 AM
57 Posts
Thanks a lot Mark

Thanks a lot Mark

Dec 1, 2014, 6:34 AM
5 Posts
Inbound Domino SMTP TLS/SSL handshake problem

Hi, Mark and Dave, we are having the same handshake problem, including the -6996 error. HTTPS is fine, the only problem we see is with ESMTP STARTTLS.

In our case, our own LPMS/xmail server couldn't send any mail to the Domino server after applying the POODLE interim fix, until we disabled STARTTLS on the Domino side. The handshake is failing, and the xmail server can't or won't fall back to unencrypted, although it is configured to do so.  Something in the Domino SMTP response to "STARTTLS" causes a failure so bad, that our xmail won't recover and considers it a 417 Temporary delivery error.

I think Mark was on to something with the Symantec technote about SSLv2Hello.

I have access to both servers. What should I try?

Dec 8, 2014, 8:35 PM
57 Posts
IBM response to support issue about TLS and SMTP

For those of you who've been following, I've had many back and forths with IBM about the failure of inbound secure SMTP connections after implementing TLS 1.0.  Their latest reply was to say they removed all SSLv2 support when the TLS hotfix was rolled out as per http://www-10.lotus.com/ldd/dominowiki.nsf/dx/SSLv2

If you look at the RFC's cited, however, you'll note that SSLv2 CLIENT-HELLO message formats CAN be accepted by servers that only support TLS.  It is perfectly legal.  The way they are handled is constrained, and the server isn't allowed to actually negotiate SSLv2 traffic, but the connection attempt and hello are NOT supposed to be rejected out of hand.

 

Here is my full email response to IBM support.  We'll see what happens.

--------------------------------------------------

My interpretation of this response is that it says my Domino server is working as designed given the Fix Pack, and Interim Fix it is running.

I was already aware of the referenced information with regards to SSLv2 Handshake Messages.  If this is the rationale being used to explain why my server can't accept various secured incoming SMTP connections (so that it, ironically, must run SMTP in the clear to function), I believe the RFC cited is possibly being misinterpreted.  From RFC 6167 in the section "3. Changes to TLS" it says:

3.  Changes to TLS

   Because of the deficiencies noted in the previous section:

   o  TLS clients MUST NOT send the SSL version 2.0 compatible CLIENT-
      HELLO message format.  Clients MUST NOT send any ClientHello
      message that specifies a protocol version less than
      { 0x03, 0x00 }.  As previously stated by the definitions of all
      previous versions of TLS, the client SHOULD specify the highest
      protocol version it supports.

   o  TLS servers MAY continue to accept ClientHello messages in the
      version 2 CLIENT-HELLO format as specified in RFC 5246 [TLS1.2],
      Appendix E.2.  Note that this does not contradict the prohibition
      against actually negotiating the use of SSL 2.0.

   o  TLS servers MUST NOT reply with an SSL 2.0 SERVER-HELLO with a
      protocol version that is less than { 0x03, 0x00 } and instead MUST
      abort the connection, i.e., when the highest protocol version
      offered by the client is { 0x02, 0x00 }, the TLS connection will
      be refused.


Yes, clients are not supposed to send SSLv2 compatible CLIENT-HELLO message format.  But, as stated in the second bullet point the server can continue to accept these SSLv2 CLIENT-HELLO message formats so long as it doesn't negotiate a SSLv2 connection.  The server should respond with an allowed protocol version AND it should only abort the connection when the highest protocol version offered by the client is insufficient.

So SSLv2 CLIENT-HELLOs are not supposed to be simply rejected out of hand, but the response type is restricted and the connection refused only if the client doesn't offer a high enough protocol version.

It appears to me from the IBM statement that possible TLS 1.0 connections are being refused simply because the client sent a SSLv2 CLIENT-HELLO, something that is allowed by the RFC.

I'd also like to reference RFC 5246 Appendix E which says, among other things:

   Note: some server implementations are known to implement version
   negotiation incorrectly.  For example, there are buggy TLS 1.0
   servers that simply close the connection when the client offers a
   version newer than TLS 1.0.  Also, it is known that some servers will
   refuse the connection if any TLS extensions are included in
   ClientHello.  Interoperability with such buggy servers is a complex
   topic beyond the scope of this document, and may require multiple
   connection attempts by the client.

AND

   However, even TLS servers that do not support SSL 2.0 MAY accept
   version 2.0 CLIENT-HELLO messages.

So, what am I dealing with here?  Since currently IBM Domino 9.0.1, even when fully patched, only supports up to TLS 1.0, is it one of these "buggy TLS 1.0 servers" as described in RFC 5246?

As you know, I currently have little choice but to run SMTP without any encryption at all because if the current TLS 1.0 encryption is turned on I can't accept secure SMTP server connections from a variety of clients of ours -- as well as that little $23-billion company named Twitter.

Is the conclusion here that Twitter only sends mail with SSLv2?  Or is it that such senders support higher encryption but that my Domino server cuts the connection when it sees an SSLv2 CLIENT-HELLO, which actually is allowed by RFC?

Dec 9, 2014, 4:06 PM
57 Posts
Re. See this - by Darren Duke

So, the current "solution" is to disable the ciphers that are supposed to be the more secure?  Googling around about ciphers gives info that says it's the RC4 ciphers that should be disabled (in general, not related to the current TLS Poodle thing):

http://blog.cloudflare.com/killing-rc4-the-long-goodbye/

We recently removed support for RC4 for browsers using TLS 1.1+. Now we are removing RC4 as the preferred cipher. Servers behind CloudFlare will prefer AES-based cipher suites...

http://www.acunetix.com/blog/articles/tls-ssl-cipher-hardening/

Furthermore, it is also crucial to disable weak ciphers. Weak ciphers such as DES and RC4 should be disabled. Using current technology, DES can be broken in a few hours while RC4 has been found to be weaker than was previously thought. While it may have been advised to use RC4 to mitigate BEAST attacks in the past, given the latest attacks on the RC4 cipher, Microsoft has issued an advisory again its use.

 

My conclusion:  ALL ciphers should be completely disabled to maximize security. </sarcasm off>

Dec 9, 2014, 4:29 PM
329 Posts
Exactly!

Which is exactly where I was when I asked what ciphers people recommended for use:  http://www-10.lotus.com/ldd/ndseforum.nsf/xpTopicThread.xsp?documentId=D804457CF4E7894B85257DA100787ABA

I'd (previously) disabled the weaker ciphers, then for the most recent issues disabled AES, then saw a post about Triple DES, then after posting found similar posts to what you mentioned about RC4.  I decided to go running RC4 for now as I found (similar to Darren Duke's blog) that by disabling 3DES, it'd raise my grade from 'F' to 'B'.

But it still doesn't look good.

I knew this InterWeb thing was just a fad, and it wasn't here to stay!

;-)

Rumor has it that there MAY be more ciphers to choose from someday down the line, but don't hold your breath.

 

Dec 9, 2014, 5:53 PM
46 Posts
FWIW, nginx proxy will give you an "easy A"
Dec 9, 2014, 5:22 PM
46 Posts
The only thing that can be said is that which has been said
Security is mostly a superstition. It does not exist in nature, nor do the children of men as a whole experience it. Avoiding danger is no safer in the long run than outright exposure. Life is either a daring adventure, or nothing.
Helen Keller, The Open Door (Doubleday, 1957)
Dec 9, 2014, 7:04 PM
57 Posts
Re. FWIW, nginx proxy will give you an "easy A"

And when nginx has an unpached security issue, you can put a different tranparent proxy in front of that.  It'll be transparent proxies all the way down.  :-)

 

The core functions of Domino should be engineered to be as secure as nginx (or whatever) for the services Domino provides.  If you want a transparent proxy for administrative advantages or to provide functionality beyond the core functionality of Domino, fine, that's cool.  But it should not be a requirement to simply make Domino work safely performing the ordinary tasks it is designed to do.

 

Love the quote from Helen Keller.

 

Dec 9, 2014, 7:04 PM
46 Posts
Re. FWIW, nginx proxy will give you an "easy A"
"The core functions of Domino should be engineered to be as secure as nginx (or whatever) for the services Domino provides."

-------------

Couldn't agree more. Lotus Notes was doing encryption before most products out there were in diapers. If it was now like it was then, we would all be chuckling when these poodle vulnerabilities were announced.

Dec 10, 2014, 6:10 AM
108 Posts
Agreed. That said, I still prefer to use a reverse proxy for the time being.
I.e., until it becomes evident that Domino *really* has caught up.
Dec 11, 2014, 5:06 AM
5 Posts
Domino IHS Workaround: strict padding

Has anybody with Windows Domino & IHS tried this workaround setting to enforce "strict CBC padding" yet?

http://www-01.ibm.com/support/docview.wss?uid=swg21692502

Workarounds and Mitigations

For all versions and releases of Apache based IBM HTTP server, IBM recommends enabling strict CBC padding enforcement. Add the following directive to the httpd.conf file to disable SSLv3 and SSLv2 for each context that contains "SSLEnable": 

# Enable strict CBC padding 
SSLAttributeSet 471 1

Dec 21, 2014, 10:44 AM
18 Posts
fix released

The latest interim fixes released on Friday fix the POODLE/TLS problem now.

 

Feb 17, 2015, 5:07 AM
7 Posts
LMES9QRUZY in 9.0.1 FP3 IF1

Concerning LMES9QRUZY: Can anyone confirm that? We implemented the IF1 for FP3 and we get now a different, more specific, "error message" but it still doesn't work. The interesting thing is we get now the following message in the debug log:

This is probably an SSLv2 ClientHello record which is not supported by default to improve "out of the box" security.  ...So by default it is not supported, how can I switch this functionality on...is there maybe a notes.ini definition for it?

 

*****************************************
15.02.2015 20:42:43.19 [0EE4:000A-0E88] int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
15.02.2015 20:42:43.19 [0EE4:000A-0E88] SSL_Handshake> Enter
15.02.2015 20:42:43.19 [0EE4:000A-0E88] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
15.02.2015 20:42:43.19 [0EE4:000A-0E88] S_Read> Enter len = 5
15.02.2015 20:42:43.19 [0EE4:000A-0E88] S_Read> Switching Endpoint to sync
15.02.2015 20:42:43.19 [0EE4:000A-0E88] S_Read> Posting a nti_rcv for 5 bytes
15.02.2015 20:42:43.19 [0EE4:000A-0E88] SSL_RcvSetup> SSL not init exit
15.02.2015 20:42:43.21 [0EE4:000A-0E88] S_Read> Switching Endpoint to async
15.02.2015 20:42:43.21 [0EE4:000A-0E88] S_Read> nti_done return 5 bytes rc = 0
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSL_RCV> 00000000: 80 77 01 03 01                                    '.w...'
15.02.2015 20:42:43.21 [0EE4:000A-0E88] S_Read> Exit, read 5 bytes
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSLReadRecord> Rejecting connection - record contentType not in range for SSLv3 or TLS
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSLReadRecord> First 4 bytes of SSL/TLS record: 0x80 0x77 0x01 0x03
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSLReadRecord> This is probably an SSLv2 ClientHello record which is not supported by default to improve "out of the box" security
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSLReadRecord> See the SSLv2 page on the Notes/Domino wiki for more information.
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSL_Handshake> After handshake state= 3 Status= -6974
15.02.2015 20:42:43.21 [0EE4:000A-0E88] SSL_Handshake> Exit Status = -6974
15.02.2015 20:42:43.21 [0EE4:000A-0E88] int_MapSSLError> Mapping SSL error -6974 to 4171 [** unknown **]
15.02.2015 20:42:43   SMTP Server: ServerName_deleted (ip_deleted) connected
15.02.2015 20:42:43   SMTP Server: ServerName_deleted (ip_deleted) disconnected. 0 message[s] received

*********************************************

Feb 17, 2015, 8:22 AM
9 Posts
LMES9QRUZY in 9.0.1 FP3 IF1

Unfortunately I can confirm your research. After applying IF1 we get the same error message. Furthermore we must disable SMTP inbound encryption so that we don't miss critical mail.

 

Feb 18, 2015, 7:08 AM
7 Posts
notes.ini parameter for SPR LMES9QRUZY

I got right now following notes.ini parameter concerning the SSLv2 ClientHello problem and SPR LMES9QRUZY from IBM support.

Please use the notes.ini parameter: SSL_ENABLE_INSECURE_SSLV2_HELLO=1

Have to wait for a service time slot to try it, will provide some feedback later on ....

 

Feb 18, 2015, 9:18 AM
9 Posts
YES, TLS handshake now successful

This notes.ini parameter was the missing link.

Handshake conversation now starts like

18.02.2015 14:59:17,89 [19E0:000A-19D8] int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSL_Handshake> Enter
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSLReadRecord> Reading an insecure SSLv2 record by administrator request
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSL2ReadRecord> Reading an insecure SSLv2 record by administrator request
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSLProcessProtocolMessage> Record Content: 0
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSLProcessProtocolMessage> Received an insecure SSLv2 record; processing by administrator request
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSL2ProcessMessage> Message: 1
18.02.2015 14:59:17,89 [19E0:000A-19D8] SSL2ProcessClientHello> Processing SSLv2 ClientHello message requesting TLS1.0 (version 0x0301)

resulting to successful handshake

18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> After handshake2 state 3
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> Protocol Version = TLS1.0 (0x301)
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> KeySize = 128 bits
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> Current Cipher = 0x002F (RSA_WITH_AES_128_CBC_SHA)
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> SSLErr = 0
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> TLS/SSL Handshake completed successfully
18.02.2015 14:59:18,08 [19E0:000A-19D8] SSL_Handshake> Exit Status = 0
18.02.2015 14:59:18,08 [19E0:000A-19D8] int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]

Feb 18, 2015, 9:32 AM
7 Posts
YES, TLS handshake now successful

@Gilbert: Hallelujah! Thank you very much for sharing your results! We will implement it asap!

Feb 18, 2015, 10:56 AM
1 Posts
Problem after upgrading from 8.5.3 FP3 to 9.0.1 FP3

I have this same problem after upgrading to 9.0.1 FP3 and injecting the NOTES.INI parameter had no effect. Anyone has been through this?

 

Feb 19, 2015, 8:41 AM
7 Posts
YES, TLS handshake now successful

As Gilbert wrote, also we can confirm it: It works!

The console log are the same as the ones from Gilbert. Great!

Feb 24, 2015, 7:38 AM
46 Posts
TLS 1.2

I'm having this problem with a client that is using TLS 1.2. Our Domino is set to use TLS 1.0 (v9.0.1 FP3IF1), of course. This is the extract of the log

 

[0B4C:000B-0F50] 02/23/2015 07:09:48.30 PM SSLCheckCertChain> Valid certificate chain received
[0B4C:000B-0F50] 02/23/2015 07:09:48.30 PM int_MapSSLError> Mapping SSL error 0 to 0 [SSLNoErr]
[0B4C:000B-0F50] 02/23/2015 07:09:48.30 PM SSL_Handshake> Enter
[0B4C:000B-0F50] 02/23/2015 07:09:48.30 PM SSL_Handshake> Current Cipher 0x0000 (Unknown Cipher)

[0B4C:000B-0F50] 02/23/2015 07:09:48.44 PM SSLReadRecord> Rejecting connection - record contentType not in range for SSLv3 or TLS
[0B4C:000B-0F50] 02/23/2015 07:09:48.44 PM SSLReadRecord> First 4 bytes of SSL/TLS record: 0x80 0xB3 0x01 0x03
[0B4C:000B-0F50] 02/23/2015 07:09:48.44 PM SSLReadRecord> This is probably an SSLv2 ClientHello record which is not supported by default to improve "out of the box" security
[0B4C:000B-0F50] 02/23/2015 07:09:48.44 PM SSLReadRecord> See the SSLv2 page on the Notes/Domino wiki for more information.

 

I'm going to try that notes.ini parameter this afternoon, but I was wondering if it won't represent any future security issue, since the initial idea was to not support SSLv2 and v3 anymore.

 

 

EDIT 02/25/2015: It worked using SSL_ENABLE_INSECURE_SSLV2_HELLO=1

 

I still don't know if this parameter may represent any kind of vulnerability. If anyone can shed some light on this matter, I'll be most grateful


This forum is closed to new posts and responses. New discussions are now taking place in the IBM Developer Answers forum.