Recently, I had a situation where internal servers were not granted access to the internet. This posed a problem because we started to see issues with the setup of conferences (both web and ad-hoc). The symptoms were:
- Clients would seemingly join the conference initially
- The first IM or two would go through
- After a period of 30-60 seconds you would no longer see the IM indicator for other users, but they would remain in the conference
- Subsequently, users would no longer receive IMs
We also saw the following errors in the Lync Server Event logs:
Log Name: Lync Server
Source: LS IM MCU
Date:
5/22/2012 4:31:22 PM
Event ID: 33019
Task Category: (1019)
Level: Error
Keywords: Classic
User:
N/A
Computer:
lyncfe1.domain.local
Description:
Lync Server IM MCU failed to connect to the SIP Front End.
Local Machine name:
lyncfe1.domain.local FE
hostname:
lyncfe2.domain.local IP address: 10.10.1.213:5061
Message: TLS Negotiation took too long to complete
Stack:
Cause: Lync Server IM MCU cannot communicate with the
Microsoft Lync Server 2010 Front End Service over SIP due to network
connectivity issues or unavailability of the Microsoft Lync Server 2010 Front
End Service. This condition prevents setting up new conferencing sessions,
forwarding Instant Messages and is disruptive and potentially fatal for
existing conferencing sessions.
Resolution:
Please ensure network connectivity and availability of the
Microsoft Lync Server 2010 Front End Service for the IM Conferencing Service to
be able to function correctly.
The details of the Event ID were somewhat of a "red herring" in that they confirmed that the problem existed, but the proposed solution was not exactly correct in this case. I used PortQry to verify that the ports were, in fact, reachable and I used the DigiCert SSL utility to verify that the certificate being returned via the SSL handshake was good.
Seeing "TLS Negotiation took too long to complete" for about the 10th time finally made me realize that this could be caused by the servers' inability to check the CRL.
How to enable CRL checking through a Web Proxy at the server level
The ultimate solution was to enable the server to use the proxy to check the CRL. Obviously configuring the proxy settings in Internet Explorer would only work for the user profile. We needed a way to do this at the server level, without requiring a user to be logged into the system. To do this, we turned to netsh. The following command was run on each internal server which then cleared up the LS IM MCU errors and resolved our issue (Note - be sure to run CMD as Admin):
netsh winhttp set proxy <ProxyServerNameorIP:ProxyPort>
While this scenario was specific to Lync, it can certainly be used in situations that involve other applications that rely heavily on certificates (i.e. Exchange).
Seeing "TLS Negotiation took too long to complete" for about the 10th time finally made me realize that this could be caused by the servers' inability to check the CRL.
How to enable CRL checking through a Web Proxy at the server level
The ultimate solution was to enable the server to use the proxy to check the CRL. Obviously configuring the proxy settings in Internet Explorer would only work for the user profile. We needed a way to do this at the server level, without requiring a user to be logged into the system. To do this, we turned to netsh. The following command was run on each internal server which then cleared up the LS IM MCU errors and resolved our issue (Note - be sure to run CMD as Admin):
netsh winhttp set proxy <ProxyServerNameorIP:ProxyPort>
While this scenario was specific to Lync, it can certainly be used in situations that involve other applications that rely heavily on certificates (i.e. Exchange).
Hi PHIL,
ReplyDeleteExcellent article.
Is there a specific schedule of CRL checks or are they done on a realtime basis?
Thanks Abhay.
DeleteCRL checks are done "on demand," meaning when the certificate is presented to the server/client. It should then download a copy of the CRL and cache it until the "Next Update" time indicated on the CRL. Then whenever a certificate is presented it will check the locally cached CRL until the "Next Update" time, at which point a new CRL should be retrieved and checked.
There are some other factors that might influence how this occurs, but what I described above would be the default.
-Phil