SSO no longer works after securing LDAP over SSL with LDAP signing required, IWA is enabled, and RC4_HMAC_MD5 is disabled in policy.

book

Article ID: 100054717

calendar_today

Updated On:

Description

Error Message

The end user at the client machine will see the following window when first connecting to the eDP server:

When using Single Sign On (SSO) the end user would just click the Sign In button and the expected behavior would be to automatically login to the eDP server.  Instead, there is no message related to the problem or the ability for the end user to login asking for Username / Password.  The end user just sees the following.

Server log on the eDP server shows the following log entries:
2022-12-20 11:34:00,900 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-25:[]) security package: Negotiate, connection id: 192.168.10.206:55217
2022-12-20 11:34:00,900 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-25:[]) token buffer: 1906 byte(s)
2022-12-20 11:34:00,903 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-25:[]) continue token: oXUwc6ADCgEBoQsGCSqGSIL3EgECAqJfBF1gWwYJKoZIhvcSAQICAwB+TDBKoAMCAQWhAwIBHqQRGA8yMDIyMTIyMDE2MzQwMFqlBQIDCr5LpgMCASmpCRsHRURQLkxBQqoUMBKgAwIBAaELMAkbB2VkcHRlc3Q=
2022-12-20 11:34:00,903 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-25:[]) continue required: true
2022-12-20 11:34:00,911 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-18:[]) security package: Negotiate, connection id: 192.168.10.206:55217
2022-12-20 11:34:00,912 INFO  [servlet.spi.NegotiateSecurityFilterProvider] (https-jsse-nio2-443-exec-18:[]) token buffer: 1849 byte(s)
2022-12-20 11:34:00,914 WARN  [ui.auth.EsaIWAAuthenticator] (https-jsse-nio2-443-exec-18:[]) [#160005] Authentication failed: error logging in user: The handle specified is invalid

NOTE:  Determine the IP Address of the machine where that the end user is connecting from.  Also determine the name of the eDP Clearwell server being connected to.  The server log for that eDP server would need to be reviewed.  In the example log entries above, the client IP Address was 192.168.10.206.

Cause

1.  On the client workstation:
Open a command (cmd) prompt window, type the command klist to see a listing of kerberos tickets.  Look at the ticket specifically for the eDP server that the end user was trying to connect to determine what type of encryption is being used:

2.  Go to that eDP server that the client was trying to access and open Local Security Policy.  Select Security Settings -> Local Policies -> Security Options as seen below.
Note: If group policy is in place the settings may be grayed out but can still see the policy.

3.  Scroll down and look for Network security: Configure encryption types allowed for kerberos and double click setting to see what encryptions are allowed on this eDP server.  In step 1 we determined that RC4_HMAC encryption was being used but in Local Security Policy on the eDP server we see that RC4_HMAC_MD5 encryption is not enabled.  This is why authentication for SSO is not working.  Also note that AES128_HMAC_SHA1 and AES 256_HMAC_SHA1 are enabled.

Additional Note:  There are a lot of moving parts to LDAP, LDAP over SSL, certificates, certificate properly added to cacerts.bcfks, etc that must all be checked for a healthy working environment prior to troubleshooting IWA/SSO.  See related article(s) below. 

Resolution

Option 1:  Change the Local Security Policy or Group Policy so that RC4_HMAC_MD5 is enabled.  If the change is made in Group Policy can go to the eDP server and run gpupdate /force in command prompt to force the change to update.  This change requires that the server be restarted in order to take affect.

Option 2:  Under the Cause section of this technote #3, it was also noted that AES128_HMAC_SHA1 and AES 256_HMAC_SHA1 encryption types are enabled which means these encryption types can be enabled for the Active Directory account that the service is using.  On the eDP server, go into Services and find the service called EsaApplicationService::FireDaemon and make note of the Log On As account for this service.  The below image shows edp\edpadmin1 account is being used:
 

Go to the Active Directory server for the eDP domain, open Active Directory Users and Computers find the account that the service is using, double click to modify, go to Account tab, and select the checkboxes for "This account supports kerneros AES 128 bit encryption" and "This account supports kerberos AES 256 bit encryption" like the following image shows:

Addtionally, if eDP Clearwell is used for EV collections, apply the same modifications to the EV Service Account.  For more details see the link in Related articles section below.

Go back to the eDP server, in Services, and restart the EsaApplicationService::FireDaemon service.


After performing Option 1 or Option 2, now go back to client and test SSO again.

 

Issue/Introduction

Single Sign On (SSO) no longer works for end users connecting to eDiscovery Platform (eDP) server with the following configuration:
  1. Lightweight Directory Access Protocol (LDAP) has been configured over Secure Socket Layer (SSL)
  2. LDAP signing is set to required in policy (local security policy or group policy)
  3. Integrated Windows Authentication (IWA) is enabled in eDiscovery Platform (eDP)
  4. RC4_HMAC_MD5 is disabled in policy (local security policy or group policy)