CRM and IIS Machine Keys

When you have multiple CRM web (front-end) servers splitting the load, requests can bounce between servers or they can be set to remain on the same server through the NLB/Firewall settings. The latter option is commonly referred to as “Sticky Sessions” or “Persistence”.

If requests bounce between servers (whether by design or by fault), you may notice issues when accessing certain functionality within CRM. Often it will be accompanied by the following error message in the event viewer:

Event code: 3012 
Event message: An error occurred processing a web or script resource request. The resource identifier failed to decrypt. 

Exception information: 
    Exception type: HttpException 
    Exception message: Unable to validate data.
   at System.Web.Configuration.MachineKeySection.EncryptOrDecryptData(Boolean fEncrypt, Byte[] buf, Byte[] modifier, Int32 start, Int32 length, Boolean useValidationSymAlgo, Boolean useLegacyMode, IVType ivType, Boolean signData)
   at System.Web.UI.Page.DecryptString(String s, Purpose purpose)
   at System.Web.Handlers.ScriptResourceHandler.ProcessRequest(HttpContextBase context, VirtualFileReader fileReader, Action`2 logAction, Boolean validatePath)

Basically what is happening is one server encrypted (validated) some of the traffic with an automatically generated key and another server received it but cannot decrypt it because it has different keys. These keys are known as “machine keys” and are found within IIS. By default, these keys are all set to generate automatically and no two are the same. When you are using multiple web servers, the machine keys need to be set statically and shared amongst them. Here is how to do this, starting on the first web server:

1. Open IIS, select the Microsoft Dynamics CRM Website, and double-click “Machine Key”
   
   
2. Uncheck the “Automatically generate at runtime” and “Generate a unique key for each application” boxes under both “Validation Key” and “Decryption Key”.
   
   
3. Click “Generate Keys” on the right side of the screen.
   
   
4. This will create new, random keys.
   
   
5. Click apply.
   
   
6. From here, repeat steps 1 and 2 on the remaining web servers. Instead of generating new keys as in step 3, copy and paste the keys generated from earlier into the remaining servers so that all servers have the same set of keys.
7. Once keys on all servers are set and applied, reset IIS on all the boxes. That should do it.

CRM 2016 Update 0.1 Bug with ADFS for Server 2012 R2

Following a successful upgrade to CRM 2016 and installation of the 0.1 Update, users could no longer authenticate against ADFS using the “internal” URL.

On the ADFS server I was seeing Event ID 364 in the Event Viewer:

Exception details:
Microsoft.IdentityServer.Web.InvalidRequestException: MSIS7042: The same client browser session has made '6' requests in the last '0' seconds. Contact your administrator for details.

In efforts to get this working, I tried completely re-configuring CRM for claims and IFD and even recreated the Relying Party Trusts. All with no luck but apparently, I was not alone on this issue and Microsoft has acknowledged it as a bug with Update 0.1. The following forum thread has more info:

https://social.msdn.microsoft.com/Forums/windowsserver/en-US/9d4040eb-81fa-4144-ae4b-85ca4610aa1d/crm-2016-problem-with-claimsbased-authentication?forum=crm

These are the only real details on this issue currently available:

There were major code changes in Ara UR1 for authentication. The affected code is in Microsoft.Crm.Core.Security.Identity.IdentityExtensions.GetUserPrincipalName(). We are unable to cast from type ClaimsIdentity to a new type CrmIdentity. Therefore, the variable is null, and we cannot retrieve the information.

Please note that only the ‘internal’ URL is affected when IFD is setup. The ‘external’ URL (e.g. https://orgname.domain.com) that uses forms authentication works fine still. I will update this blog post as more information becomes available.

UPDATE: Service Pack 1 for CRM 2016 resolves this issue.

Host Header Configured on Selected Site Does Not Resolve to an IP Address that is Assigned to the Local System.

I was recently performing an in-place upgrade from CRM 2015 to CRM 2016 when during the system checks I received the following warning:

Host Header "notCRM.domain.com" configured on selected site does not resolve to an IP address that is assigned to the local system. This conflict must be reconciled for CRM Server to function properly. 

The fact that it was giving me a host header that did not pertain to the CRM website was odd so I took a look at IIS. What I discovered was that there was another website setup in IIS on the CRM Server. And that site had the site ID of 1.

As I was the original installer of CRM for this deployment, I knew that this site wasn’t here when CRM was installed. CRM originally had the site ID of 1, not this other site. Somebody had been in here messing around and changed the site IDs manually. After asking enough questions, I learned who did this and why. To make a long story short, their site needed to have the ID of 1. CRM, on the other hand, does not care so I rolled with the punches and decided I would work around this rather than cause a fuss.

From what I could deduce about the error message, the CRM installer was being told to look at the IIS website that had an ID of 1 to perform the upgrade on. I decided to switch the CRM site back to the ID of 1. To do this, you must first change the other site’s ID to a number not in use (e.g. 3, in my case). To do this, go into the site’s advanced settings.

After the other site was changed, I reverted the CRM site back to 1 and reset IIS.

The upgrade wizard will need to be restarted for it to pick up the change but once you make it back to the system checks, the warning should be gone.

The upgrade succeeded without issue but we weren’t done just yet. Since the other site needed to have the ID of 1, I needed to revert changes to the site IDs which in itself isn’t complicated. What was not taken into account was a registry key in HKLM\Software\Microsoft\MSCRM called “website”.

This registry key indicates which website directory in IIS CRM is using. Since CRM is using the ID of 2, this key needed to be switched to “LM/W3SVC/2”. In case you have been wondering what the ID controls, it is just an identifier used in the directory name for trace files and log files. It’s nothing critical that will not let CRM operate as normal.

By now you are probably thinking “Why didn’t you change this key rather than switching the site IDs to resolve the warning message during the upgrade?!?!” Well... it didn’t occur to me until afterward I completed the upgrade. Hindsight is 20/20!