Tracing ADFS Logon Failures - Enabling ADFS Auditing

If you are ever faced with a situation where you are seeing a ton of logon failures in your ADFS logs and you’re not sure where they are coming from, you will soon learn that the basic logs do not provide any insight into their origins. But fear not! Some additional auditing can be enabled to help track down your problem child.

You will likely start with “Event 342 – The user name or password is incorrect” in the ADFS Admin logs – this is about as useless as it gets.

In my scenario, I was seeing about 1,500 of those events per minute. Needless to say, something was wrong but we had no idea what could possibly be trying to authenticate that much. Luckily, ADFS has some built-in auditing that can be of more use in situations like this. Start out by opening the ADFS Management Console and choose the option “Edit Federation Service Properties…” (it’s in the column on the right). Once in the properties screen, click on the “Events” tab. Here you should see 5 checkboxes – 2 of which are unchecked. Check those boxes (Success audits and Failure audits) and click OK.

Now that ADFS is setup for auditing, you need to tell the server to allow it. Head on over to the Local Security Policy of the ADFS server – this can be found from the Server Manager. Once here, drop down “Advanced Audit Policy Configuration” and then “System Audit Policies – Local Group Policy Object”. Locate the “Object Access” policy and then open the “Audit Application Generated” subcategory. Check the three boxes on this screen and then click OK.

You should now be all set to revisit your Event Viewer. Look into the Security events under the Windows Logs and you should now see events with ID 411 for “Classic Audit Failure” with the source as “AD FS Auditing”.

Go ahead and open one of those bad boys up…. Ahhhh finally some useful information! A quick look through will reveal the Client IP, which should be the machine sending the invalid authentication requests to ADFS.

The process from here is self-explanatory but head on over to the machine with the offending client IP address and find out what the problem is. In my case, there was a long-forgotten email router still setup which kept trying to access CRM. After figuring out the problem, it is recommended to disable auditing.

To Run The Email Router Configuration Manager, You Must Be A Local Administrator

Just had an unusual error message pop-up when trying to open the Email Router Configuration Manager following the update to Dynamics 365 (on-prem). It was something I had never seen before with previous versions of CRM – 2016 included. The message read:

To run the Email Router Configuration Manager, you must be a local administrator. Make sure that you are a member of the Administrator group before you start this program.

Well… this was an internal Dynamics 365 environment in which my account was a Domain Admin so I was a bit confused but I played along. Went ahead and added my account in as a local admin on the server and rebooted for good measure. That didn’t help. No matter how much I restarted the router service, rebooted, tried adding permissions, etc… it all fell on deaf ears. Googling the error message resulted in no meaningful results – which usually means 1 of 2 things – my router is completely hosed or I just uncovered a nice new bug with Dynamics 365!

After thinking about it for a few more minutes, I figured why not try running it AS AN ADMINISTRATOR!?!?! Sure enough, 3 clicks later and I was in… Easy workaround and I hope it saves some head-scratching!

An attempt was made to access a socket in a way forbidden by its access permissions

When setting up the Microsoft Dynamics Email Router, there can be a lot of obstacles in getting emails to send and receive properly. One of which turns out to be McAfee VirusScan. If you get the following error when testing the connection on the Outgoing profile, you may want to see if McAfee (or any other virus protection software) is running on the server:

Outgoing Status: Failure – An error occurred while checking the connection to the email server {smtp server name}. An attempt was made to access a socket in a way forbidden by its access permissions.

In order to fix this, right click on the McAfee icon in the system tray and open up the VirusScan Console.

In the VirusScan Console, double-click “Access Protection”.

Under Categories, click on “Anti-virus Standard Protection” and then check to see if there are any checkmarks next to “Prevent mass mailing worms from sending mail”. If there are, simply uncheck them.

Click OK and close out of the VirusScan Console. Open the email router and try testing again. You should now be seeing green!

The Authentication Endpoint Username Was Not Found On The Configured Secure Token Service and Port 808

While not recommended, circumstances sometime present the need to install CRM and ADFS on the same server. If this is done on Windows Server 2012 R2, you will likely run into issues connecting anything through the discovery service such as the Outlook Add-In, Email Router, and Plugin Registration Tool. The error will read:

The authentication endpoint Username was not found on the configured Secure Token Service.

If taken literally, one would be tempted to go to the ADFS Configuration and enable the /adfs/services/trust/13/username endpoint as shown below:

DO NOT DO THIS! On the outside chance that it does actually fix the immediate issue it can cause further authentication problems and is NOT recommended – despite what some other blogs/forums may tell you.

Further investigation should eventually lead you to this error in the ADFS event viewer logs:

System.ServiceModel.AddressAlreadyInUseException: There is already a listener on IP endpoint 0.0.0.0:808. This could happen if there is another application already listening on this endpoint or if you have multiple service endpoints in your service host with the same IP endpoint but with incompatible binding configurations. —> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted

With ADFS for Server 2012 R2, net.tcp port 808 is utilized for a function of the federation service. You may recall that port 808 is also used by CRM for its sandbox service so it’s easy to see the problem here. Port conflict! Luckily, the solution is rather simple. Just open a PowerShell prompt on the server and enter the following:

Set-ADFSProperties -nettcpport 809

(Note: In the above the command, 809 is just an example of a port that can be used and is not required. If something else is running over port 809 on your server, substitute it for some other number.)

After the command has been run, restart the ADFS service, IIS, and the email router service (if the email router was having the issue connecting) – to be on the safe side, you could just reboot the whole server. Give it another go and you should be in business! If you are wondering why this hasn’t been an issue with past versions of ADFS, it's because the federation services used to run on ports 1500 and 1501.