The first day of a new year is a great moment to relax and prepare for what is ahead – but spare a thought for Microsoft Exchange administrators who may have woken up to seized up installations of their on-premises email servers. I was among those affected, but only on my tiny system. Messages were stuck in the submission queue, suspiciously since midnight or thereabouts (somehow a message sneaked through timed 12.14 am) and the last error reported by the queue viewer was “Messages deferred by categorizer agent.”
As usual I went down a number of rabbit holes. Restart the Exchange Transport service. Reboot the server. Delete the first message not to be delivered in case it was corrupt and somehow clogging up the queue. Check for certificate issues.
It was none of these. Here is the guilty party in the event viewer:
The FIPS-FS Microsoft Scan Engine failed to load, with the error can’t convert “2201010001” too long.
The impact was that the malware filter could not check the message, hence the error from the categorizer agent.
The solution is to run the Exchange Shell on the server and navigate to the Scripts directory where Exchange is installed, for example C:\Program Files\Microsoft\Exchange Server\V15\Scripts. Here you will find a script called Disable-AntimalwareScanning.ps1.
should work. Run it, restart the Exchange Transport service, and email will start to flow.
Once the problem is patched, there is a companion script called Enable-AntimalwareScanning which restores it. Though I am not sure of the value of the Exchange malware filter since Microsoft considers that even on-premises installations should use the Microsoft 365 services for spam and malware scanning, and the on-premises protection features are not kept up to date, meaning that a third-party or open source spam and malware filter is a necessity anyway, unless you go the Office 365 route.
Another reason not to run Exchange on-premises – but Microsoft still says that hybrid systems using Azure Active Directory Connect should do so in order to manage mailboxes.
Note: the maximum value for a 32-bit signed integer is 2,147,483,647. Yesterday which was perhaps represented as 2,112,310,001 would have fitted within that whereas today 2,202,020,001 did not. Dates and times are awkward for programmers.
Update: Microsoft has an official fix here. Thanks to Erik in the comments for the link.