Windows Schannel error event ID 36871 a fatal error occurred while creating an SSL server credential. The internal error state is 10011

I recently encountered this error on a Windows 7 system, after a vendor-provided update to some third-party software.  RDP stopped working after the update, and the problem turned out to be layered.  The TL;DR version is this:  FIPS was enabled but the Cipher suite settings did not have any FIPS-compliant algorithms defined.

First, the simple part:  the vendor disabled RDP via GPO.  That's an easy thing to undo and I won't detail it here.

But even after enabling RDP via GPO, it still wouldn't work.  The RDP client wouldn't connect.  When I connected with openssl s_client I got this:

# openssl s_client -connect system:3389
CONNECTED(00000003)
write:errno=104
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 247 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

Looking in the Windows Event Log, I found an error from Schannel with Event ID 36871, and the error text "a fatal error occurred while creating an SSL server credential.  The internal error state is 10011."

I eventually narrowed this down to the fact that the vendor had turned on FIPS-compliant algorithms.  However, on this system, I had set the allowed cipher suites to "modern" algorithms like ECDHE-RSA-AES256-SHA384, which is not FIPS-compliant but is more secure; i.e. FIPS-compliant algorithms are old and less secure.

So the fix was to disable FIPS again.