May 26, 2017—KB4023680 (OS Build 14393.1230)

Windows Server 2012 R2 and Server 2016 computers that experience disconnections to iSCSI attached targets may show many different symptoms. These include, but are not limited to:

• The operating system stops responding

• You receive Stop errors (Bugcheck errors) 0x80, 0x111, 0x1C8, 0xE2, 0x161, 0x00, 0xF4, 0xEF, 0xEA, 0x101, 0x133, or 0xDEADDEAD.

• User log on failures occur together with a "No Logon Servers Available" error.

• Application and service failures occur because of ephemeral port exhaustion.

• An unusually high number of ephemeral ports are being used by the System process.

• An unusually high number of threads are being used by the System process.

Cause

This issue is caused by a locking issue on Windows Server 2012 R2 and Windows Server 2016 RS1 computers, causing connectivity issues to the iSCSI targets. The issue can occur after installing any of the following updates:

Windows Server 2012 R2

Release date	KB	Article title
May 16, 2017	KB 4015553	April 18, 2017—KB4015553 (Preview of Monthly Rollup)
May 9, 2017	KB 4019215	May 9, 2017—KB4019215 (Monthly Rollup)
May 9, 2017	KB 4019213	May 9, 2017—KB4019213 (Security-only update)
April 18, 2017	KB 4015553	April 18, 2017—KB4015553 (Preview of Monthly Rollup)
April 11, 2017	KB 4015550	April 11, 2017—KB4015550 (Monthly Rollup)
April 11, 2017	KB 4015547	April 11, 2017—KB4015547 (Security-only update)
March 21, 2017	KB 4012219	March 2017 Preview of Monthly Quality Rollup for Windows 8.1 and Windows Server 2012 R2

Windows Server 2016 RTM (RS1) 

Release date	KB	Article title
May 16, 2017	KB 4023680	May 26, 2017—KB4023680 (OS Build 14393.1230)
May 9, 2017	KB 4019472	May 9, 2017—KB4019472 (OS Build 14393.1198)
April 11, 2017	KB 4015217	April 11, 2017—KB4015217 (OS Build 14393.1066 and 14393.1083)

Verification

• Verify the version of the following MSISCSI driver on the system:
  
  c:\windows\system32\drivers\msiscsi.sys
  
  The version that will expose this behavior is 6.3.9600.18624 for Windows Server 2012 R2 and version 10.0.14393.1066 for Windows Server 2016.

• The following events are logged in the System log:

  Event source	ID	Text
  iScsiPrt	34	A connection to the target was lost, but the Initiator successfully reconnected to the target. Dump data contains the target name.
  iScsiPrt	39	The Initiator sent a task management command to reset the target. The target name is given in the dump data.
  iScsiPrt	9	Target did not respond in time for a SCSI request. The CDB is given in the dump data.

• Review the number of threads that are running under the System process, and compare this to a known working baseline.

• Review the number of handles that are currently opened by the System process, and compare this to a known working baseline.

• Review the number of ephemeral ports that are being used by the System process.

• From an administrative Powershell, run the following command:
  
  Get-NetTCPConnection | Group-Object -Property State, OwningProcess | Sort Count
  
  Or, from an administrative CMD prompt, run the following NETSTAT command together with the "Q" switch. This shows "bound" ports that are no longer connected:
  
  NETSTAT –ANOQ
  
  Focus on ports that are owned by the SYSTEM process.
  
  For the three previous points, anything more than 12,000 should be considered suspect. If iSCSI targets are present in the computer, there is high probability that the issue will occur.

Resolution

If the event logs indicate that many reconnections are occurring, work with your iSCSI and network fabric vendor to help diagnose and correct the reason for the failure to maintain connections to iSCSI targets. Make sure that iSCSI targets can be accessed over the current network fabric. Install updated fixes when they become available. This article will be updated with the specific KB article number of the fix to install when it becomes available.

Note We do not recommend that you uninstall any of the March, April, May, or June security rollups. Doing so will expose the computers to known security exploits and other bugs that are mitigated by monthly updates. We recommend that you first work with iSCSI target and network vendors to resolve the connectivity issues that are triggering target reconnects.