Setting up a Windows VM for Kernel Debugging

If you search the web using the title of this post, you'll get many links to articles most of which explain how to set this up using a virtual COM port on the VM. Coming from driver background that goes back over 20 years, at first this seemed quite natural. But then I remembered that using WinDBG over COM port to be painfully slow. And recent adventures in OSX land using LLDB led me to wonder if WinDBG over network would to be possible?

Well it turns out that WinDBG over Ethernet is indeed possible and is documented by Microsoft here. However, they miss out on certain key steps, which won't be obvious for the casual reader unless one reads the entire driver debugging documentation from the beginning.

This post captures all the requisite steps as a quick reference guide. I used one of the freely available MSEdge on Windows 10 VM for  VirtualBox. It's only valid for 90 days, but 90 days was more than enough for my little excursion into the driver land. If you want a more permanent solution, you can use a Windows setup with a perpetual license.

1. Setup a Windows VM. Take the virgin system snapshot.

2. Default VM will only have one Network Adapter, which will be configured to NAT. You may continue to use NAT or change it to bridged. For remote debugging, however, we won't be using this adapter. For this we create a dedicated adapter and configure it to use the Host-Only network. So essentially the VM should have two NICs, one of which has to be configured to use the Host-Only network. Set this up and boot the VM. Make sure that this NIC has a valid IP address assigned and that you can ping the VM from the host through this IP. By default this NIC is configured to use DHCP. You may wish to change this to use static IP so that you can consistently refer to the VM through this fixed IP. But this is optional.

3. Install the Virtual Guest Additions package. This allows enabling clipboard sharing & folder sharing between host & guest machines, something that'll come very handy. Configure the above two (as well as device sharing if you're developing a driver for a physical device such as USB device) and take another snapshot.

4. Setup a dedicated boot configuration for debugging drivers. Use the following commands from Administrator Command Prompt:

   C:\Windows\System32> bcdedit /copy {current} /d "Debug"
   

   This creates a copy of the current boot configuration named 'Debug'.

5. Set the newly created boot configuration, Debug, as the default.

   C:\Windows\System32> bcdedit /default <{debug_configuration_guid}
   

6. Reboot into the 'Debug' boot configuration just created. We need to configure this configuration with additional boot settings. It's a lot easier if we reboot to this making it the current configuration so that we can omit specifying the configuration guid in every bcdedit command.

7. Configure the newly created boot configuration, Debug, which is now the current configuration:

   C:\Windows\System32> bcdedit /set nointegritychecks on
   C:\Windows\System32> bcdedit /set testsigning on
   

8. Open up registry editor and create a DWORD (32-bit) entry named DEFAULT under HKLM\SYSTEM\CCS\Control\Session Manager\Debug Print Filter, with the value 0xF. If the key 'Debug Print Filter' does not exist, create it. This is the default debug message filter mask for messages printed from the kernel. By default this mask is set to 0x3 (I think), which only displays messages at ERROR and WARNING levels. If you want INFO & DEBUG level messages to be displayed at the debugger, you need to set the relevant bits (2 & 3) on this mask.

9. Follow steps in Microsoft's own documentation here to configure the boot configuration dbgsettings & set the busparams to the VM's Host-Only NIC bus, driver and function number values (Steps 3, 4 & 5).

10. Take another snapshot so that you can revert to this image if you end up messing up your environment through repeated test driver deployments!

11. Don't forget to setup the symbols path for WinDBG!

12. Assuming you did all these, you should be able to proceed with driver development tasks