Can't Remove Bluetooth Device despite no entry in registry

I have a machine running windows 10.

I am trying to remove a bluetooth device and reconnect it. At the start, when I tried to remove the device through the windows' settings I got the message that "remove failed". The same thing happened when I tried to remove it through the control panel.

I removed the driver for the device in device manager, but after restarting, the device still persisted. The driver was not listed in device manager though.

The last step I took was to remove the entry for the device in the registry file, but after restarting the device was still there. There is currently no entry in the registry file. I removed it from:

HKLM\SYSTEM\CurrentControlSet\Services\BTHPORT\Parameters\Devices

Right now, I see the device in the windows settings and in the "devices and printers" section in the control panel and I can't remove either one. There is no entry for the device in the registry nor is there a driver for the device. I also don't have a backup of the registry before I removed the entry.

How do I get rid of the thing so I can reconnect it? The device is a pair of headphones so it's something I use a lot.