setting up ssh between two windows machines - permission denied

I am trying to establish ssh connection between two Windows machines. Here are things I did:

• I installed OpenSSH on both machines, generated the ssh keys and placed the public key on the host machine in the authorized_keys file in .ssh folder and kept the public and private key files in my local machine in the .ssh folder.
• The sshd and ssh-agent are in the Running state, the sshd_config file has set "PubkeyAuthentication yes" and "PasswordAuthentication no".
• To make sure it is not a firewall issue I created a rule to allow inbound traffic on port 22.
• I disabled inheritance in the file permissions. In file permissions for authorized_keys there is only my user and it has Full control.
• Also I commented the line below in sshd_config:

# Match Group administrators                                                    #       AuthorizedKeysFile __PROGRAMDATA__/ssh/administrators_authorized_keys 

Another thing I noticed is that when I do Stop-Service sshd and run sshd on host then I am able to successfully ssh into host, however when I do Start-Service sshd then there is Permission denied issue. What is the difference here?

What could be the reason here for the Permission denied? What I am missing?

The context (not sure if important): I am having a stationary computer and a laptop, I am trying to set up the ssh so I can access the stationary computer from my laptop remotely.