For nikko I would recommend staying at the onsen areas scattered around the fringe
Kinugawa onsen is very nice. Heaps of hotels there. Big onsen resort with many choices. All of them are pretty OK. No point staying in one with a riverside view. Can just walk down to it. Expect to pay 7000-9000 yen for 2 meals one night and all u can drink, karaoke and all sorts of stuff
Yumoto onsen, near onsen ji is worth the 1.5hr trip. The onsen water is sulphurous and stinks slightly but it is milky and really really good quality. Your skin feels soft and silky smooth. 10/10 would go again and bring my family there someday.
Yumoto onsen is also 2400m above sea level so expect it to be cold. It was foggy and really cold when I was there in July. Expect colder conditions. Snow a possibility