The issue with “talking Wayland directly” is that Wayland as a definitive entity that all Wayland hypervisors understand or can run doesn’t exist. Wayland has a very tiny base library and everything else is implemented by the specific running hypervisor. And those can do things under the hood very differently to each other, with APIs that can be mostly compatible but differ in small details. So when you “talk Wayland directly” you have to say “this only runs on proton and hyperland but not sway and Mutter, but maybe on Weston” (just an example for a sentence) because that’s what you have tested and found that it works with your implementation or you have to implement all the different ways in which the Wayland implementations do their things (and maybe implement workarounds if one of them doesn’t implement a specific function (like HDR support or fractional scaling) at all. “Talking Wayland directly” is a huge pile of very basic level stuff that you have to implement and maintain and keep track of to change in case one of the implementations changes something in a newer version that is incompatible to your code oh and better implement it in a way so you are still compatible with the old code too because that will be in use for the next 100 years on Debian.

There is a very good reason to use a framework that handles all that stuff and provides the stable API and functions that the display manager should provide (and what it did with X for decades).