Clean Docker Desktop install, starts WSL 2, no container running. Let's check the memory usage:

Ouch, that's too much for having (literally) nothing running. I don't know why it eats so much RAM, but there's a quick fix.

Create the file C:\Users\<username>\.wslconfig like the example below:

[wsl2]
processors=1
memory=1GB

Run wsl --shutdown from a terminal, try it again, and there you have it:

In the meanwhile I suggest you to stay tuned to this issue, hopefully there will be a proper fix in the future.

P.S.:

Even when limiting the memory, vmmen still hungry, when set to 512MB I get an error: