I'd mix the lye with the water and add the powder to the oils, or the water after its cool as opposed to mixing the power with the water before lye. This way you wouldn't have to worry about burning the milk.
I made goats milk soap a couple weeks ago with canned goat milk that I froze. It had a weird chemical odor that I noticed super strong when unmolding, but my bf couldn't smell it. Said it smelled chocolately to him from the cocoa for color! Its gotten much less, and I think is completely gone now but maybe if I sniffed up close I could smell it. Haven't tried the soap yet of course. I can't wait to, its goatsmilk with oatmeal flour.
I bought the powder to use from now on with goatsmilk soap just to simplify things, so I don't have to freeze the goatsmilk a day ahead. And I got a big container of powder for about the same price as 2 cans, but it will make more volume than 2 cans so its cheaper.
Interestingly with your title, I have a soap that smells like modeling clay. It still does now so I wonder if it will go away. Its unscented but used celery to color it green, and a somewhat unusual mix of oils to try to enhance the green color and because I was making embeds so I wasn't as worried about it being a perfect soap on its own. Its embed in the goat milk soap, but I've got a couple extra trees and then a big puck of the scraps smashed together.