It may be that sugars in the milk products contribute but I think the fat content should also make a difference, since it's the fat that gets saponified and adds X quality to the soaps. The higher the fat content, the more it would affect the soap, I would think. I use heavy cream in my soaps and I think it adds a level of creaminess (and hardness) that I wouldn't get with milk, although I have never used just milk, to be honest. My understanding is that goat milk has a higher fat content than cow's milk, but I doubt it has the fat content of heavy cream. I have not compared side to side.