I agree! Unfortunately many of the 'naturally mined' substances contain unacceptable levels of heavy metals for human use, even in wash-off products. It would need to be lab tested first, just to be sure.
I have used cosmetic-grade iron oxide powder in the past to color soap, in my experience it makes a pretty, soft, orange-red color. (use too much and you will have orange-y bubbles!) Like many of the natural colorants there is usually a bit of fading.