I have used actual ground spices in my soap - allspice, ginger, cinnamon, clove, nutmeg. You do get a bit of a scent from those. But I don't think that it's possible for us to infuse enough of the aromatic compounds in water or oil to survive the lye monster. Smell the strength of an essential oil and then smell the strength of an infusion.