Wow. $5 per ounce is a spendy FO, but it doesn't automatically follow that the amount of money spent means the FO will work well in soap.
What supplier are you getting this from? Has the FO been tested in soap? What is the recommended dosage per the supplier? Other people have good results with the Bay Rum blends they use (
@IrishLass). Perhaps do some research on SMF to see what FOs have worked for others?
Another thing is your use of clay. I know ... I know ... it's supposed to be an anchor, but I've never been convinced that actually works. It's "common wisdom" that everyone thinks should work but few or no people will confirm it actually does.
And even if clay actually does work as an anchor, what an anchor is supposed to to is SLOW the rate at which fragrance is released into the air, thus causing the fragrance to last longer. If you want a strong scent up front, using an anchor like clay isn't the solution. More appropriate solutions include using more of the fragrance and making sure the fragrance is stable in soap. Not all fragrances are soap-stable.
Also people can become "nose blind" to fragrance. Like if you use cologne, you can smell it for awhile after you apply it, but you can't smell it after some time passes although others might be able to smell it then.
Another issue to keep in mind is that fragrance sometimes fades or changes while the soap is curing and then the fragrance can reappears after some weeks.