I have used powdered indigo, spirulina and madder root mixed into glycerin and added drop by drop to M&P soap to good effect for blue, green and red. I would figure the same would work for alkanet for purple - just go easy or it will go black. I would think just soaking annatto seeds in glycerin would give a nice yellow. You just want the glycerin to be as strong in color as possible so that you add as little as possible.
Edit to add: the spirulina does fade over time, but to a paler green, not beige as in CP soap.