A soaper in another active thread explains her soap batter is reaching trace in a couple of minutes and she asks if that's normal. The poster talked about using her stick blender for (speaking from memory) something like 20-30 seconds at a time, hand stir, another SB for 20-30 seconds, etc. I replied something to the effect that this is normal when using a SB that much.
Just for grins, I looked at a video I did of a recent gardeners' soap that has lard, coconut oil, and 5% castor. The bottom layer of soap contains finely ground coffee. That is topped with a pencil line of cocoa and finished with a thicker layer of plain soap. That's a lot of fancy stuff for me!
I counted the seconds of stick blending from the time I added the lye until the soap was at a stable emulsion. I used the SB for a total of about 10 seconds in 2 minutes elapsed time. When the soap was at emulsion, I continued to color, manipulate, and layer the soap batter for another 19 minutes. The only reason why I was not able to work with the soap even longer was my EO fragrance blend was accelerating things -- the soap batter went from soupy liquid to a medium trace quickly after adding the EO blend.
At over 20 minutes of working time, I can't say the castor was causing any trouble! I wouldn't have guessed the elapsed time was that long, but the video footage is obviously more accurate than my memory.
ETA: So I guess what I'm saying in a roundabout way is that putting all the blame (so to speak) on a particular oil might be missing the boat. You may want to look at the whole method of how you make soap to see what you can tweak to get more working time with your soap batter. You might want to limit castor in some cases if you want to do really fancy intricate work, but you might see big changes just by altering your technique. If you want to troubleshoot your soap in this broader sense, tell us more about how you make your soap. You might get some tips that would be helpful.