Good timing on the question!
I just made a 7lb batch last night, with 75% lard, 20% CO, and 5% castor. I accidentally added 4 TBSP of sugar instead of 4 tsp to my lye liquid, which was vinegar for 100% of the water required. I usually use SL but was out last night.
I used the heat transfer method to melt the room-temp lard and CO, and then blended in my goat milk powder. The batter ended up at just under 120º after stirring and a bit of stick-blending. The extra heat was probably due in part to all the extra sugar I added!
Still, even at 120º, I had over 30 minutes to split my batter into four different pours: two colors going into a small loaf mold for the June Unchallenge, and a few others for variations (colors, scents) on my usual lard-GM bars for friends and family. It was just reaching light-to-medium trace at about 40 minutes.
This recipe is very consistent for me: lots of swirl time, hard bars, and very nice creamy bubbles after about 8 weeks cure. It's good at 6 weeks, but definitely nicer at 8 .... and really nice at 10 if you can hold out that long.
All that to say, I think you will be fine with either castor, or sugar, or both. I don't find that 5% castor or 20% CO affects my swirl time at all. Then again, I didn't stick-blend it too far past emulsion, either. Let us know how it works out for you!