Bacteria, mold, etc - the bugs that spoil things - need a few things to grow / survive: food, water, correct temperature, correct pH, etc.
The baddies usually don't grow in soap because the pH is too high, there's nothing nutritious for them in soap, and moisture is too low. Also, the saponification process provides a bit of a "kill step" (especially when heating is uniform throughout the bar). Between the pH drop, heat from saponification, and drying during curing - soap becomes a pretty inhospitable place for the bugs.
The typical ingredients (oils, lye, distilled water, and powdered or liquid colorants and fragrances) also don't support a lot of micro growth / survival - so the incoming micro load is low.
When you have pockets of organic material that aren't mixed well (chunks of avocado, for example), the organic material can not only bring in more bugs and provide nutrition for them, but also can create a micro-environment where moisture, lower pH, etc can allow them to grow - creating a little protective pocket of yuck that bugs love. The heat transfer rate in the chunks will also be different and may provide a bit of protection. Large particles (that contain a high moisture content) will dry slower than the surrounding soap during curing, so the extra moisture will make the bugs happy and happy bugs will make unhappy soapers.
The antimicrobial nature of soap has less to do with presence or absence of oxygen (as different bugs have different oxygen requirements - the baddest one in the food industry grows only in the absence of oxygen). And, while soap is a salt, salt is antimicrobial due in large part to it's ability to bind water (bound water isn't available for microbes to use and they dehydrate).
For all of the reasons above, and as previous posters mentioned, it's important to puree and mix well - even puree that isn't well incorporated can create islands of bug paradise
Strictly from a chemistry standpoint, you also want to mix well to incorporate the liquid in the puree through the soap so you have consistent saponification throughout the bar.
(I try not to science here, but I get paid to grow and kill bacteria...and I like to talk about bacteria - it's the small things that matter
)