>"unlike neural networks which we will need to entirely redesign, rebuild, and retrain to get similar progress"

I don't think you would need to redesign or rebuild anything for that. You would need to train the network on additional examples, which I supposed you could call retraining (although it need not be from scratch), but that is the same in the case of the child.