How do you choose the right features to emphasize in a caricature without crossing the line into distortion?

I tend to ask my clients exactly what they need or want and it also helps if they provide references for what they ultimately want as a result. In terms of the right features to emphasize, it sort of comes with experience and solid fundamentals before stylization. It's also better to show your progress to the clients and see if they like it. 
Basically sort out your basic fundamentals, especially the face which connects to clients the best and move onto the form factors and body anatomy. 
Make sure to always have references to guide your process so you don't cross that distortion line or uncanniness that may result from over stylization or not enough detailing.