Replying to Salaam by physically shaking hands (Musafahah) during prayer does not break the prayer as long as no words are spoken.

Invalidations of Prayer (مفسدات الصلاة) Part 1: Speech and Sounds