Looking aside with the head is generally a disliked movement. Turning the body far enough to lose the qiblah direction is a separate question affecting validity.
Opening interactive controls...
Head turning versus leaving the qiblah
Disliked Acts in Prayer: Movement, Dress, Turning and Validity
Looking aside with the head is generally a disliked movement. Turning the body far enough to lose the qiblah direction is a separate question affecting validity.