The matn recommends that the Imam, when he intends voluntary prayer after the Fard, move toward his left side before praying it.
If there is no following Nafl/Sunnah, or after completing it, the Imam may turn to face the congregation. The commentary qualifies this: he should not sit directly facing another person who is still praying. The restriction is therefore about facing a praying person, not specifically about every latecomer wherever that latecomer may be standing.
Opening interactive controls...
Remembrances after Fard Prayer and the Sunnah That Follows
The matn recommends that the Imam, when he intends voluntary prayer after the Fard, move toward his left side before praying it.
If there is no following Nafl/Sunnah, or after completing it, the Imam may turn to face the congregation. The commentary qualifies this: he should not sit directly facing another person who is still praying. The restriction is therefore about facing a praying person, not specifically about every latecomer wherever that latecomer may be standing.