The source allows people outside the Ka'bah to follow a prayer leader who is inside when the door is open.
The open door preserves the practical connection needed for following. This is a specific source rule and should not be expanded into a claim that every barrier or every closed building is always irrelevant to congregational following.