فَدَلَفْتُ إِلَيْهِ
So I walked up toward him.
Dalafa means to move or advance toward something. Ilayhi (“toward him”) refers to the preacher at the center of the gathering.
Rather than listening from a distance, the narrator draws near deliberately. The next words explain what he hopes to gain.