«إِلَى الْمَكْتَبَةِ ذَهَبْتُ»
To the library I went.
In this sentence, the fronted phrase may emphasize the destination, especially when answering “Where did you go?” It does not, in isolation, establish that the speaker went nowhere else.
A claim of absolute restriction needs a supporting context that excludes other destinations. Fronting invites interpretation; it does not settle every possible interpretation.