If a man's only available pure covering is silk, he must pray naked rather than use the silk.

Awrah Exposure and Clothing Scarcity in Prayer