Why is not every departure from the usual referent figurative?

Defining Literal Usage: Linguistic Assignment and Speech Convention