Last month, I wrote about the unfounded “rule” limiting which to non-restrictive clauses and that to restrictive clauses. I hoped to show that restrictive which is common, standard, and unobjectionable, and has been for centuries. I’ve been updating the post with subsequent commentary from editors and linguists.
Restrictive which is ubiquitous, but non-restrictive that is very rare. The Merriam-Webster Dictionary of English Usage has several citations, including Shakespeare (“Fleance his son, that keeps him company”) and Oliver Goldsmith (“Age, that lessens the enjoyment of life, increases our desire of living”). The same source says the construction is used mostly by poets.
On Language Log in 2005, Geoffrey Pullum posted the following data from the Longman Grammar of Spoken and Written English (Note: integrated relative = restrictive; supplementary relative = non-restrictive).

The remarkable bottom-row figures will have caught your eye. Pullum describes finding a non-restrictive that as “like spotting the syntactic analog of an ivory-billed woodpecker”.
So you’ll understand why I felt a moment of excitement when I read what I thought was a genuine example in the wild: specifically, in The Blue Flower by Penelope Fitzgerald (chapter 34, “The Garden-House”):
Then as he dried himself he repeated, ‘Never does the heart sigh in vain, Justen,’ and she scarcely knew whether to be unhappy or not. In her mouth was something bitter, that tasted like the waters of death.
Reading it again, though, I decided it was more likely a restrictive clause, albeit one set off by an unusual comma. The comma supplies a pause, but it probably doesn’t mark a supplementary relative clause. That is, Fitzgerald’s line is equivalent to this:
In her mouth was something bitter that tasted like the waters of death. [restrictive]
and not this:
In her mouth was something bitter, which tasted like the waters of death. [non-restrictive]
What do you think?
Update:
My excitement has diminished as I’ve become more accustomed to seeing the construction. I recently read Anne Enright’s novel The Gathering, which has several instances of non-restrictive that. Here are two of them:
Ada bringing us for red lemonade into a pub, that had a black roof with huge letters of white written across it.
If Ada believed in anything she believed in this persistence, that other people might call the soul.
And another, this one from Obstinate Uncle Otis, a short story by Robert Arthur:
Maybe he thought he c’d ignore that lightning, like he ignores Willoughby’s barn across the road, or Marble Hill, that his cousin Seth lawed away from him so that now he won’t admit there is any such hill.
A recent post on Language Log has an example from a comic strip, while Alex Segal, in a comment, shares several examples of non-restrictive that and discusses their distribution and grammaticality.