Perverted by Language is the first Fall album to feature Brix Smith, then-wife of Mark E. Smith, who performs lead vocals on "Hotel Blöedel". However, the bulk of the album was recorded before she joined the band.
The album contains the track, "Eat Y'self Fitter", which John Peel picked as one of his Desert Island Discs; when Peel had first heard the track – in a session the band recorded in March 1983 – he stated on air that he had fainted and his producer, John Walters, had had to resuscitate him.The album was the only full-length product of the band's renewed relationship with Rough Trade, whom they had previously left in 1981. This time, they fell out over the full-length video the band wished to make for the album. The video Perverted by Language Bis went ahead funded by the group themselves and producers Ikon, who released the finished product on VHS in 1984 (a DVD edition was released by Cherry Red in 2003). By that time, the group had signed to Beggars Banquet.