"Back on the Chain Gang" was recorded after James Honeyman-Scott, the Pretenders guitarist, died of a drug overdose at the age of 25 on June 16, 1982. This came two days after the Pretenders fired their longtime bassist, Pete Farndon, because of his drug abuse problem. On July 20, 1982, the band began recording the song at AIR Studios in London. At that time, only two Pretenders were left: singer-songwriter Chrissie Hynde who was about three months pregnant with her first daughter, and drummer Martin Chambers. Other musicians were hired to fill out the session: lead guitarist Billy Bremner of Rockpile, guitarist Robbie McIntosh, and bassist Tony Butler who was already at the studio for a Big Country recording project. The producer was Chris Thomas who was familiar to the band from his integral role in making the Pretenders' earlier records, using Bill Price as his engineer, but for this session Steve Churchyard replaced Price because Price was committed to another AIR project at Wessex Sound Studios.
Most of the song was recorded quickly with the band placed close together in the studio, arranged as if performing live, with Chambers' drums up on a riser. Small loudspeakers were aimed at the musicians from behind Chambers to reinforce the sound of selected drums such as the snare. Bremner's featured guitar solo was performed in one take. Later, alone in the studio as was her preference, Hynde performed her main vocal line with three or four overdubs dropped in to fix minor imperfections. She then recorded her own backing vocals. Finally, the rest of the backing vocals were performed by Chambers and Butler, along with the chain-gang chant. The sound of clanging hammers was made by banging various metal pieces together, especially the 25-pound (11 kg) weights that the studio used as ballast for large boom stands. The recording of extra parts for the song and the final mixing process continued for several days after initial recording began.
Hynde wrote "Back on the Chain Gang" as a memorial to Honeyman-Scott, and she dedicated it to him.
The hammering sounds and the chain-gang chant heard during the chorus of the song echoes the earlier production of Sam Cooke's song "Chain Gang", released in 1960.
In an interview with Guitar Player in 1992, George Harrison claimed that "Back On The Chain Gang" utilizes a chord that he had "invented" and incorporated into the Beatles song "I Want to Tell You": "That's an A7 with an F on top and I'm really proud of that because I invented that chord… There's only been one other song, to my knowledge, where somebody copped that chord – Chrissie Hynde and the Pretenders on 'Back On The Chain Gang.' However, the chord Harrison describes is widely known as A7+5 or augmented A7 or A7#5 and is a standard use of harmony in many genres.