It was spoken by Gable, as Rhett Butler, in his last words to Scarlett O'Hara. It occurs at the end of the film when Scarlett asks Rhett, "Where shall I go? What shall I do?" if he leaves her. The line is memorable not only because it contains profanity (which was generally not allowed in films of that time period), but because it demonstrates that Rhett has finally given up on Scarlett and no longer cares what happens to her.
In the novel Gone with the Wind, Rhett does not say "Frankly," but simply "My dear, I don't give a damn." The context is also different; he is speaking quietly to Scarlett in a room, not storming dramatically out of the house.
Prior to the film's release, censors objected to the use of the word "damn" in the film, a word that had been prohibited by the 1930 Motion Picture Association's Production Code that began to be enforced in July 1934. However, before 1930 the word "damn" had been relatively common in films. Although legend persists that the Hays Office fined producer David O. Selznick $5,000 for using the word "damn," in fact the MPA board passed an amendment to the Production Code on November 1, 1939, a month and a half before the film's release, that forbade use of the words "hell" or "damn" except when their use "shall be essential and required for portrayal, in proper historical context, of any scene or dialogue based upon historical fact or folklore … or a quotation from a literary work, provided that no such use shall be permitted which is intrinsically objectionable or offends good taste." With that amendment, the Production Code Administration had no further objection to Rhett's closing line.