Victorian England. Merchant ship the Esther Lohse docks at Southampton. Ex-sea captain William Targett disembarks with a young black boy; concealed in their luggage is an even younger black toddler.

They journey by train to a village in Dorset, their progress attracting curious crowds. Targett and the boy enter a dress shop, whereupon the boy is transformed into a pretty girl, his wife, Tulip.

William rents a village pub, The Sailor's Return, with the aim of providing a living for the family. He is able to do this because Tulip comes from a wealthy family who gave him a small fortune as Tulip's dowry. William hires Tom, a local lad to help in the pub. Though Tulip finds English customs strange and is very homesick, she and Tom get on well together.

William's sister, Lucy Sturmy, who lives in a nearby village, is deeply opposed to his marriage. She fears for her good name and gives him an ultimatum: either he sends the black woman and her piccaninny back to Africa or he does not come to her house again. William firmly rejects her offer.

The Taggarts receive an unexpected visitor, Harry, William's younger brother. The three stay up drinking and enjoying each other's company. Later that night William rides with Tulip to his sister's house, where he causes a commotion. At daybreak the couple make love on the beach and declare their love to each other.

The village divides over the presence of the Taggarts. A few are welcoming; the majority react with hostility and racism. Customers stay away from the pub. One day Tulip and her child, Olu, stumble on the churchyard. Reverend Pottock tells Tulip that Olu will die in a burning fire unless he is christened. Tulip is very distressed and thinks that the reverend intends to do them harm.

One night a village mob surrounds the pub with blazing torches. After a tense standoff, William manages to disperse the crowd and threatens legal action should anything happen to his family. The next day William reiterates the threat to Reverend Pottock, whom he blames for inciting the villagers with his metaphorical talk of burning.

William makes an effort to fit in with the village customs. He has Olu christened; he and Tulip remarry in the village church and their new baby Sheba is baptised. Sadly, the baby dies soon after, causing Tulip to fall into a depression. By now the Taggarts are in desperate need of money, and William pawns Tulip's jewellery.

Relations with the villagers continue to deteriorate. In a relaxed moment Harry reveals that while he loves Tulip he believes that William was very wrong to bring her from Africa.

One day Tulip has an altercation with a customer who refuses to be served by her. She throws the beer at him and he threatens her. The feud escalates into a fistfight between William and an ex-boxer, an ally of the customer. William wins the fight but is then ambushed and killed.

After the funeral, Tulip decides to flee the village back to her home in Africa. She and Olu make it to Southampton. However, at the docks the captain refuses to take a female on board. The captain agrees to take the boy and deliver him safely to his relatives. Tulip gives him the rest of her money and returns to The Sailor's Return, where she spends the rest of her days as a maid to the new landlords.