From Chapter 1:
Another place making a similar claim is Hemstridge, on the Somerset and Dorset border. Just before reaching Hemstridge from Milborne Port, at the cross-roads, there is a public-house called the Virginia Inn. There, it is said, according to Mr. Edward Hutton, in his "Highways and Byways in Somerset," "Sir Walter Raleigh smoked his first pipe of tobacco, and, being discovered by his servant, was drenched with a bucket of water."

From Chapter 7:
There is no mention of Parson Trulliber's pipe, but that pig-breeder and lover can hardly have been a non-smoker. Both the other clerical characters who appear in the book, the Roman Catholic priest who makes an equivocal appearance in the eighth chapter of the third book, and Parson Barnabas, who thinks that his own sermons are at least equal to Tillotson's, smoke their pipes. The other smokers in "Joseph Andrews" are the surgeon and the exciseman who, early in the story, are found sitting in the inn kitchen with Parson Barnabas, "smoking their pipes over some syderand"—the mysterious "cup" being a mixture of cider and something spirituous—and Joseph's father, old Gaffer Andrews, who appears at the end of the story, and complains bitterly that he wants his pipe, not having had a whiff that morning.