Kate Riley’s début novel, “Ruth,” is about the workings of an insular religious community—and the irresistible pleasure of making up rules.