Well Voldy would have murdered any wizarding family, (plus Mrs Figg, as she was part of the Order,) but not any random muggle family. Voldy's soul still existed, and by that point (at least 1981-1990) he took Quinirus Quirrell's body and could kill people if he wanted (but not Harry, because of a combination of love, sacrifice, and the Bond of Blood,) and I think the bond of blood only worked with relatives. That is why Harry had to stay with the Dursleys.