In Harry's second year, Dobby came and use a charm in the Dursleys' house, causing Harry in trouble with the MOM since they tracked who cast the spell due to the address and thought Harry had did the charm
In Harry's fourth year, Mr. Weasley came pick up him to the Burrow for the Quidditch World Cup. He use a spell to repair Dudley's tongue. There's no letter. In Harry's 6th year, when Dumbledore went to pick up him, he did use magic in Privet Drive. No letter
What's happening?