Harry only remembered his parents being murdered (his dementor memory) because he’s a Horcrux of Voldemort.

Now, Human beings generally cannot remember anything from when they were 4 or younger, and absolutely nobody remembers their infancy. Even if they could, they wouldn’t be able to remember words people said; to a baby, words just sound like gibberish. But Harry became a Horcrux of Voldemort, and we know Horcruxes contain memories from before they split off from the main soul piece. Harry’s worst memory just so happened to be Voldemort’s worst memory: the night Voldemort lost his body and his powers was the same night, and the same event, as Harry’s parents being murdered.