Fudge was only worse in the 5th book when he refused to believe that Voldemort was back just so that the magical community wouldn’t panic, which was pretty stupid. Overall, probably him though.
Scrimgeour was a jerk, but at least he meant well. He was trying to preserve the Ministry and fight Voldemort. Fudge really only cared about his own political power.
