Ok, so I noticed something recently, and it might explain why James bullied Snape, or why he specifically targeted him since it was implied that he picked on others as well. And just to be clear, this is in no way meant to justify or excuse James' bullying, just explain why he specially went after Snape.
So, in Snape's memories, we saw a scene of him, Lily, James, and Sirius riding the Hogwarts Express on their way to their first year. The dialogue went as such:
Snape to Lily: "You better be in Slytherin."
James overhearing: "Slytherin? Who wants to be in Slytherin. I think I'd leave, wouldn't you?"
Sirius in response: "My whole family have been in Slytherin."
James: "Blimey, and I thought you seemed alright."
Sirius: "Maybe I'll break the tradition. Where are you heading if you've got the choice?"
James: "Gryffindor, where dwell the brave at heart. Like my dad."
Snape makes a disparaging noise.
James to Snape: "Got a problem with that?"
Snape with a sneer: "No, if you'd rather be brawny than brainy."
Sirius: "Where're you hoping to go, seeing as you're neither."
Lily: "Come on, Severus, let's find another compartment."
Sirius: "Oooooo..."
James trips Snape as he passes by: "See ya, Snivellus!"
THAT is when James bullying of Snape began. Now, pay attention to their dialogue. Look at their interaction. James made a comment about how he wouldn't want to be in Slytherin House. And based on Slytherin's reputation of the students being sleazy and rotten, and that it's commonly known that all witches and wizards who went to Hogwarts that went bad were from Slytherin (not to mention knowing what Sirius' family is like), who can blame him for not wanting to be in it? But that's irrelevant. Look what happened next.
When James said he wanted to be in Gryffindor like his father, Snape made a sound of belittlement, and when James asked if he had a problem, Snape straight-up called Gryffindor's stupid by saying that they were brawn and not brains. By doing so, he directly insulted James' father, who James very clearly said was in Gryffindor, meaning Snape called James' father stupid. All James said prior to that was that he didn't want to be in Slytherin (and again, Slytherin's reputation is apparently common knowledge) and that's it. He didn't say anything negative to Snape or about anyone in particular. Snape, however, directly insulted James' father, making things personal. In short, he drew first blood and started the hostility between them.
THAT is why James started bullying Snape. Because Snape kicked things off between them by insulting James' father. Would James still have been a bully in general? Yeah, but the reason he went after Snape specifically was due to this interaction of Snape insulting his dad. Again, I'm not saying Snape brought this on himself or that this excuses James' bullying, but it was indeed Snape that started things between them with his insult to James' father.
Thoughts?