Do te think if Severus hadn't lashed out when calling Lily a mudblood and drawed attention to himself James would have....

I don't think so. I don't think he'd have hexed her first, however he was arrogant and proud enough that if Lily were to hex him, then he would retaliate. He gave Lily that warning because he knew himself well enough and did not want to find himself in that situation with her.

No. For starters, I think if James was really going to hex her, he would have actually done it.

James also likes a show. With Lily standing up to him, calling him out in front of everyone, all eyes were on him. Everyone was able to see what he was doing, hearing what he was saying. He very much liked that Lily stood up to him, he wasn't going to make her stop.

And, lastly, unlike with Severus, people would have stood up for Lily. Had James actually hexed Lily, Lily's Friends would have rushed to her defense. And that would have made the odds even. And James hated fighting when he didn't have the upper-hand in numbers.

No, because:

1) James found Lily attractive,
2) James poked fun of Snape because Severus was into Voldemort and Death Eater politics; Lily wasn't, and
3) Snape was James's specific target. James made fun of others, but grew out of that. James never stopped with Snape.

Plus, James married Lily, so... uh...
Yeah...