"Did Jesus Exist? (5): Nazareth

Another concern I have with Did Jesus is Exist? is how quick Ehrman is to label an argument irrelevant to whether Jesus existed when the worst that can be said about it is that it's not dispositive of whether Jesus existed.  One of the most frustrating examples of this is his discussion of the argument that Nazareth did not exist at the time Jesus is claimed to have lived there (an argument about which I have no strong feeling).  Ehrman writes: 

I could dispose of this argument fairly easily by pointing out that it is irrelevant.  If Jesus existed, as the evidence suggests, but Nazareth did not, as this assertion claims, then he merely comes from somewhere else.  (p.191)

This sounds quite logical except for one thing: Nazareth is frequently cited as one of those things that can be known with relative certainty about the historical Jesus.  Ehrman makes this very point a couple of pages earlier:

Nazareth was a little one-horse town that no one had ever heard of, as far as we can tell before Christianity.  The savious of the world came from there?  Not from Bethlehem?  Or Jerusalem? Or Rome?  How likely is that?  And so we have a multiply attested tradition that passes the criterion of dissimilarity.  Conclusion:  Jesus probably came from Nazareth. (p.189)

Now technically I suppose that Ehrman isn't claiming in this passage that Nazareth constitutes evidence Jesus existed.  If he were, his earlier quoted claim would boil down to the self-refuting "If Jesus existed, as the existence of Nazareth suggests, but Nazareth did not, the he merely comes from somewhere else."   Nevertheless, he is citing it as something that we can know about the historical Jesus.  It seems like more than a little smoke and mirrors to claim that the validity of the things that we think we can know about Jesus are irrelevant to whether we can know that he existed.

I will concede that the non-existence of Nazareth would not be dispositive of Jesus' existence, but surely it is relevant.