They are both discussing an issue raised by Richard Dawkins in The God Delusion. In his section on "The Poverty of Agnosticism" (pp. 46-54), Dawkins describes agnostics as fence-sitters, and this was not meant as a compliment. Dawkins knows full well that there is a deep metaphysical sense in which we can never know anything for certain.
If we're all being perfectly philosophical, then we have to admit to being agnostics about the tooth fairy and Santa Claus. But what good is that? Do we really go around telling everyone that we just don't know whether Santa Claus will visit on Christmas Eve? Of course not. We don't believe in Santa Claus, even though we can all write an essay in Philosophy 101 about not being able to prove a negative.
This is what Dawkins means when he says, "I am agnostic only to the extent that I am agnostic about fairies at the bottom of the garden" (p. 51). He makes the same point a few pages later when he says,
That you cannot prove God's non-existence is accepted and trivial, if only in the sense that we can never absolutely prove the non-existence of anything. What matters is not whether God is disprovable (he isn't) but whether his existence is probable. That is another matter. Some undisprovable things are sensibly judged far less probable than other undisprovable things.John knows this but, nevertheless, he says, "Do I think there is a God? No, I don't. Am I an atheist? No, I'm not."
John, with all due respect, if you walk like an atheist and talk like an atheist then, to all intents and purposes, you're a practicing atheist, whether you want to admit it or not. You can be an agnostic atheist in the sense that you act as if there's no God but still want to be true to your profession. They won't drum you out of the philosophers' union if you confess your atheistic lifestyle as long as you make the right noises from time to time. I've was with you at a conference of philosophers last year and we met several atheists who were still card-carrying philosophers.
We spent a whole Sunday together and I know you didn't go to church. You are not a theist. The word that describes that non-believer lifestyle is "atheist," not "agnostic." Please join Jason Rosenhouse, Richard Dawkins, and me, and come all the way out of the closet. :-)