When we speak of God, we use a masculine word, but let no thoughtless person accuse us of saying that God, is a man. It is not gender that is expressed but rather his name, its customary meaning and the way in which we habitually use words. The deity is not male, even though his name is of the masculine gender. In contrast, [pagans] attribute gender to the gods, by calling them either “god” of “goddess.” We cannot believe that God has a body, because if he did, he would have to be either male or female. (Against the Nations, 3.8, ACD vol 1, p38)
We should remember that he is a new convert, coming directly from the rhetorican school of the Greeks.
Update – Suzanne has posted on this passage.