Here the words of Everett Ferguson are appropriate: “Early Christian inscriptions, which in the largest numbers come from the environs of Rome, furnish some instances of child and infant baptism for the third century . . . Nearly all the early Christian inscriptions are epitaphs. A considerable number of these are for the graves of children. The vast majority give no evidence whether the child was baptized or not . . . Actually the word “baptism” is seldom used. The idea is expressed by “received grace,” “made a believer” or “neophyte” (newly planted ” used to mean “newly baptized”) — from Everett Ferguson, Early Christians Speak: Faith and Life in the First Three Centuries; Revised Edition (Abilene: ACU Press, 1984) .
To the sacred dead. Florentius made this monument to his worthy son Appronianus, who lived one year, nine months, and five days. Since he was dearly loved by his grandmother, and she saw that he was going to die, she asked from the church that he might depart from the world a believer. (ILCV I:1343, from the third century; edited by E. Diehl (second edition; Berlin, 1961))Postumius Eutenion, a believer, who obtained holy grace the day before his birthday at a very late hour and died. He lived six years and was buried on the fifth of Ides of July on the day of Jupiter on which he was born. His soul is with the saints in peace. Felicissimus, Eutheria, and Festa his grandmother to their worthy son Postumius. (ILCV I:1524, from the early fourth century)
Sweet Tyche lived one year, ten months, fifteen days, Received [grace] on the eighth day before the Kalends. Gave up [her soul] on the same day. (Inscriptiones latinae christianae veteres, Vol. I number 1531)
Irene who lived with her parents ten months and six days received [grace] seven days before the Ides of April and gave up [her soul] on the Ides of April. (ILCV I:1532)
To Proiecto, neophyte infant, who lived two years seven months. (ILCV I:1484)
All of these inscriptions are from the early life of the Church. If they had the covenant view of baptism and children who participating in the church life of their children, then baptizing infants would not be out of the ordinary.