If we imagine our child to be like an illiterate medieval peasant, we can point them towards the traditional stained glass, statues, stations, and other art and architecture around them as an invitation to contemplate the Divine.
Arriving early at Mass can provide a good opportunity to quietly identify the the altar, tabernacle, candles, etc…
Sacramentals are also handy for quietly engaging the young child.
We have made judicious use of free holy cards on the table at back of church, and when we remember to bring them, a child friendly rosary is great for teething or fidgeting. I also find young ones who cannot even read are somehow thrilled to flip endlessly through a hymnal on occasion. We just make sure they treat the book with care.