This post will thus attempt to show how genuinely Christian the practice is, by showing how early the practice is and pointing to the Scriptural support given for it.
Tertullian, from his Apology, says the following.
Others certainly, with greater semblance of nature and of truth, believe the sun to be our God. [But] If this be so, we ought be ranked with the Persians; though we worship not the sun painted on a piece of linen... this suspicion arises from hence, because it is well known that we pray towards the quarter of the east... In a like manner, if we give up to rejoicing [on] the day of the sun, for a cause far different from the worship of the sun, we are only next to those who set apart the day of Saturn for resting and feasting, deflecting themselves also from Jewish customs, of which they are ignorant. (Apology, chapter 16, c. A.D. 197)
Turns out, Tertullian and the other Christians of his day were accused of "sun-worship" just like Catholics and other Christians are by certain outlier groups. Tertullian, however, highlights just how arbitrary such notions are; Christians worshipping on Sunday has as little to do with the sun as those who take their rest on Saturday have to do with the god Saturn.
Tertullian's testimony has an added dimension of significance. Most of the Church Fathers, though they consistently repudiate the false pagan worship of the surrounding Greco-Roman culture, nevertheless respected certain elements found within the culture, namely philosophy. Tertullian, however, was NOT one of them. Being a purist, he famously said "What hath Athens to do with Jerusalem?" (On the Prescription of Heretics, chp 7) The fact that he, out of all people, would admit that Christians pray facing east without giving even a second thought makes for a good argument that, as a Christian practice, it did not originate out of paganism.
And since the dawn is an image of the day of birth, and from that point the light which has shone forth at first from the darkness increases, there has also downed on those involved in darkness a day of knowledge of truth. In correspondence with the manner of the sun's rising, prayers are made looking towards the sunrise in the east. Whence also the most ancient temples looked towards the west, that people might be taught to turn to the east when facing the images. Let my prayer be directed before You as incense, the uplifting of my hands as the evening sacrifice, say the Psalms. (The Stomata, Book VII, 7, 43, c. AD 198)
Clement explains that praying towards the east carries a symbolic meaning: Those who have lived in darkness have seen a great light (Is. 9:2) so to speak, as the truth of Christ's Gospel has illuminated those whose minds and souls were darkened by false worship. This ties in well with any of the Scriptural concepts which might be brought forth as the foundation of the practice: that Jesus Christ is the Light of the World (Jn 8:12), a Sun of Righteousness (Mal 4:2) to banish the dark night of immorality.
Clement then actually bases the practice on the tradition of the ancient Jews. Jewish worship of old (as well as contemporary) was done facing the east. The tabernacle on Mt. Sinai faced east, the images referred to by Clement being those such as the cherubim woven into the curtains (Ex. 25-26). He's pointing back to it as something which was originally employed by the Hebrews in the Old Covenant.
These are the two earliest testimonies to Christians praying eastward that I have been able to find. They are both written within years of each other, right before the close of the second century AD (long before Constantine or the Council of Nicaea.) Tertullian is writing from the Latin West, while Clement is writing in the Greek-speaking East, meaning the tradition is widespread and existing in multiple churches. They both mention the practice but do not devote a lot of time to it, as John Damascene would later on book four in his Exposition on the Orthodox Faith in the 7th Century, which might imply that the practice is actually rather fixed.
So, there's pretty good evidence that the Christian practice of facing east in prayer stemmed from the Jewish equivalent, and, in a word, is definitely NOT of Pagan origin! Yes, different Pagan cultures practiced it as well, but that does not mean it can't exist in a Judeo-Christian context -- after all, if being Holy depended on being unique in every external respect, then things like temples, incense, circumcision and baptism -- all things the Jews and Pagans used alike -- would have to be dismissed and thrown away.
One other early source, the Didascalia (based off of the earlier Didache) has a section where it touches on the liturgical facing east. This document is usually dated to the year 230, and is most commonly believed to have originated in Syria, perhaps Antioch. From its twelfth chapter, the Didascalia Apostolorum reads thus:
And in your congregations in the holy churches hold your assemblies with all decent order, and appoint the places for the brethren with care and gravity. And for the presbyters let there be assigned a place in the eastern part of the house; and let the bishop's throne be set in their midst, and let the presbyters sit with him. And again, let the lay men sit in another part of the house toward the east. For so it should be, that in the eastern part of the house the presbyters sit with the bishops, and next the lay men, and then the women that when you stand up to pray, the rulers may stand first, and after them the lay men, and then the women also. For it is required that you pray toward the east, as knowing that which is written: Give ye glory to God, who rideth upon the heaven of heavens toward the east [Ps 67.34 LXX]
So, Tertullian in Carthage (very Northern Africa, within Rome's vicinity), and Clement of Alexandria (in Egypt) both write around the same time that Christians pray East, and roughly thirty-five years later we observe that Christians in Syria do the same thing. This doesn't mean they adopted it only by that time, but it is only by these times that we see the practice documented at all. The fact that it's so early and so widespread, I personally believe, says quite a bit!