Here is a brief quote from an official LDS source on the priesthood that mentions two Bible passages to support their belief that theirs is the only true church:
“When Jesus Christ was on the earth, He established His Church and gave His Apostles the priesthood, which is the power to act in His name (see Luke 9:1–2). After His Crucifixion and the death of His Apostles, priesthood authority and the fulness of the gospel were taken from the earth because of wickedness and a rejection of the truth (see 2 Thessalonians 2:1–3).”
This is what Luke 9:1-2 (NIV) says:
When Jesus had called the Twelve together, he gave them power and authority to drive out all demons and to cure diseases, and he sent them out to preach the kingdom of God and to heal the sick.
This is what 2 Thessalonians 2:1-3 (NIV) says:
Concerning the coming of our Lord Jesus Christ and our being gathered to him, we ask you, brothers, not to become easily unsettled or alarmed by some prophecy, report or letter supposed to have come from us, saying that the day of the Lord has already come. Don’t let anyone deceive you in any way, for that day will not come, until the rebellion occurs and the man of lawlessness is revealed, the man doomed to destruction.
Although the quote below is not taken from the Bible, it goes some way to explaining the LDS view:
During the time of the Apostasy, precious doctrines were lost from the Bible through carelessness, uninspired translation, or deliberate efforts to erase the truth. A restoration of that lost doctrine and truth was necessary. The Book of Mormon and other scripture revealed to the Prophet Joseph Smith brought back many of those plain and precious parts of the gospel....
Today we can be members of “the only true and living church upon the face of the whole earth” (D&C 1:30) because the Lord restored His gospel and conferred priesthood authority to the Prophet Joseph Smith to organize His Church (see D&C 27; D&C 65; D&C 128:18–21). We are blessed to live in this time, when the gospel in its fulness has been restored—a time when you can go to church, bear your testimony, and read the scriptures."
Because the LDS eighth article of faith says, “We believe the Bible to be the word of God as far as it is translated correctly” perhaps that is why they rely on the Book of Mormon?
Another LDS article about the restoration of the gospel through Joseph Smith has this to say:
“In the early 1800s, Heavenly Father and Jesus Christ restored the gospel to the earth through the Prophet Joseph Smith. This Restoration included the restoration of priesthood power and authority.
While Joseph Smith and Oliver Cowdery were working on the translation of the Book of Mormon, they read about baptism for the remission of sins. On May 15, 1829, they went to a wooded area near Joseph’s home in Harmony, Pennsylvania, and prayed about what they had learned.
In answer to their prayer, John the Baptist, under the direction of the Lord Jesus Christ, appeared and conferred upon them the Aaronic Priesthood, saying, “Upon you my fellow servants, in the name of Messiah I confer the Priesthood of Aaron, which holds the keys of the ministering of angels, and of the gospel of repentance, and of baptism by immersion for the remission of sins” (Doctrine and Covenants 13:1) ...
Sometime after John the Baptist’s appearance, the ancient Apostles Peter, James, and John also appeared to Joseph and Oliver, again under the direction of Jesus Christ, and conferred upon them the Melchizedek Priesthood (see Doctrine and Covenants 128:20). “The Melchizedek Priesthood holds the right of presidency, and has power and authority over all the offices in the church in all ages of the world, to administer in spiritual things” (Doctrine and Covenants 107:8). With this authority again on the earth, the Church of Jesus Christ could be restored in its fulness.”
Unfortunately, I have been unable to find any reference from the Holy Bible but I continue to look.