These are historic times for the Mormon Church; it is in the midst of an ongoing reformation the likes of which the world has not seen since the Protestant Reformation. Interestingly enough, the original calling of Joseph Smith was to effect a reformation, not a “restoration”:
And thus, if the people of this generation harden not their hearts, I will work a REFORMATION among them, and I will put down all lyings, and deceivings, and priestcrafts, and envyings, and strifes, and idolatries, and sorceries, and all manner of iniquities… (D&C 5:20, Joseph removed that sentence, see Book of Commandments IV:5)
When the Book of Mormon came forth, it had been 313 years since Martin Luther posted his 95 theses on the door of the Wittenberg Chapel in Germany (October 31, 1517), and 293 years since the martyr John Rogers published the first authorized English Bible (Thomas Matthew Bible 1537).
Mormons who leave the Church are currently doing so out of disdain for the later acts of Joseph Smith and are missing the real reason they should be upset. Yes, he was an adulterer, but so was King David. The main reason Church members should be troubled is that Joseph went beyond the scope of his original calling, which was to bring forth the Book of Mormon; and then to preach on it (like Billy Graham, a well known preacher who never started a church):
8 And I will give unto him a commandment that he shall do none other work [a], save the work which I shall command him.
11 But a seer will I raise up out of the fruit of thy loins; and unto him will I give power to [b] bring forth my word unto the seed of thy loins—and not to the bringing forth my word only, saith the Lord, but to [c] the convincing them of my word, which shall have already gone forth among them. (2 Nephi 3)
Originally Joseph understood this limitation and said so to David Whitmer and others:
After the translation of the Book of Mormon was finished, early in the spring of 1830, before April 6th, Joseph gave the stone to Oliver Cowdery and told me as well as the rest that he was through with it, and he did not use the stone any more. He said he was through with the work that God had given him the gift to perform, EXCEPT TO PREACH THE GOSPEL. (David Whitmer, Address to All Believers in Christ, 1887, p. 32)
Oliver Cowdery was the one called to start a church (Summer of 1829). His calling started like this:
A Commandment from God unto Oliver how HE should build up his church & the manner thereof…
And ended with:
Behold I am Oliver I am an Apostle of Jesus Christ by the will of God the Father & the Lord Jesus Christ Behold I have written the things which he hath commanded me for behold his word was unto me as a burning fire shut up in my bones & I was weary with forbearing & I could forbear no longer Amen Written in the year of our Lord & Saviour 1829 — A true Copy of the Articles of the Church of Christ.
The full calling can be found here: Articles of the Church of Christ. The idea now is to return the Church to what it was meant to be based on, the Bible and the Book of Mormon.