Covenant Presbyterian Church of Cleveland, Mississippi had its beginning in 1982 when First Presbyterian Church of Cleveland became divided over doctrinal issues and the direction in which the denomination was going. In March 1982, the Presbytery of St. Andrews suspended the session and the pastor of First Presbyterian Church, Dr. W. Wilson Benton. On March 28, 1982, a body consisting of about half of the membership of First Presbyterian Church met for worship in a local restaurant. The following Sunday, the group met at Bayou Academy's old building in Skene, Mississippi, which would be its home for over five years.

On April 25, 1982, the group organized itself into Covenant Presbyterian Church. The organizing members signed a charter and a covenant binding themselves to be faithful to the Word of God.

On May, 16, 1982, the congregation called Dr. Benton to be its pastor. His advice was invaluable to the new body organizing itself into a church. In October of that year, following a study of various Presbyterian denominations, the congregation voted to join the Presbyterian Church in America.

Because of the church's proximity to Delta State University, a ministry to the students on campus had always been important to its members. Billy Dempsey, a student at Reformed Theological Seminary in Jackson, Mississippi, was hired as an intern to work with the students at Delta State. In 1983, he was replaced by Tim Starnes, a graduate of Reformed Theological Seminary, who served as campus minister and associate minister. Reverend Starnes was ordained and installed on June 26, 1983. Dr. Benton was called to serve as pastor of Kirk of the Hills Presbyterian Church in St. Louis, Missouri, and the Reverend Starnes was called on June 8, 1986 to replace him at Covenant Presbyterian Church.

Our prayers for a home of our own were answered when we moved into our fellowship hall area on October 3, 1987. It was dedicated by Dr. Frank Barker, pastor of the Briarwood Presbyterian Church in Birmingham, Alabama, on November, 1, 1987. God blessed us richly, and the debt for the existing structure was soon paid off. The sanctuary was begun in 1990 and was dedicated on June 2, 1991 by Reverend John Sartell, pastor of the Independent Presbyterian Church in Memphis, Tennessee. Again, we had a need for additional space. A wing containing Sunday school rooms and an enlarged and renovated kitchen was dedicated by Dr. Benton on August 27, 2000.