Semi-rigid base materials have been widely used in recent years. However, this material is easy to crack, which adversely affects the performance of the pavement. In this paper, an innovative reclaimed phosphogypsum-based material is used as an anti-cracking agent added in the semi-rigid base. Several laboratory tests and field observations have been taken to evaluate the potential of the material that we are proposing to be used as an anti-cracking agent. The shrinkage tests show that the shrink space of the road base caused by the variations of the temperature and humidity has decreased dramatically because of the involvement of the anti-cracking agent. The influence of the delay on the molding time and on the strength of gravel mixtures is also analyzed, and the results show that the anti-cracking agent has a slow setting performance. By adding an anti-cracking agent into the semi-rigid base material, early strength of the material is enhanced, curing period is shortened, and lasting quality is improved. The continuous construction of the base and asphalt pavement are technically feasible in the testing road because of the excellent expansion, densification, and water-resistance characteristics of the anti-cracking agent. As a result, cracks in the pavement can be retarded or lightened, and, thus, the pavement service life can be extended.