The evaluation of fatigue performance between asphalt pavement layers highly depends on fatigue life and shear stiffness. However, fatigue life and shear stiffness are comprehensive evaluation indexes. In the study of the formation mechanism and decay process of fatigue resistance, both of them cannot provide effective theoretical support. In this study, the shear stiffness decay curve was obtained through a four-point shear fatigue test under different loading conditions. The initial shear stiffness and the shear stiffness decay rate of interlayers were chosen to evaluate the initial resistance and process resistance. The effects of temperature, loading frequency, compressive stress, and shear stress on the evolution of interlayer fatigue performance were investigated. The findings of this study revealed that first, with the increase of the compressive stress perpendicular to the interlayer interface, the interlayer shear stiffness decay curve transitions from a cubic curve type to a quadratic curve type and gradually changes to a linear equation. Second, the ranking of influence degree of four factors on initial resistance and process resistance was obtained, and the influence mechanism analysis was carried out based on the variance analysis result. Finally, a multiple nonlinear regression model was established successfully to predict the fatigue life and to explain the internal connection between the initial shear stiffness, shear stiffness decay rate, and fatigue life.