This paper presents a customized image-processing algorithm for the high-speed and real-time detection of pavement surface distresses. The algorithm was developed based on the “grid cell” analysis, in which a pavement image is divided into a grid of 8 × 8 pixel cells, and each cell is classified as a non-crack cell or a crack cell based on the statistics of the grayscales of the cell pixels. A crack cell can be regarded as a seed for crack formation. Adjacent crack seeds or seed clusters are connected to a crack segment. Each segment has it own direction and contrast traced from all seed in the path. A full crack is a connection of nearby segments with similar directions and contrasts. Most importantly, there must be a clear crack path along these segments. Because many operations are performed on the grid cells rather than on the original image, the algorithm can detect the cracks in the current image during the time when the camera is capturing a new image. Therefore, the survey can run at real time at a highway speed. The trial test results showed a good repeatability and accuracy when the system conducts multiple surveys and runs at different speeds and different weather conditions.