In the multi-passed weld heat-affected zone (HAZ), the occurrence of mechanical and metallurgical heterogeneity is inevitable. For the fracture toughness evaluation of the steel weld HAZ, the effect of the inhomogeneity shall be considered, particularly in relation to determination of the appropriate testing procedures such as the extraction of test specimens from welded joints. In the weld HAZ of the recently developed high strength steels, the occurrence of the local brittle zones (LBZs) and the local hard/soft zones is often observed. The effect of local heterogeneity is an important problem to be solved to clarify the testing methods and understand the significance of toughness obtained by a specific toughness testing method. In this paper, two kinds of heterogeneity, that is, heterogeneity in toughness and strength mismatch, will be considered. The toughness value is significantly dependent on the specimen types and the extraction method of specimen from the multi-passed welded joints, that is, the size of LBZs existing in the vicinity of the crack tip is one of the dominant controlling factors. The size effect of LBZs on the scattering of toughness is discussed based on the weakest link model (WLM). Moreover, the mechanical heterogeneity, such as the distribution of strength in HAZ, is a considerably important factor controlling the fracture toughness. Mechanical inhomogeneity influences not only the deformation behaviors in the vicinity of crack tip, that is, the evaluation of toughness parameter, but also the critical toughness value. In this paper, both effects are proved by the experimental results, and discussions have been conducted on the appropriate evaluation methods for CTOD testing of steel weld HAZs with considering the effect of the local mechanical heterogeneity.