The heterogeneity inherent in tau positron emission tomography (PET) imaging data across different tracers challenges the integration of multi-site tau PET data, thereby necessitating the trustful harmonization technique for a better utilization of the emerging large-scale datasets. Unlike other imaging modalities, the harmonization among multi-site tau PET data involves more than intensity mapping but contains intricate pattern alterations attributed to tracer binding properties, which makes the existing statistical methods inadequate. Meanwhile, the effective data preprocessing is required to eliminate the artifacts caused by off-target binding and partial volume effect for meaningful comparison and harmonization. In this paper, we propose a systematic tau PET harmonization framework that involves the surface-based data preprocessing and diffusion model for generating the vertex-wise mapping between multi-site tau standardized uptake value ratio (SUVR) on the cortical surface. In the experiments, using large-scale Alzheimer’s Disease Neuroimaging Initiative (ADNI) and Health and Aging Brain Study-Health Disparities (HABS-HD) data with different tracers, we demonstrate our method can successfully achieve harmonization by generating the SUVR maps with consistent pattern distributions and persevering the individual variability.