The bolted joint is widely used in heavy-duty CNC machine tools, which has huge influence on working precision and overall stiffness of CNC machine. The process parameters of group bolt assembly directly affect the stiffness of the connected parts. The dynamic model of bolted joints is established based on the fractal theory, and the overall stiffness of joint surface is calculated. In order to improve the total stiffness of bolted assembly, an improved particle swarm optimization algorithm with combination of time-varying weights and contraction factor is proposed. The input parameters are preloading of bolts, fractal dimension, roughness, and object thickness. The main goal is to maximize the global rigidity. The optimization results show that improved algorithm has better convergence, faster calculation speed, preferable results, and higher optimization performance than standard particle swarm optimization algorithm. Moreover, the global rigidity optimization is achieved.