The reuse of waste tire rubber and plastic in modified asphalt has the potential to enhance the sustainability and durability of asphalt pavement. Dry modification of asphalt has significant benefits in the simple production and construction ease while the mechanical properties are largely determined by the modification process. Herein, the dry-modification process of waste tire rubber and ethylene-vinyl acetate is optimized through laboratory tests and microscale characterization, thereby unraveling the modification mechanisms and mechanical properties. The dry modification process of rubber and plastic in asphalt was optimized by the orthogonal experiments based on the dynamic stability, maximum flexural tensile strain, and freeze-thaw splitting strength ratio. The rheological tests, microscale characterizations, pullout test, and compactness test indicate that the rubber-plastic modifier acts as the binder and particle filler in the modified asphalt, thereby enhancing durability and adhesion. The mechanical properties of rubber-plastic dry and wet modified asphalt mixture were comprehensively evaluated, showing that the rubber-plastic dry modified asphalt mixture demonstrates similar properties as compared to the wet modified asphalt mixture. The synergetic effect of rubber and plastic is proven to enhance the mechanical properties of asphalt, in which rubber particles contribute to better high-temperature performance and the plastic modifier improves the low-temperature performance and water stability. This study optimized the dry modification process of rubber and plastic in an asphalt mixture, thereby achieving durable and sustainable pavement construction.