A methodology for relating the microstructure of asphalt concretes to their viscoelastic behavior is described. Imaging techniques are used to capture the asphalt concrete microstructure, and the finite element method (FEM) is used to model its stress-strain behavior in the time domain. Aggregates are modeled as linear elastic, and the binder is modeled through mechanistic models as either linear viscoelastic or nonlinear viscoelastic. The binder viscoelastic properties are input into the FEM algorithm by two methods: a built-in viscoelastic function and a user-specified material characterization subroutine. The latter handles non-linearity in an iterative piecewise linear fashion, whereby the mechanistic binder model parameters are updated as a function of the strain level. For each strain level, mechanistic models are fitted to describe binder viscoelastic behavior based on dynamic shear rheometer data. The two approaches used for specifying binder viscoelastic properties into the FEM algorithm were verified by comparing binder response predictions with direct measurements. Finally, the asphalt concrete micro-structure model was verified by comparing FEM predictions of dynamic shear modulus and phase angle with measurements obtained by using a Superpave shear tester.