Based on the panel data of 284 cities at the prefecture level and above in China during 2009 to 2019, this paper uses the systematic GMM method to empirically examine the effects of FDI inflows, local industry agglomeration patterns and their interactions on urban innovation in cities of different regions and sizes. The results show that FDI inflow has a significant positive effect on urban innovation. Compared with diversified industrial agglomeration, specialized industrial agglomeration has a more significant effect on innovation, but it has an inhibiting effect on innovation. In the interaction between FDI inflows and specialized agglomeration, the latter has a greater influence factor. Diversified agglomeration in eastern cities significantly promotes innovation, while specialization and diversification agglomeration in central and western cities do not significantly promote innovation. Diversified agglomeration can promote the innovation capacity of small cities, while the interaction between FDI inflows and specialized agglomeration has an inhibiting effect on large cities.