Until recently, much of the microbial world was hidden from view. A global research effort has changed this, unveiling and quantifying microbial diversity across an enormous range of critically-important contexts, from the human microbiome, to plant-soil interactions, to marine life. Yet what has remained largely hidden is the interplay of ecological and evolutionary processes that led to the diversity we observe in the present day. In this thesis we introduce two theoretical frameworks, one at the macroevolutionary scale and the other at the mesoscopic scale where intricacies of abundances and environmental specificities begin to matter. At the macroscopic scale we identify an imbalance between gradual, ongoing diversification and rapid bursts across a vast range of microbial habitats and find universal quantitative similarities in the tempo and mode of diversification, independent of habitat type. This signature persists even when the quality and length of our sequence data and consequent resolution of the phylogeny is relatively low compared to the timescale of the processes. At the mesoscopic scale we discover a rich hierarchy of organization and niche signals in the pattern of abundances in the microbial diversity of the global ocean and in the process identify three putatively novel microbiomes.