Late Pleistocene Neandertal-Early Modern Human Population Dynamics: The Dental Evidence

Abstract

Recent genetic studies have confirmed that there was admixture between African early modern humans and archaic populations throughout the Old World. In this dissertation, I examine European early modern human dental morphology to assess the evidence for Neandertal-human admixture. The focus of this study is not on the question of taxonomic designations of Late Pleistocene Homo, but rather on the interactions of these populations in Europe. This focus on gene flow itself redefines the Neandertal question. Rather than asking if Neandertals are a different species from H. sapiens, I focus on the nature of the interactions between archaic and modern populations, which is essential to understanding the history of modern H. sapiens regardless of species definitions.

I recorded dental metric measurements and morphology observations on 85 fossil Neandertals and early modern humans and a recent modern human comparative sample of 330 Native Americans and Spaniards. I examined each trait distribution individually and through the use of Mahalanobis D2, mean measure of divergence, principle components analysis, discriminant function analysis, k-means cluster analysis, and a population genetics program, structure.

Through these methods, I found evidence of admixture in the dental trait distributions of European early modern humans. However, it is not evident in traditional distance measures or cluster analyses. The earliest European modern humans do not follow the trend of dental reduction found throughout the Pleistocene and into the Holocene and do not uniformly classify with any fossil population in discriminant function analysis of metric traits. The non-metric trait sample size is too small to make any definitive conclusions, but a mosaic pattern of trait frequencies also suggests admixture. The recent modern human sample shows that while increased variation and a mosaic of non-metric traits persist through many generations after admixture has ceased, traditional methods of distance analysis cannot detect low levels of admixture within 200 years. The program structure is effective in finding patterns of variation within and among populations using morphological data. It will be useful for future analyses of dental traits and other fossil data, given the ability to use it with an incomplete data set.