Denisovans, named after the Siberian cave in which they were discovered, are a distinct group of hominins that were identified through aDNA. Analysis of the ancient mtDNA from teeth and bone fragments revealed they had haplotypes outside the range of variation of modern humans and Neanderthals. The phalanx bone from which the DNA of the Denisovan was recovered did not have traits that indicated that it was another species. A haplotype is a set of genetic variants located on a single stretch of the genome. Shared or similar haplotypes can be used to identify ancestral relationships and to differentiate groups. Dubbed lineage X, the mtDNA sequence from these fossils suggested that Denisovans diverged from modern humans and Neanderthals.
Subsequent high-coverage sequence of a Denisovan (Denisovan 3) nuclear genome showed that Denisovans are a sister group to Neanderthals and thus more closely related to them than indicated by the mtDNA data (Brown et al. 2016). Because mtDNA and nuclear DNA have different patterns of inheritance, they can paint different pictures about the relationships between two groups. The Denisovans are now thought to have a mtDNA sequence derived from an ancient hominin group that hybridized with Denisovans and introduced the mtDNA sequence.
Sequences from three other Denisovans (Denisovan 2, 4, and 8) that provide insight into how old the specimens are, along with the usual dating methods (such as radio carbon dating and uranium dating), show that Denisovans occupied the Denisovan cave from around 195 kya to 52 kya to 76 kya. DNA can assist with dating because, compared to older sequences, younger sequences will have accumulated more mutations from the putative common ancestral sequence. Thus, it is possible to conclude from sequence data, that Denisovan 2 is 54.2 kya to 99.4 kya older than Denisovan 3, and 20.6 kya to 37.7 kya older than Denisovan 8. Molecular data indicates that Neanderthals and Denisovans separated between 381 kya and 473 kya and that the branch leading to Denisovans and modern humans diverged around 800 kya. Denisovans are also more closely related to another set of fossils found in the cave Sima de los Huesos dated to 430 kya. Thus, the split between Neanderthals and Denisovans must have occurred before 430 kya (Meyer et al. 2016).
Ancient DNA has helped us understand the demographics of Neanderthals and Denisovans and make inferences about population size and history. Three lines of evidence suggest that these groups had small populations toward the end of their existence.
The first line of evidence uses coalescent methods. This process is used to determine which population dynamics in the past are most likely to give rise to the genetic sequences we have, and it allows us to understand population changes of the past, including recombination, population subdivision, and variable population size.
The second indicator that Neanderthals and Denisovans had smaller population sizes is that these groups carried many deleterious (or harmful) genomic variants. Genomic variants are considered deleterious when the change in genomic sequence alters the amino acid sequence of a protein and affects the function of the protein; such variants are known as nonsynonymous mutations. By contrast, synonymous mutations that occur in protein-coding regions of the genome do not change the amino acid sequence nor affect the proteins produced. Denisovans and Neanderthals have a higher ratio of nonsynonymous to synonymous mutations when compared to contemporary modern human populations. This is indicative of a small population size, because if the population were larger, natural selection would have acted on these deleterious variants and weeded them out.
A third indicator of small population size is that the Neanderthals sequenced thus far have low levels of heterozygosity. Heterozygosity is measured by looking at how often two different alleles are found within a certain stretch of DNA. When you find many regions on the genome with different alleles, there is a high level of heterozygosity. When you find very few positions where there are two different alleles, this indicates a low level of heterozygosity. Both Neanderthals and Denisovans appear to have low levels of heterozygosity, indicating smaller population sizes. Ancient Neanderthal genomes also revealed that there were consanguineous relations (mating relationships between two closely related Neanderthals). This was determined by looking at the stretches of homozygosity in a female neanderthal’s genome that were longer than expected and could not be explained by small population size alone.