The effect of branch lengths on phylogeny: an empirical study using highly conserved orthologs from mammalian genomes

Phylogenetic analyses were applied to 269 families of putative orthologs represented by a single member in the genomes of human, mouse, dog, and chicken. Five methods were used: maximum parsimony (NP), neighbor-joining (NJ) with Poisson and Gamma distances; and maximum likelihood (ML) with JTT and JTT+gamma models. When applied to the concatenated sequence of all families, all methods strongly supported a tree in which mouse branched before human and dog. In analyses of individual families, the same topology was supported more than any other. Although there was evidence of an increased rate of amino acid replacement in the mouse lineage in comparison to the other two mammals, there was no evidence that support for the mouse's basal position was due to long-branch attraction; rather, this topology was seen in the families with the lowest rate variation among the three mammalian branches. In families with highly divergent mouse sequences, ML with both JTT and JTT+gamma and NJ with the gamma distance tended to support a topology in which the dog, rather than the mouse, branched first. Thus, in these data, a tendency of long and short branches to cluster together ("opposite-branch attraction") seemed to be more of a problem than long-branch attraction.