Arbuscular Mycorrhizal Fungi (AMF) are well known for their ecological importance and their positive influence on plants. The genetics and phylogeny of this group of fungi have long been debated. Nuclear markers are the main tools used for phylogenetic analyses, but they have sometimes proved difficult to use because of their extreme variability. Therefore, the attention of researchers has been moving towards other genomic markers, in particular those from the mitochondrial DNA. In this study, 46 sequences of different AMF isolates belonging to two main clades Gigasporaceae and Glomeraceae have been obtained from the mitochondrial gene coding for the Cytochrome c Oxidase I (COI), representing the largest dataset to date of AMF COI sequences. A very low level of divergence was recorded in the COI sequences from the Gigasporaceae, which could reflect either a slow rate of evolution or a more recent evolutionary divergence of this group. On the other hand, the COI sequence divergence between Gigasporaceae and Glomeraceae was high, with synonymous divergence reaching saturated levels. This work also showed the difficulty in developing valuable mitochondrial markers able to effectively distinguish all Glomeromycota species, especially those belonging to Gigasporaceae, yet it represents a first step towards the development of a full mtDNA-based dataset which can be used for further phylogenetic investigations of this fungal phylum.