The Grey Mountains (or Ered Mithrin in Sindarin) was a large mountain range to the north of Rhovanion. They were the last remnants of the wall of the Iron Mountains, which once stretched all over the north of Middle-earth, but were broken at the end of the First Age.
Where the Grey Mountains met with the Misty Mountains lay Mount Gundabad, an ancient Dwarven holy site and later the capitol for the Orcs of the north. During the Third Age, the branch of the Grey Mountains west of the Misty Mountains were also known as the Mountains of Angmar, as they were within the Kingdom of Angmar.
The eastern end of the Grey Mountains was split in two branches, and in between lay the Withered Heath, where Dragons still bred. After that was a long gap, until the Iron Hills continued the old line of the Iron Mountains again. Erebor, the Lonely Mountain, was not part of either range.
After fleeing Khazad-dum Durin's folk scattered, and later gathered again in the Grey Mountains, but after six hundred years all the Dwarven strongholds had been abandoned or raided by dragons. Its sole purpose now seemed to be to divide Forodwaith from Wilderland. Although it seems that the Orcs started moving into the mountains after the Dwarves fleed, and were a large threat until after the Battle of Five Armies.
It seems according to some hints in The Hobbit, and Appendix A of the Return of the King, that some Dwarves still dwelt in the Ered Mithrin during the late Third Age. So it is possible after the War of the Ring, the Dwarves may have driven the Dragons and Orcs totally from the mountains and refounded a kingdom or colony their once again.
Other Versions of the Legendarium
Another line of "Grey Mountains" in Middle-earth are seen on the Ambarkanta map: these are a series of mountains which continue the line of the Blue Mountains as the western edge of Endor, but on the southern half of the continent. Since no maps of the entire world exist after the First Age, it is unknown if this mountain line still existed in the Third Age. In any case they do not appear in any narrative.