Born in Sparta, Tennessee in White County, his father and two of his sisters played music professionally. From childhood, he learned the fiddle taught to him by Carl Alverson, Sr., of Sparta and ukulele, as well as the guitar and in his early teens left home to go to Nashville to pursue a full-time career as a country musician.
Martin was working at radio station WLAC in Nashville, Tennessee in 1948 when he was asked to replace Bill Monroe's fiddler Chubby Wise who was going to leave the Bluegra