Joe Jordan first moved to St. Louis to study at the Lincoln Institute (now Lincoln University). While in St. Louis he took private music lessons, played piano-rag music in small cafés, and sang with small ensembles. From St. Louis, Jordan moved to Chicago for a year before he went to New York to make show and syncopated dance music⁷¹. In New York Joe Jordan and James Reese Europe composed and directed the Memphis Students Orchestra, which was an immense success, performing at New York's most prestigious venues. James Weldon Johnson described the Memphis Students as "the first modern jazz band ever heard on the New York stage."⁷²

Jordan returned to Chicago in 1906 to become director of music at the historic Pekin Theater. The Pekin Theater was run by Robert T. Motts. Before Jordan assumed the directorship Motts had agreed to a deal with Ida B. Wells-Barnett who wanted to use the newly renovated theater to host a benefit for her Frederick Douglas Center. Motts granted her request under the agreement that she would guarantee greater attendance at the Pekin's shows. The new Pekin Theater was home to a spectrum of musical genius from Will Marion Cook to Samuel Coleridge Taylor. In 1909, Jordan left the Pekin for a short time to work on the Cole and Johnson production of The Red Moon; he returned in 1911.

Jordan was a consummate composer of rags, show music, dance music, and concert music. In 1910 he composed the tune that made Fanny Brice a star, Lovie Joe. Brice performed the song in a ridiculous spectacle of blackface. Jordan, who was forced to stand outside of the theater because blacks were not allowed in, wept when he heard it⁷³. His most important works include Nappy Lee—A Slow Drag (1903), Pekin Rag (1904), J.J.J. Rag (1905), and That Teasin' Rag (1909). At age 46 he conducted the musical Keep Shufflin. He composed for a dozen black choruses, directed Will Marion Cook's syncopated orchestra, and directed WPA orchestras.