The continued presence of Russian forces in Azerbaijan became problematic when Russian troops were alleged to have assisted Armenians in an attack that killed hundreds of civilians in the town of Khodzhaly, in southwestern Azerbaijan, in February 1992. In the face of widespread demands from the political opposition in Baku, components of a 62,000-member Russian force began to withdraw from Azerbaijan almost immediately. Striking a contrast to the protracted withdrawal of Russian troops from the Baltic states, the last Russian unit, the 104th Airborne Division, withdrew from Azerbaijan in May 1993, about a year ahead of the schedule that the two countries had set in 1992.

According to an agreement between Russia and the Transcaucasian states calling for distribution of former Soviet military assets among the participating parties, Azerbaijan would receive most of the matériel of the 4th Army that had been stationed there, together with part of the Caspian Flotilla. The Russians destroyed or removed much of their weaponry upon withdrawing, but a substantial amount was stolen, exchanged, or handed over to Azerbaijani forces. Some Russians answered appeals from Azerbaijani military leaders to serve in the Azerbaijani armed forces. By agreement with Russia, many former members of the Soviet Border Guards also continued their duties under Azerbaijani jurisdiction, with Russian assistance in training and weapons supplies. In January 1994, Russia and Azerbaijan discussed possible use of Russian forces to bolster Azerbaijan's border defenses.

Data as of March 1994