Galatians is directed to Gentiles who were being persuaded to become circumcised and adopt the Mosaic law (Torah).
The lapse which Paul mentions in the verse following does not mean that they had followed the law previously. Rather, he has an overarching salvation-historical shift in mind. The world prior to (and outside of) Christ is made up of "elements" (Greek stoicheia), just as in the classical philosophical imagination the world was literally made up of earth, wind, water, and fire. For Paul, Torah belonged to that old creation and was one of its constitutive "elements."
Thus, when these Gentiles began to be convinced of circumcision and practicing Torah, Paul's claim is that they were returning to the same aeon (age; see Gal 1:4) from which they had been rescued by the gospel. They were being "unbirthed" from the new creation, so that Paul was labouring in birth for them again (4:19).
The reference to slavery must be understood within the broader context. In 3:23ff Paul has described Torah as a paidagogos, which was a household slave who served as a child custodian. If the child is subject to the governance of a slave, that implies (says Paul in 4:1–2) that his own position differs nothing from a slave. The advent of Christ is the coming of the mature Heir, who is born of a woman (and thus under the conditions of the kosmos as a whole) and born under Torah (and thus subject to it). His death provides "redemption" (liberation, like that of the slaves from Egypt) from Torah for Jews; it also triggers the sending of the Spirit promised to all flesh (i.e. including Gentiles).
Seen in this overall light, the before- and outside-Christ world is a realm of slavery, while the "in Christ" new creation is a realm of inheritance and freedom. By adopting circumcision and the calendar of Torah, the Galatians were regressing from the new creation and becoming subject again to the elements of the old kosmos.
