1. Question Restatement :
Deuteronomy 23:1-8 lists some people that are forbidden to enter "the assembly of the LORD". What does “the assembly of the LORD” refer to?
Note: Other answers might indeed show how "Assembly of God" came to be understood. This answer is only suggesting what it originally meant.
2. Context and Translation :
"Assembly of God", and "Church of God" are linguistically the same exact thing - notwithstanding theological differences.
Deuteronomy 23:1-8, in Hebrew, is translated by the Greek Septuagint Deuteronomy 23:3 : קהל יהוה and ἐκκλησίαν κυρίου.
See also : other examples of "Assembly of God" in the Hebrew texts.
The context of Deuteronomy 23:1-14 contains prohibitions to keep Israel away from anything unclean, and is specifically addressing health issues:
NASB, Deuteronomy 23:9 “When you go out as an army against your enemies, you shall keep yourself from every evil thing.
Deuteronomy 23:1 “No one [whose testicles are swollen, crushed, or cut] shall enter the assembly of the Lord.
Note: See also Leviticus 21:20, Leviticus 22:24 regarding "male" health.
Deuteronomy 23:2 - Nor a Eunuch, or his descendants to the 10th generation ...
Notes:(See θλαδίας definition and also LXX Deuteronomy 23:2 defining "מַמְזֵ֖ר (Mamzer)" as a sterile person, a Eunuch). Also, see Zechariah 9:6 regarding a sterile nation. Perhaps significantly Acts 8:27, the Ethiopian Eunuch.
Deuteronomy 23:3-6 - Nor Ammonite, or Moabite, [because they are cursed].
If God was making prohibitions to perhaps safeguard Israel from peoples with disease - then it is understandable that prohibitions safeguarding Israel from cursed peoples would be in the same context.
3. Answer - The Multitude of People made Holy by God :
The "Congregation of the Lord - קהל יהוה", in Scripture, was understood as the multitude of peoples that are sanctified, (made holy), by God. It is the people where the presence of God is. It was NOT a reference to an ethnically pure Hebrew race.
Note: King David is a third generation Moabite, (Ruth 4).
"Holy", (קָדַשׁ definition) - is to separate something, from another - for one specific purpose.
NASB, Exodus 19:6 - And you shall be to Me a kingdom of priests and a holy nation.’
NASB, Numbers 16:3 - They assembled together against Moses and Aaron, and said to them, “You have gone far enough, for all the congregation are holy, every one of them, and the Lord is in their midst; so why do you exalt yourselves above the assembly of the Lord?”
Temple Period Jews Held this "Holy Nation" View :
NASB, Acts 20:28 - Therefore take heed to yourselves and to all the flock, among which the Holy Spirit has made you overseers, to shepherd the church of God, (ἐκκλησίαν τοῦ κυρίου καὶ θεοῦ) which He purchased with His own blood.
Note: This is a Passover reference, as Exodus.
NASB, 1 Peter 2:9 - But you are a chosen generation, a [kingdom of priests], a holy nation,
The Assembly of God was a Mixed Multitude :
NASB, Exodus 12:37-38 Now the sons of Israel ..., about six hundred thousand men on foot, aside from children. 38 A mixed multitude also went up with them ...
NASB, Numbers 15:15 - As for the assembly, there shall be one statute for you and for the alien who sojourns with you,
NASB, Deuteronomy 23:7 - “You shall not detest an Edomite, for he is your brother; you shall not detest an Egyptian, because you were an alien in his land. 8 The sons of the third generation who are born to them may enter the assembly of the Lord.
Conversion Ritual was Irrelevant :
NASB, Joshua 5:5 - For all the people who came out were circumcised, but all the people who were born in the wilderness along the way as they came out of Egypt had not been circumcised.
Note: Circumcision / Conversion was separate from being a part of the "Assembly of God".
4. Nehemiah 13 is not about a single ethnicity in Israel :
Nehemiah is cited in another answers in ways that affirm "ethnical purity" interpretations of Deuteronomy 23. And certainly, Rabbinicism has used texts like this to justify "Mamzer" prohibitions against inter-marriage.
But Nehemiah 13 - in Context - is a specific prohibition against people from the nations of Ashdod, Ammon, and Moab - not simply against any of the ethnicities in those nations.
NASB, Nehemiah 13:3 - So when they heard the law, they [divided (בָּדַל) all the multitude] from Israel.
Note: Nehemiah 13:3, in Hebrew, does not at all say that Israel cut-off or kicked out foreigners from Israel. (See Hebrew occurrences of "בָּדַל".) In Nehemiah 13:30, "טָהֵר | purge", is used. In this case, "בָּדַל" conveys the idea that certain Israelis were "identified".
NASB, Nehemiah 13:23 - In those days I also saw that the Jews had married women from Ashdod, Ammon and Moab.
In Nehemiah 13:3, Nehemiah refers to dividing Israel and the "Multitude", but increases specificity to "The Jews" and then eventually to "The Levites and Priests". But, why does Nehemiah inconsistently reference "The Jews" instead of "Israel" as he had before? This may have been a Euphemism for "Religious leaders" - as perhaps in Nehemiah 2:16, Interlinear, (where Nehemiah didn't report to any authority). Also seen in the Gospel of John many times. Also, Nehemiah is explicitly comparing them to King Solomon, (Nehemiah 13:26). Regardless, in the end - it was the Levites and Priests that were purified, not merely divided.
Nehemiah 13:29-30 - Remember them, O my God, because they have defiled the priesthood and the covenant of the priesthood and the Levites. 30 Thus I purified them [the priests] from [every foreigner] and appointed duties for the priests and the Levites, each in his task
Note: Nehemiah took action against the Levites and Priests. Many translations render the Hebrew - "מכל־נכר, (interlinear)", as: "everything foreign" rather than "every foreigner". But, the context is clearly about Nehemiah purifying "foreign peoples" from the priesthood, and not "foreign things".