Was the world populated through incest…

Here is Ryrie’s comment on this issue from his book Basic Theology (1986 ed):

We know that Adam and Eve had other sons and daughters in addition to Abel, Cain, and Seth (Gen. 5:4), and if there was only one original family, then the first marriages had to be between brothers and sisters. Such marriages in the beginning were not harmful. Incest is dangerous because inherited mutant genes that produce deformed, sickly, or moronic children are more likely to find expression in children if those genes are carried by both parents. Certainly, Adam and Eve, coming from the creative hand of God, had no such mutant genes. Therefore, marriages between brothers and sisters, or nieces and nephews in the first and second generations following Adam and Eve would not have been dangerous.

Many, many generations later, by the time of Moses, incest was then prohibited in the Mosaic laws undoubtedly for two reasons: first, such mutations that caused deformity had accumulated to the point where such unions were genetically dangerous, and second, it was forbidden because of the licentious practices of the Egyptians and Canaanites and as a general protection against such in society.

Entire Article