The origins of this idea seem now to be equivalent to an urban myth. Steven Hawking famously attributed it to Bertrand Russel in his book "A Brief History of Time" but there are many more references to the idea going back hundreds of years in history. The idea is relatively simple. The explanation for how the Earth stays up is that it is sitting on the back of a giant turtle. The obvious question follows "So what is holding the turtle up?" and the answer is equally obvious "It is turtles all the way down."
On the one hand it is patently ridiculous. On the other hand it is metaphorically profoundly true. If you make the mistake of taking it literally then you are simply mistaken and there are any number of ways to prove this. (Go to Australia and take a look down into the sky.) (That, of course, is a northern hemisphere arrogant joke.) But metaphorically or analogously it contains a significant truth. The truth is that so far in our experience it doesn't matter how far we go in reducing things to their constituant parts for an explanation we are always left with the requirement to go further for the next explanation.
This illustration of the recursive explanation is common in the debate about God and it's/her/his existance. The idea being that if God made the world then who made God? It is reasonable to use the illustration that it is turtles all the way down. It is reasonable to claim that that is true. What is not reasonable is to claim that it is literally true and to base further argument or justification for action on the literal idea that a turtle is holding the world up. If you then inculcate children with your idea that we are all sitting on the back of a giant turle then you could arguably be described as a little mad.