Humans and neanderthals interbreeding

Recently the report of possible interbreeding between humans and neanderthals has been a hot topic. As I understand it, a research team has compared their draft sequence of the neanderthal nuclear genome with the genomes of five modern (contemporary) humans. They found that there was more sequence similarity between neanderthals and three of the humans than with neanderthals and the other two. They concluded that this meant there was interbreeding.

This raised a host of questions, such as, are we all part neanderthal? If so, then is the image of God tarnished? This fits nicely into an old-earth creation model because (1) it shows that even though there may have been a small amount of interbreeding, humans and neanderthals are distinct species, thereby refuting the young-earth creationist model (that neanderthals were descendants of Adam and Eve), and (2) it shows humans did not evolve from neanderthals, further eroding the multi-regional hypothesis of human evolution (that ancient hominid species such as Homo erectus and neanderthals evolved together into the different races of modern humanity).

In fact, I have heard speculation that the humans and neanderthals interbreeding may be one way in which God could have used as a form of imposing racial diversity at the time of the Tower of Babel.

But there's one hypothesis that I haven't heard talked about: What if the interbreeding was only one-way? What if sinful human males had offspring with neanderthal females, but not the other way around? It would seem to fit the data just as well as the alternative, and perhaps even better. This scenario would definitively explain, for example, why there is no mitochondrial DNA evidence of humans and neanderthals interbreeding.

It also makes "social" sense. What I mean is, if we are all part neanderthal, that means at some time in the past, a half-human : half- neanderthal must have been raised by a human woman/family, and must have integrated into society somehow, and then passed on its genes via
a human partner. Then a 3/4-human : 1/4-neanderthal must have done the same thing, etc. Of course, as the neanderthal part gets diluted out, it becomes more imaginable. But even if a neanderthal male has offspring with a human female, and the mother does not want to "give
up" her baby, and raises it like a fully human child, what are the chances that the child would find a willing human partner to pass on the part-neanderthal genes? Just speculating here, of course. (In this day and age of political correctness, I almost feel like I'm being racist, but my arguments are actually speciesist...)