Here's my take--I have two adopted kids (from infancy), both difficult children. I would not adopt again, even an infant. I think the risks of a difficult child are simply too great, and I can't imagine that adopting a non infant would be any easier. While I love my children, the toll they have taken on me, my husband and my marriage are enormous. Even if I could, I am not sure I would even have a bio child at this point. I just can't face anymore. And I suspect that my difficult children will be with me for a long time--no off to college etc at age 18.
I think if you are even asking the question, you probably know the answer. It is not a question of being selfish, but rather self-aware. I think there are people out there, no doubt some on this board, who are truly up to the myriad challenges of parenting a difficult child. I admire them no end. It is truly a gift. But I am not one of them.
Most of us will rise to the occasion of being the best parent we can be to our difficult children. But most of us did not go into parenting thinking that we were going to be parenting a difficult child.
I sympathize with the desire to have another child. I wonder what it would be like to parent a more easy child child. But I am burnt out at this point.
Sounds terrible what I have written, but that's where I am. Good luck to you in making this tough decision.
If you truly love children (not just your own children), perhaps there is a way to get some satisfaction without having to adopt one of your own.