I think you should not adopt him. You must consider your priorities carefully and make your decision based on the welfare of everyone for whom you are responsible. You have four other children (biological or adoptive is irrelevant) who are relying on you to provide a safe, loving, appropriate home for many years to come.
You will not be able to provide that if you bring this child into your family permanently. It would not be right for you to keep him knowing that his welfare would come before the welfare of your other children AND that he might (probably will) endanger them and possibly hurt them and you in the future. He's only 6 now. You need to spend time thinking about how it will be when he is 16, bigger and stronger than you and possibly your husband. Adolescence is a time of upheaval and stress. I can only imagine what his adolescence is going to throw up for him to cope with given the horror of his earliest years and memories. You are playing with fire to believe that you can help him without placing everyone in harm's way.
It is very sad and you may be right that his future will be really bad. But it is also possible that a better placement is waiting for him.
I hesitate to address the issue of his sister. In my head I hear her asking you when she's about 6 - why didn't you adopt my brother too? And there will never be an easy answer to that question. If her brother gets a healing placement then it may be possible for them to reunite in the future.
I think that if it's an option for you to keep her and give up her brother then you should consider that option carefully. Remember that she is very young and she may grow into disabilities and problem behaviors/mental illness due to her genetic heritage and any drug or alcohol exposures she had in utero or as an infant. I would go so far as to say that you should EXPECT her to develop problems and you need to factor that into your decisions. Yes, having a loving family raise her will help but honestly, it will only go so far to protect her from her genetic programming and brain damage from drug/alcohol exposure.
I've excerpted a small part of the article found at this link. The article discusses very recent findings that demonstrate the very high level of heritability of schizophrenia and bipolar disorder. If you don't know what the mother's diagnosis was, I think you should assume the worst case = in which case it is either schizophrenia or bipolar disorder.
http://www.medscape.com/viewarticle/587148
Overall heritability for schizophrenia and bipolar disorder was 64% and 59%, respectively. The comorbidity between disorders was 63% and mainly due to additive genetic effects common to both disorders.
Shared environmental effects were small, but substantial 4.5% for schizophrenia, 4.4% to 7% for bipolar disorder, and 2.3% to 6.2% for both disorders.
According to Dr. Hultman, these results demonstrate that members of families in which someone has either of these affective disorders run an increased risk of developing the same condition and that this is chiefly the result of genetic factors, with only a slight influence of shared environmental factors.
If they are a package deal then I think you will have to walk away - as hard as that will be.
You have given them both a better chance than they would have had without you. Hug that to you tightly when you send them on to the next loving person who will hold them close and try to help them heal.