I know I can assume -(P -> -Q) and use RAA to assume -(P&Q), but from there I'm stuck on how to proceed because of the negations on the outside of the parentheses.
That's a good start. Just continue on in the same vein.
|_ ¬(P → ¬Q) Premise
| |_ ¬(P & Q) Assumption
: : :
| | # ¬ Elimination
| ¬¬(P & Q) ¬ Introduction }
| P & Q ¬¬ Elimination }= Reduction to Absurdity
Your target is to derive a contradiction from assuming ¬(P → ¬Q) and ¬(P & Q). Well, those are the only things you have to contradict so obviously you need to derive either P → ¬Q or P & Q (maybe both, along the line).
Clearly you might derive P → ¬Q by a conditional proof: where you further assume P aiming to derive ¬Q, if possible.
|_ ¬(P → ¬Q) Premise
| |_ ¬(P & Q) Assumption
| | |_ P Assumption
: : : :
| | | ¬Q ¬ Introduction
| | P → ¬Q → Introduction
| | # ¬ Elimination
| ¬¬(P & Q) ¬ Introduction }
| P & Q ¬¬ Elimination }= Reduction to Absurdity
Well, now, how could you prove that negation of Q? Of course, by further assuming Q aiming to derive the other contradiction P & Q, ie a proof of negation !
|_ ¬(P → ¬Q) Premise
| |_ ¬(P & Q) Assumption
| | |_ P Assumption
| | | |_ Q Assumption
| | | | P & Q & Introduction
| | | | # ¬ Elimination
| | | ¬Q ¬ Introduction
| | P → ¬Q → Introduction
| | # ¬ Elimination
| ¬¬(P & Q) ¬ Introduction }
| P & Q ¬¬ Elimination }= Reduction to Absurdity
Done.