/sci/ - Science & Math

How would you formulate this question correctly? pic related is my attempt. I made the objective function the formula for the plane, then my constraints were that, when the points from the first set were inserted into the plane equation it was greater than or equal to zero and the other set were less than or equal to zero. My thinking is that when you substitute points into the plane equation, the sign of the result will be the same if the points on are on the same side, so I figured this would constrain my function enough to get any feasible solution, but all I get is 0.

halp pls