Because the negative values are required so that X*inv(X) = I.
Since your example is a 2x2 matrix, there is even a formula for the inverse. for any numbers a,b,c, and d, define the matrix
A = {a b, c d};
Then
inv(A) = {d -b, -c a} / (ad-bc);
So you can see that if all the original elements are positive, then exactly two of the elements in the inverse must be negative.