Signed areas came up in connection with winding numbers even before Gauss, see Sunada, From Euclid to Riemann and Beyond:
"Gauss briefly noticed in the letter to Bessel in 1811 , the winding number reveals the nature of the complex logarithm. Winding numbers appeared in the study of signed areas by Albrecht Ludwig Friedrich Meister (1724-1788)."
Meister's Generalia de genesifigurarum planarum, et inde pendentibus earum affectionibus (Novi Commentarii Soc. Reg. Scient. Gott., 1 (1769/1770), 144-180) had very few equations, but derived what is now called "shoelace formula". Gauss used the idea of winding numbers in his proofs of FTA (1799), see Eisermann, Fundamental Theorem of Algebra Made Effective.
Cauchy came to signed areas, and volumes, around 1812-13 from a different perspective, not in connection with winding numbers and contour integrals. It was the usual byproduct of algebraic treatment of geometric objects - algebra naturally uniformizes geometrically distinct cases in theorems. Chasles inserted signed segments into the cross-ratio for similar reasons a bit later (1830). Here is Boyer, History of Analytic Geometry:
In 1812-1813, for example, J.P.M. Binet (1786-1856)
wrote a long and tedious paper in which he made use of analogous notations (which he called "resultants") in analytic theorems on volumes, areas, and lengths of rectangular configurations in three dimensions. His work includes the equivalent of the multiplication of determinants; but in this early use of determinants the familiar square array was missing. This element, together with the double-subscript notation, was supplied by Cauchy at the very time of Binet's paper. Cauchy applied his "resultants" in attributing signed values to angles, areas,
and volumes in coordinate geometry."