That's a really interesting idea. On the other hand it does add quite a lot of complexity to something that is quite simple. And it raises some awkward questions like how do you represent NaN for unsigned integers and if you don't then you have the awkward fact that there's one more unsigned than signed number.

I can imagine an alternative timeline where it worked like that though.