a few more edge case tests, aiming for complete coverage

Fix overflows while handling 64 bit integers.
Based on FreeBSD's expr and NetBSD's old regression test suite.

with input by and ok schwarze