Re: string.unpack segfaulting due to integer overflow and mixed sign comparisons

[Chris Emerson <chris-lua@...>]

Hi,

On Tue, Jun 26, 2012 at 03:51:43PM -0700, Sam Roberts wrote:
> Also, while (isdigit(*f)) N=10*N+(*f++)-'0'; will wrap N for large
> enough repeat counts, though this should just result in confusion if
> it wraps back into the 0 < N < len range, not segfaults.
> 
> fix:
> 
> Index: pack/lpack.c
> ===================================================================
> --- pack/lpack.c        (revision 27854)
> +++ pack/lpack.c        (working copy)
> @@ -129,7 +129,7 @@
>     case OP_STRING:
>     {
>      ++N;
> -    if (i+N>len) goto done;
> +    if (i+N < i || i+N>len) goto done;

I don't think this is right - in C overflow of signed integers is undefined
behaviour.  The C compiler can (and some now do) assume that "i+N < i"
(with N positive) can't happen, and that test can be optimised out.

I haven't looked at it in enough detail to provide the right fix, sorry.
:-)

Regards,

Chris