Re: Reporting possible BUG on string.gsub

[Peter Cawley <lua@...>]

In string.gsub, % acts as a magic character in both the pattern and
the replacement. The behavoiur of % in patterns is well documented,
the behavoiur of it in the replacement is not documented quite so
fully. The documentation states "The character % works as an escape
character: any sequence in repl of the form %n, with n between 1 and
9, stands for the value of the n-th captured substring (see below).
The sequence %0 stands for the whole match. The sequence %% stands for
a single %". The slightly extended description would be:
%0 -> the whole match
%[1-9] -> the n-th capture
%[other character] -> the character itself (i.e. %X becomes X, %%
becomes %, etc.)

Furthermore, Lua stores all strings with a NULL byte after the last
character (this is an implementation detail, not a language
requirement), causing a % at the end of the replacement to be
interpreted as "%\0" and hence insert a "\0" character into the
output. The printing to stdout which is performed by the standard
print function may interpret embedded NULL characters as the end of
the string. Your example can be modified to show it though:
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> str = [[This is a string with @@ANYTHING@@ chance of survival]];
> replace = "5%"
> print(("%q"):format(str))
"This is a string with @@ANYTHING@@ chance of survival"
> print(("%q"):format(replace))
"5%"
> str = str:gsub("@@ANYTHING@@", replace)
> print(("%q"):format(str))
"This is a string with 5\000 chance of survival"
>

There are three possible issues here;
1) Not quite complete documentation of '%' in string.gsub replacements
2) string.gsub reading beyond the end of the replacement string
(although this is never a buffer overflow as there is always a NULL
just beyond the end of strings in the current implementation)
3) print(s) to stdout stopping at any embedded NULL characters

On Fri, Jan 23, 2009 at 1:24 PM, Daniel van Ham Colchete
<daniel.colchete@gmail.com> wrote:
> Hello yall,
>
> please take a look at the following code:
>
> ----BEGIN---
> str = [[This is a string with @@ANYTHING@@ chance of survival]];
> replace = "5%"
> print(str)
> print(replace)
> str = str:gsub("@@ANYTHING@@", replace)
> print(str)
> -----END----
>
> The result I have here is
> ----BEGIN---
> This is a string with @@ANYTHING@@ chance of survival
> 5%
> This is a string with 5
> -----END----
>
> But the last sentence should be "This is a string with 5% chance of
> survival". The problem doesn't happen if the replace string doesn't end with
> "%". In cases where there is an "%" in the middle (like '5%a') I still have
> a result different from what I would expect.
>
> I tested it on Lua 5.1.3 and Lua 5.1.4 on Linux. I also tested changing the
> line replace = "%5" for replace =[[5%]].
>
> GCC version: gcc version 4.1.2 (Gentoo 4.1.2 p1.0.2)
> GNU LibC version: 2.6.1
> Kernel version: 2.6.26-gentoo-r1
>
> Do I get to have my name on http://www.lua.org/bugs.html??
>
> Best regards,
> Daniel
>