Re: Formatting numbers without precision loss

[Thijs Schreijer <thijs@...>]

> On 20 Jun 2023, at 16:14, Hugo Musso Gualandi <hgualandi@inf.puc-rio.br> wrote:
> 
> In Pallene we try from "%.6g" to "%.17g" and pick the smallest one that round-trips to the original number. (We can't use %a because our output is C, not Lua). We need special cases for +∞ and -∞. It doesn't work for NaNs. We also care whether the number ends up as a float or integer literal, so we check if %g gave it a decimal digit or exponent.
> 
> https://github.com/pallene-lang/pallene/blob/b1c0c87b749a3a9e4ebfd500a1a08af53492b8fc/src/pallene/C.lua#L56 
> 
> -- Hugo
> 


I like this approach, thx Hugo.

I wrote some code that checks if “%a” is available, and if not falls back onto something else. But as you mentioned, the generated code for “%a” is quite ugly with all the hex literals all over the place.

NaN and infinite are not really issues, since the code compiles a JSON schema, which itself is JSON, hence will not have those.

The Pallene code loops and tests the number of decimals for each number it outputs into the generated code. That bugs me a little. Then again my lib compiles a JSON schema into a lua function, so it would also be a one-off cost during generation, not in the hot-path.

In short, the options so far;

- use “%a”, but it is ugly and not always available
- determine the number of decimals for each number (Pallene approach)
- determine once the number of significant decimals for the current system, and use every time with a "%g” modifier