[squeak-dev] Rounding to n significant digits?
nicolas.cellier.aka.nice at gmail.com
Thu Aug 4 13:12:58 UTC 2011
I see two workaround if you know the increment is 0.01
1) use a ScaledDecimal rather than a Float with 2 decimal places.
2) use printShowingDecimalPlaces: 2 to render the morph
2011/8/4 karl ramberg <karlramberg at gmail.com>:
> We have a issue in Etoys where this bites us.
> We get a huge float instead of a expected zero.
> On Thu, Aug 4, 2011 at 2:20 PM, David T. Lewis <lewis at mail.msen.com> wrote:
>> On Thu, Aug 04, 2011 at 11:20:41AM +0200, Bert Freudenberg wrote:
>> > Hi,
>> > I'd like to print decimal Floats with at most n significant digits. E.g.
>> > 123456 roundToSignificantDigits: 3
>> > => 123000
>> > 0.00123456 roundToSignificantDigits: 3
>> > => 0.00123
>> > Would this generally be useful? If so, what would a good selector be?
>> > And what a good implementation? Instead of rounding I'd be happy to just get
>> > a String.
>> IMO, if it isn't considered useful, then it *should* be ;) Having grown
>> up with slide rules, I'm always annoyed by display of numbers with more
>> precision than is physically possible or computationally meaningful.
>> So +1 from me.
>> Answering a String seems good, because the typical use case would
>> be to display a number to some number of significant digits, but
>> not to change the value of the number itself.
More information about the Squeak-dev