☆ Yσɠƚԋσʂ ☆@lemmy.ml to Programmer Humor@lemmy.mlEnglish · 1 个月前First exposure to floating-point arithmeticpeertube.mesnumeriques.frvideomessage-square5fedilinkarrow-up118arrow-down11cross-posted to: [email protected]
arrow-up117arrow-down1videoFirst exposure to floating-point arithmeticpeertube.mesnumeriques.fr☆ Yσɠƚԋσʂ ☆@lemmy.ml to Programmer Humor@lemmy.mlEnglish · 1 个月前message-square5fedilinkcross-posted to: [email protected]
minus-squarevext01@lemmy.sdf.orglinkfedilinkarrow-up1·1 个月前I looked into this at some point in the past. From memory, despite 0.3 being unrepresenatable, the routine that formats the output rounds it (stringily) to 0.3. Someone can correct me, because it’s only a vague memory at this point.
I looked into this at some point in the past.
From memory, despite 0.3 being unrepresenatable, the routine that formats the output rounds it (stringily) to 0.3.
Someone can correct me, because it’s only a vague memory at this point.