2021
DOI: 10.1119/10.0003915
|View full text |Cite
|
Sign up to set email alerts
|

The floating point: Tales of the unexpected

Abstract: Digital computation is central to almost all scientific endeavor and has become integral to university physics education. Students collect experimental data using digital devices, process data using spreadsheets and graphical software, and develop scientific programming skills for modeling, simulation and computational work. Issues associated with the floating-point representation of numbers are rarely explored. In this article, problems of floating point are divided into three categories: significant-figure l… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2

Citation Types

0
2
0

Year Published

2021
2021
2022
2022

Publication Types

Select...
2
1

Relationship

1
2

Authors

Journals

citations
Cited by 3 publications
(2 citation statements)
references
References 21 publications
0
2
0
Order By: Relevance
“…Although it is tempting to use floating point values to represent “continuous” time, these data types can become problematic ( 91 , 232 , 233 ). Not only are such values ultimately quantized in some way ( 234 ) but the resolution of floating point data types is dependent on the magnitude of the value being represented ( 235 ).…”
Section: Improving Timing Accuracy and Precisionmentioning
confidence: 99%
See 1 more Smart Citation
“…Although it is tempting to use floating point values to represent “continuous” time, these data types can become problematic ( 91 , 232 , 233 ). Not only are such values ultimately quantized in some way ( 234 ) but the resolution of floating point data types is dependent on the magnitude of the value being represented ( 235 ).…”
Section: Improving Timing Accuracy and Precisionmentioning
confidence: 99%
“…Not only are such values ultimately quantized in some way ( 234 ) but the resolution of floating point data types is dependent on the magnitude of the value being represented ( 235 ). Additionally, both floating point and integer data types are unable to exactly represent some frequencies ( 91 , 236 ). These properties of floating point data types can create problems during numerical analysis, and may result in models that are not reproducible ( 237 ).…”
Section: Improving Timing Accuracy and Precisionmentioning
confidence: 99%