FACTOID # 18: Sick of crowds? Move to Greenland! Greenlanders have 38 square kilometres of land per person.
 
 Home   Encyclopedia   Statistics   Countries A-Z   Flags   Maps   Education   Forum   FAQ   About 
 
WHAT'S NEW
RECENT ARTICLES
More Recent Articles »
 

SEARCH ALL

FACTS & STATISTICS   

Search encyclopedia, statistics and forums:

 

 

(* = Graphable)

 

 


Encyclopedia > Precision (arithmetic)

The precision of a value describes the number of digits that are used to express that value. In a scientific setting this would be the total number of digits (sometimes called the significant digits) or, less commonly, the number of fractional digits or places (the number of digits following the point). This second definition is useful in financial and engineering applications where the number of digits in the fractional part has particular importance. In mathematics and computer science, a numerical digit is a symbol, e. ... Significant figures (also called significant digits and abbreviated sig figs or sig digs, respectively) is a method of expressing errors in measurements. ... In mathematics, radix point refers to the symbol used in numerical representations to separate the integral part of the number (to the left of the radix) from its fractional part (to the right of the radix). ...


In both cases, the term precision can be used to describe the position at which an inexact result will be rounded. For example, in floating-point arithmetic, a result is rounded to a given or fixed precision, which is the length of the resulting significand. In financial calculations, a number is often rounded to a given number of places (for example, to two places after the point for many world currencies). For the IEEE binary floating-point standard, see its page. ... The significand (also coefficient or, more informally, mantissa) is the part of a floating-point number that contains its significant digits. ...


As an illustration, the decimal quantity 12.345 can be expressed with various numbers of significant digits or decimal places. If insufficient precision is available then the number is rounded in some manner to fit the available precision. The following table shows the results for various total precisions and decimal places (rounding is towards zero). The decimal (base ten or occasionally denary) numeral system has ten as its base. ...

 
Precision
Rounded to
significant digits
Rounded to
decimal places
Five 12.345 12.34500
Four 12.34 12.3450
Three 12.3 12.345
Two 12 12.35
One 1E+1 12.3
Zero n/a 12

The notation 1E+1 means: 1 × 10+1.


See also


  Results from FactBites:
 
Arbitrary-precision arithmetic - Wikipedia, the free encyclopedia (575 words)
On a computer, arbitrary-precision arithmetic, also called bignum arithmetic, is a technique that allows computer programs to perform calculations on integers and rational numbers with an arbitrary number of digits of precision, limited only by the available memory of the host system.
Perhaps the earliest widespread implementation of arbitrary precision arithmetic was in Maclisp.
Arbitrary precision arithmetic is also used to compute fundamental mathematical constants such as pi to millions or more digits and to analyze their properties.
Precision (arithmetic) - Wikipedia, the free encyclopedia (229 words)
The precision of a value describes the number of digits that are used to express that value.
For example, in floating-point arithmetic, a result is rounded to a given or fixed precision, which is the length of the resulting significand.
If insufficient precision is available then the number is rounded in some manner to fit the available precision.
  More results at FactBites »

 

COMMENTARY     


Share your thoughts, questions and commentary here
Your name
Your location
Your comments
Please enter the 5-letter protection code


Lesson Plans | Student Area | Student FAQ | Reviews | Press Releases |  Feeds | Contact
The Wikipedia article included on this page is licensed under the GFDL.
Images may be subject to relevant owners' copyright.
All other elements are (c) copyright NationMaster.com 2003-5. All Rights Reserved.
Usage implies agreement with terms.