Skip to content

Numeric Calculations

This section explains commands and functions that you can input after performing the operation: – [Numeric Calc].

GCD, LCM

GCD determines the greatest common divisor of two values, while LCM determines the least common multiple.


Example 1: To determine the greatest common divisor of 28 and 35

  • – [Numeric Calc] > [GCD]
    28(,)35

Example 2: To determine the least common multiple of 9 and 15

  • – [Numeric Calc] > [LCM]
    9(,)15

Absolute Value

When you are performing a real number calculation, this function simply obtains the absolute value.


Example: |2 - 7| = Abs(2 - 7) = 5

(Input/Output: MathI/MathO)

  • – [Numeric Calc] > [Absolute Value]
    27

(Input/Output: LineI/LineO)

  • – [Numeric Calc] > [Absolute Value]
    27

Recurring Decimal

You can use the menu item below to input a recurring decimal.
– [Numeric Calc] > [Recurring Decimal] For details, see "Recurring Decimal Conversion (Recurring Decimal Calculations)".

Round Off

Using the Round Off function (Rnd) causes decimal fraction values of the argument to be rounded in accordance with the current Number Format setting. For example, the internal and displayed result of Rnd(10 ÷ 3) is 3.333 when the Number Format setting is Fix 3. Using the Norm 1 or Norm 2 setting cause the argument to be rounded off at the 11th digit of the mantissa part.


Example: To perform the following calculations when Fix 3 is selected for the number of display digits: 10 ÷ 3 × 3 and Rnd(10 ÷ 3) × 3

(Input/Output: MathI/DecimalO, Number Format: Fix 3)

  • 1033
  • – [Numeric Calc] > [Round Off]
    1033
Top of page