Module 1 – AI Runtime Engineering · 1
Numerical tolerance and a reference
15-minute session: read for 3 minutes, trace for 3, experiment for 7, and complete the check for 2. Spend longer whenever you need.
Floating-point calculations can differ slightly after reordering. Compare an optimized implementation with a trusted reference using documented absolute and relative tolerances, and reject NaN when it is not expected. Never choose an enormous tolerance merely to make a test pass. Test zero, negative values, large magnitudes, and the application’s normal range.
Practice and evidence: Predict the example or draw its state transitions. Change one assumption, test the boundary, and record the actual result. Explain what this small model does not establish about a production system.
import math
actual=0.1+0.2
print(math.isclose(actual,0.3,rel_tol=1e-9,abs_tol=1e-12))
Expected result: True
Practice and evidence
Keep three notes: your prediction, the actual result, and why they match or differ. The completion check grades understanding; the practice task builds skill.
Language reference: git.agilaos.com
Complete this lesson
Answer every check correctly to earn your lesson certificate. Feedback helps you review and try again.
Sign in · Save your progress and earn a certificate for this lesson.