# Machine tool calibration: standards and methods

By Charles Wang, Ph.D., president, Optodyne Inc.
Edited by Bruce Vernyi, editor-in-chief

During the past decade, product innovation, improvement, and cost reduction programs have led to growing interest in machining parts with ever-tighter tolerances. Accurate parts reduce costs by cutting assembly time, maintenance and warranty costs; and improve products, leading to increased demand.

How well a part meets the specified tolerances is a function of CNC machine tool accuracy.

While the design and construction of a machine tool is determined by its ability to hold tolerances, all CNC machine tools are subject to positioning errors over time. Errors originate from natural wear and tear, crashes, and failing or worn out moving parts.

Regular calibration and compensation is supposed to identify errors and provide the tools for correcting them. However, as the accuracy of machine tools has increased, accuracy standards and calibration methods must change accordingly.

Until the past few years, machine tool accuracy was measured based on displacement, e.g. linear measurement along each of the three axes of a machine tool.

Unfortunately, linear measurements along each axis do not consider roll, pitch, and yaw angular errors, vertical and horizontal straightness errors, and squareness errors. Measurements of all these errors determine 3D volumetric position errors and ultimately provide a better gage of machine tool accuracy.

To better understand the standards and methods for measuring 3D volumetric errors, it’s important to understand the theory of Rigid Body positioning errors. Based on ASME B5.54 and ISO 230-6 standards, these are the errors that must be measured to determine 3D volumetric accuracy.

For a 3-axis machine, the theory of rigid body errors holds that there are 21 possible errors, including three each of the following types of errors: Linear displacement (errors along the direction of travel), vertical straightness (errors perpendicular to the direction of travel), horizontal straightness, angular (pitch, yaw and roll angular errors), and squareness (squareness between all 3 axes).

Current Definition of Positioning Error
For a 3-axis machine, only 3 linear displacement errors are measured, that is the distance along each axis.

The 3D volumetric positioning error is defined as the RMS (rootmean- square) of the maximum errors of each axis.

This definition has been used by most of the machine tool builders and users for more than 30 years. Back then, the largest errors were linear displacement errors, e.g. lead-screw pitch error.

Improved lead screws, linear encoders, and error compensation have minimized displacement errors.

Today, straightness, squareness, and angular errors have become the largest errors. For this reason, the current definition is inadequate.

Definitions of 3D Volumetric Errors
In the current ISO-230-2 standard, which is being revised, all the rigid body errors are considered (21 errors).

The volumetric error is defined as the maximum of the RMS of the total errors in each axis direction. Similarly, the volumetric error for angular errors is defined as the maximum of the RMS of the deviations in 3 angular directions.

The math behind the definition follows:

Assuming rigid body motion, the error E in each axis direction is Ex(x, y, z) = Dx(x) + Dx(y) + Dx(z), Ey(x, y, z) = Dy(x) + Dy(y) + Dy(z) + xy * y/Y, Ez(x, y, z) = Dz(x) + Dz(y) + Dz(z) + yz * z/Z + zx * z/Z.

The amplitude of the volumetric error, Verror, is defined as the maximum of the RMS of the deviations in 3 axes directions as the following:

Verror = Max{SQRT{Ex(x, y, z) * Ex(x, y, z) + Ey(x, y, z) * Ey(x, y, z) + Ez(x, y, z) * Ez(x, y, z)}

3D Volumetric Error Measurement with a Conventional Laser Interferometer
Using a conventional laser interferometer, linear displacement errors are fairly easy to measure.

However, to measure straightness and squareness, a conventional laser interferometer must use a Wollaston prism and special optics. It is very difficult and time consuming to setup and align.

A Wollaston prism is made of two orthogonal calcite prisms bonded together that form two right triangle prisms with perpendicular optic axes. The outgoing light beams depart from the prism as two polarized rays, with the angle of divergence determined by the prisms’ wedge angle and the wavelength of the light.

To determine the 3D volumetric error, a conventional laser interferometer must measure all 3 linear displacement errors, 6 straightness and 3 squareness errors. Typically, these measurements require several days to perform. For this reason, 3D volumetric calibration has not been widely used.

Body Diagonal Displacement Measurement
For a quick check of 3D volumetric accuracy, the body diagonal displacement errors, based on the ASME B5.54 and ISO 230-6 standard, can measure volumetric accuracy.

The measurement can be performed in a few hours, and the measured errors include the 3 displacement errors, 6 straightness errors and 3 squareness errors.

The operator mounts a laser on the machine tool bed. Mounted on the spindle, the retro-reflector, sometimes called a mirror or target, reflects a laser beam aligned along the machine diagonal.

The laser points along the body diagonal direction, and the retroreflector moves along the body diagonal at an operator-specified increment.

The displacement error measurement begins at the zero position and is taken at each increment along the three axes, which move together to reach a new position along the diagonal.

Positive or negative axis movement defines the body diagonals, four positive and four negative body diagonal measurements.

Therefore only four-body diagonal directions with forward movement and reverse movement (bi-directional); and only four-setups with measurements taken after each simultaneous X, Y, and Z move.

The accuracy of each position along the body diagonal depends on the positioning accuracy of all three axes and the machine tool’s geometrical errors.

However, there are only four sets of data and nine sets of errors, which is not enough information to pinpoint the source of the errors. In other words, the body diagonal method identifies the problem, but it does not reveal to you how to fix it.

Sequential Step Measurement of Volumetric Error
A technique introduced several years ago that has quickly gained acceptance by major airframe manufacturers — the Sequential Step measurement method — measures the volumetric errors of a 1 cubic meter working volume in two hours.

The 3D positioning errors are used to generate a positioning error map to compensate, align and adjust the machine tool.

The Sequential Step method can only take measurements using a laser calibration system, based on Laser Doppler Displacement Meter technology, such as Optodyne’s volumetric calibration system for machine tools.

This system uses two optics, including the laser and a retroreflector, eliminating bulky systems that use tripod mounted optics that are time-consuming to setup and use.

With Optodyne’s system, a modulated laser reflects a beam off of a movable/moving target.

The beam is detected and processed for information used by the controller to determine position.

The laser features a stability check of better than 0.1 PPM, accuracy of 1.0 PPM and resolution to 1 micro inch. The system includes metrology analysis software and is calibrated and traceable to NIST and supports ISO and ASME B5.54 standards.

The Sequential Step measurement method collects 12 sets of data with the same four diagonal setups as the Body Diagonal method.

Based on the measurement data, all 3 displacement errors, 6 straightness errors and 3 squareness errors may be determined.

This allows 3D volumetric positioning errors to be measured without high costs and lengthy machine tool down time. Furthermore, the measured positioning errors may be used to generate a 3D (volumetric) compensation table for correcting the positioning errors to improve positioning accuracy.

The Sequential Step method differs from the Body Diagonal method in that the movement is alternatively along the X axis, the Y axis, and Z axis. It is repeated until the opposite corner of the diagonal is reached.

The trajectory of the target is not a straight line, and the lateral movement is quite large. A conventional interferometer cannot make these measurements because it cannot tolerate a large lateral movement.

The Sequential Step method collects three times the amount of data as with the Body Diagonal method, and allows measurement of the positioning error from each separate axis movement.

As a result, the Sequential Step method identifies the errors and their source.

That provides enough information to generate a compensation table for the controller. Additionally, the method identifies issues for repair, such as worn, failing, and crashed moving parts.