This hand crafting was necessary because, even though machines could be employed to make various parts, a craftsman was needed to make them all fit together. In other words even though parts looked identical they were not interchangeable and had to be individually fitted to work together.
It took companies like Singer and others about 100 years to produce parts that would all work together even though it was claimed and generally understood that the country had entered the era of interchangeable parts.
The magic was tolerance. Individual parts can vary a certain amount before they will not work together. The maximum amount of variation in an attribute is called tolerance. Making parts more precise than allowed by the tolerance is not necessary and actually more expensive.
Gauges, Jigs and fixtures were designed that made it possible to manufacture parts that would lie within their prescribed tolerance. This made it possible for a machine to be built with parts manufactured anywhere in the world.
Tolerance is always some positive or negative variation from a particular value say 1.25 inches plus or minus .003 inches. If a part is 4 thousandths of an inch larger or smaller it will not interoperate with other parts that must be mated to it. Anything less than 3 thousandths of an inch larger or smaller will work. Plus .00001 of an inch will also work but is not necessary and the machinery required to produce this precision may be cost prohibitive.