Real vs. Synthetic Training Data: A Cost and Time Breakdown for Small CV Projects

Ask anyone who has shipped a small computer vision model where the time went, and the answer is rarely "training." It went into collecting images, arguing about labels, relabeling, and then discovering that the camera in production sits two meters higher than the one used for the dataset. For a team of one to three people, data work routinely eats two thirds of the project calendar before a single epoch runs.

That is why the real-versus-synthetic question comes up early now, even on modest projects. Rendering images from 3D scenes with labels generated automatically used to be a research-lab tool. Today computer vision synthetic data is a service you can order, or something you can produce yourself with Blender and a few hundred lines of Python. The question is not whether it works, but when it is cheaper. Below is a stage-by-stage breakdown using a concrete example project, with the assumptions written out so you can swap in your own numbers.

The example project

To keep the comparison honest, assume one project throughout: detecting five types of small hardware parts (screws, washers, nuts, clips, springs) on a conveyor belt, using a fixed overhead camera. Target: a YOLO-style detector with usable precision on a held-out real test set. Team: one developer, part-time help for labeling. Target dataset size: 5,000 labeled images.

Every cost below is an estimate built from stated assumptions, not a market survey. Your rates will differ; the structure of the comparison will not.

Stage 1: Collection

Real data. Someone has to put parts on the belt, in varied arrangements, under varied lighting, and capture frames. Assume 200 usable frames per hour once the rig is set up, plus a day to build the rig. That is 25 hours of capture plus 8 hours of setup for 5,000 images. Then subtract the frames that are blurry, duplicated, or show the operator's hand.

Synthetic data. Collection is replaced by scene setup: modeling or sourcing five part meshes, a belt surface, and a camera matching the real intrinsics. Assume 16 to 24 hours for someone comfortable in a 3D tool, or a few days of turnaround if you outsource it. After that, generating 5,000 frames is a render job that runs while you sleep.

Stage 2: Labeling

This is where the gap opens.

Real data. Drawing bounding boxes on 5,000 images with an average of eight parts per frame is 40,000 boxes. At a realistic pace of 150 to 200 boxes per hour with a decent tool, that is 200 to 270 hours of labeling. Even at a low outsourced rate, this is the single largest line item in the project. Add a review pass, because a labeling vendor's first delivery is never clean.

Synthetic data. Labels are a by-product of rendering. The renderer knows exactly where each mesh is, so bounding boxes, segmentation masks, and depth maps come out pixel-accurate with zero human hours. There is no inter-annotator disagreement because there is no annotator. The cost here is verifying that the export format (COCO, YOLO, KITTI) matches what your training script expects, which is an hour of work, once.

Stage 3: Coverage of rare cases

Real data. Your detector needs to handle a bent clip, a rusted washer, two screws overlapping, and a shadow from a passing forklift. In 5,000 real frames captured over two weeks, you might get a handful of each. Some you will never see until production.

Synthetic data. Rare cases are parameters. Set occlusion probability to 30 percent, add a rust material to 5 percent of washers, randomize light direction and intensity per frame. This is domain randomization, and it is the main technical reason synthetic data helps generalization rather than hurting it.

Stage 4: Iteration

Every real project changes requirements at least once. A sixth part type gets added. The camera is moved. The belt is repainted a lighter color.

Real data. Each change means a partial or full repeat of Stages 1 and 2. A new part type alone means capturing and labeling a few hundred new images, plus mixing them into the existing set without skewing class balance.

Synthetic data. Add the mesh, adjust the scenario config, re-render overnight. The labels regenerate with it.

The part synthetic data does not remove

The sim-to-real gap is real. A model trained only on renders will underperform on real frames until you close that gap. Two things work reliably:

• Fine-tune on a small real set. A few hundred labeled real images, mixed in or used as a final training stage, typically recover most of the loss.
• Validate only on real data. Your test set must be real frames from the real camera, labeled by a human. Never score a synthetic-trained model on synthetic images and call it done.

So the honest synthetic budget still includes some real collection and labeling. For the example project, assume 300 real images, roughly 15 hours of capture and labeling combined. It is a fraction of the all-real path, not zero.

Totals for the example project

The numbers shift with your rates and tooling, but the ratio is the finding. On a project like this, labeling dominates, and labeling is exactly what synthetic data removes.

When real data still wins

Synthetic is not the default answer for everything. Stick with real capture when the subject is hard to model in 3D (textiles, food, organic materials), when the scene is unconstrained and you cannot enumerate what varies, when you already have a labeled dataset that works, or when the project is so small that 300 images is the whole dataset anyway. For a fixed camera looking at manufactured objects, though, the math is not close.

Comments

Loading comments…