One early-access pilot had these labeled instance counts: conveyor belt section (4,312), gripper (3,847), object-A (2,103), object-B (1,890), pallet (1,204), and emergency stop zone (89). Total annotated instances: 13,445. The team called it "roughly 13,000 labeled instances" and felt ready to train.
Five of six classes had enough coverage. The emergency stop zone, a safety-critical requirement in the deployment specification, had about 5 percent of the minimum count we would suggest. Early tests showed unreliable e-stop detection. The architecture was not the issue. The aggregate total had hidden a class coverage gap. We learned the hard way that a large dataset total can create false confidence.
How Class Imbalance Starts in Factory Data
Factory robotics datasets usually become imbalanced because annotators label what appears most often. Conveyor belts fill production-line frames, and grippers show up nearly as frequently. Safety markers, emergency stop buttons, and error indicators appear at particular locations or only during non-nominal events.
Footage from a running line naturally adds thousands of dominant objects. Rare but important classes grow slowly. The difference between the dataset total and each class total can widen even when every individual annotation decision seems reasonable.
Targeted collection creates another bias. Teams often record normal operation to build labeled volume, but normal runs contain few failures, safety events, or unusual object states. Ten hours of successful picks, carefully labeled, can train excellent successful-pick detection while leaving the rest unreliable.
Why the Total Count Can Mislead
Teams commonly report total instances, or sometimes total frames. Neither metric reveals coverage well. A dataset with 15,000 instances drawn from only three of eight classes is weak. A dataset with 4,000 instances, distributed as 500 per class across eight classes, gives training a better starting point.
The useful measure is the minimum count for each class. That floor depends on visual complexity, appearance variation in the environment, and the precision required at deployment. A simple, stable object such as a solid-color bin in fixed lighting may need 300 to 500 labeled instances. A gripper with major orientation changes may need 1,000 to 2,000 examples spanning its operating range.
Safety-critical classes need their own threshold. A robot detecting pallets at 93% accuracy and emergency stop zones at 78% has a strong pallet detector but only a borderline e-stop detector. Missing those two classes has different consequences, so coverage targets should not be identical.
Measure Coverage Before Training
Start with a per-class instance histogram, sorted by count and marked with coverage thresholds. It exposes the gap quickly. If this report does not exist before a training run, the team is guessing about a common robotics dataset failure mode.
Audit three more dimensions beyond count. Spatial coverage asks whether each class appears across every usable scene position. A gripper with 2,000 labels from only the upper frame may fail in the lower half. Appearance coverage asks whether the object states are varied. A flexible pouch with 1,500 upright examples may fail when pouches tip on the belt. Temporal coverage checks recording sessions, lighting, and time of day instead of accepting one afternoon as representative.
Vision Lab's class coverage report shows these dimensions beside each class count. It runs on the labeled dataset before export, giving teams a chance to find gaps and address them before a training run.
Closing Coverage Gaps
After finding thin classes, teams have three practical routes.
First, collect targeted footage. Return to the environment and record the under-represented class on purpose. For safety markers, capture shift-start and shift-end procedures when workers approach them. For failed grasps, deliberately induce failures. This usually gives the strongest added labels, but it requires site access and a careful collection protocol.
Second, use synthetic augmentation: copy-paste, geometric changes, or appearance variation applied to existing labels. For simple objects, this can roughly double or triple effective instance count and close a moderate gap. It cannot represent genuinely unseen states. Copy-pasting 80 e-stop examples into random frame positions adds correctly labeled training examples, but not the variation targeted recording would reveal.
Third, use active learning. Train on the current set, run the model on unlabeled footage, and surface uncertain frames from thin classes. Labeling those frames is more directed than random sampling because it aims at the missing coverage. Vision Lab supports uncertainty-based queue generation in its export workflow.
Not All Class Imbalance Is a Problem
Severe instance imbalance is not automatically wrong. Background, meaning no foreground object, naturally dominates detection datasets and does not need to match foreground classes. Common training methods address it with hard-negative mining or focal loss.
A dataset can also reflect a real environment where conveyor belts appear often and e-stop zones rarely. That ratio is not inherently defective. The problem comes when the rare class is also safety-critical or otherwise high stakes. Natural recording frequency is then a poor training target, so the rare class should be intentionally over-sampled even when that changes its observed ratio.
The key distinction is whether a class is rare because it matters less, or because it seldom appears in recordings despite its importance when present. The first may be acceptable. The second is the coverage gap to close before deployment.