Post-mortems for warehouse robot failures often blame model generalization or sensor quality. That redirects attention to architecture or hardware while leaving the training data untouched. Teams retune hyperparameters, replace a sensor, and retrain on the same dataset. Three weeks later, the failure returns under slightly changed conditions.
The pattern persists because the root cause is usually neither model nor sensor. Across early deployments we examined, the issue was training data: class definitions unlike production, gaps in particular classes or conditions, or inconsistent labels that sent conflicting signals at important boundaries.
This post maps common data-side failure modes in warehouse robot deployments and shows how each appears in practice.
Wrong Class Definitions at Deployment Time
Our most common data issue is a mismatch between the training taxonomy and the object categories the robot must distinguish in production.
Taxonomies are often set before meaningful runs in the target facility. A team may define "pallet" as one class and label examples from a test site. At a distribution center, it may find three types: standard GMA pallets, half-pallets stacked two-high, and custom wooden skids for one product line. A model trained on one generic "pallet" class cannot reliably separate them. A grasp that works on GMA pallets fails on wooden skids, appearing to be a grasping issue rather than a labeling issue.
Diagnostic check: when testing is successful but one production object subset fails, ask whether those objects form a visual category absent from the training taxonomy. Often, they do.
Revise the taxonomy and label production-specific variants. Do not start with an architecture change. The model is behaving as trained; the training simply omitted part of the production vocabulary.
Coverage Gaps in Rare but Critical Classes
Dataset size is obvious to measure. Coverage becomes visible when the model fails.
Warehouse datasets naturally overrepresent conveyors, standard pallets, and the robot body, which appear in most frames and receive dense labels. Emergency stop zones, restricted corridors, charging docks, and maintenance panels occur rarely and remain underrepresented without deliberate sampling.
A dataset with 12,000 labeled frames, including 4,800 pallet labels but only 90 emergency-stop-zone labels, will detect pallets reliably while missing stop zones at an unacceptable operational rate. It may look large, but task coverage is uneven.
Aggregate metrics can hide this failure. A properly converged loss curve and 92% validation accuracy can coexist with near-total blindness to a class present in 2% of frames. The accuracy mostly reflects the 98% without emergency stop zones.
Check per-class precision and recall, not overall accuracy alone. Any safety or task-critical class with recall below 0.75 needs more labels, whatever the aggregate score. Adequate coverage depends on class importance. For safety-critical classes, 500 instances is a minimum; 1,500 to 2,000 is where reliable performance begins for most architectures.
Label Inconsistency at Class Boundaries
One careful annotator can label inconsistently across sessions when class boundaries are unclear. Multiple annotators expose the same problem sooner and at greater volume.
At a class boundary, the model may receive contradictory signals: the same object and orientation is class A in 180 frames but class B in 140 because annotators disagreed. It learns that this visual state is ambiguous, lowering confidence precisely where robot behavior depends on the decision.
Ambiguity commonly comes from overlapping definitions. "Forklift" and "powered-industrial-truck" are separate even though a forklift is a subtype. "Robot-arm" and "end-effector" lack a precedence rule when both appear. "Person" and "worker-in-safety-gear" are split even though every facility person wears safety gear.
Correcting inconsistency after dataset growth is costly; addressing it during taxonomy design can take less than an hour. For any visually overlapping or confusable class pair, write a disambiguation rule in the annotator guide. It can be simple: "if both robot-arm and end-effector appear, label the full assembly robot-arm and omit the separate end-effector label."
Lighting Condition Gaps
Warehouse lighting varies by location and time. Dock doors admit changing daylight with clouds and sun angle. Overhead HID fixtures need 5 to 15 minutes to warm up, altering color temperature. Night-shift footage also differs from day-shift footage in ambient light.
Data gathered only during daytime business hours will underperform in early morning, evening, and night footage, though the cause may not be obvious. Objects remain detectable, but night confidence is consistently lower. More frames then enter human review during night operations, increasing oversight when the smallest on-site team is working.
Review timestamps for labeled footage. If all training data comes from a 4-hour business-hours window, the model has not seen the full day's lighting range. Sample and label early morning, evening, and at least one overnight session before deployment.
This is a condition-coverage gap, not a per-class count problem. Inspect how training data is distributed across conditions as well as object categories.
The Model Is Rarely the First Thing to Fix
Model architecture and sensor choice still matter. A suitable detector trained on clean, well-distributed data beats a poor choice, and better sensors can reduce perception difficulty. They are not always the first boundary to investigate.
The question is where to look first when performance falls short. In most warehouse deployments we have seen, upgrading architecture on flawed data brings small gains at high cost. The same engineering time spent auditing taxonomy against production, checking per-class coverage, reviewing annotator agreement, and sampling lighting conditions usually improves performance more quickly.
The dataset captures decisions made before training. A stronger model trained on flawed, production-mismatched data still inherits its baked-in failure modes. The same architecture trained on clean, representative data handles them better. Fixing data before architecture is often the quickest route to dependable behavior in the actual environment.