On footage from 3 factory floors, a standard YOLOv8 model pretrained on COCO reached about 27 to 31% mAP on our internal benchmark. The same model's published COCO result is above 50. That shortfall reflects a domain gap, not simply weak modeling, and it catches many small robotics teams when they start an industrial dataset.
This gap follows recurring visual differences between factory scenes and the natural images used for pretraining. Once those differences are separated, annotation choices can address them directly instead of adding labels without a sampling plan.
Factory Lighting Changes Appearance
ImageNet and COCO are dominated by daylight and softly lit indoor scenes. Light is often directional but diffuse, shadows are soft, and illumination changes gradually. Surfaces therefore tend to appear close to what people expect from ordinary viewing.
Factory fixtures create a different pattern. Fluorescent and LED tubes cast hard light downward, while metal frames, robot joints, and parts generate sharp reflections that shift with angle and position. Coverage is uneven, with bright areas below fixtures and darker gaps between them. A part can pass through several lighting conditions during one conveyor run.
A gripper may look different from frame to frame as it rotates, the light angle changes, or nearby metal reflects onto it. A model exposed mainly to natural scenes has little basis for this within-class variation, so detection confidence can swing even when the object has not changed.
Fixed Camera Viewpoints
Natural-scene video and many ImageNet-derived benchmarks include camera motion. Viewpoint changes are part of the challenge, and current detectors are generally built to tolerate moderate shifts in viewpoint.
Factory cameras are usually mounted in place. That can simplify detection, but every label then reinforces one view. Move the camera slightly and performance may fall. Fixed mounts also produce unusual appearance statistics: parts are frequently seen straight down or from one side, views that natural-scene training data contains far less often.
Use the fixed view as a sampling prior. You know where objects tend to enter the frame and how they move, which can guide annotation automation. It also creates deployment bias. If the system will always use that exact camera, broader viewpoint coverage may be unnecessary. If cameras or sites will change, the same dataset becomes fragile.
Class Vocabulary Mismatch
COCO has 80 classes, but none is an industrial robotic gripper, conveyor belt section, pallet, emergency stop zone, or robot arm joint. Its nearby labels, such as person, sports equipment, and vehicles, are a poor semantic fit for ordinary factory footage.
That mismatch affects transfer learning. COCO's early layers capture edges, textures, and simple shapes that can carry into a new domain. Its later layers combine those features around COCO's 80 categories. Fine-tuning must reorganize them around factory objects, which takes more labeled data than adapting within a related domain. A clear taxonomy is necessary from the start.
COCO pretraining is still useful. Its early representations can shorten training compared with random initialization. But the vocabulary gap means a COCO model will not reliably learn factory classes from a small label set. Plan factory-specific annotation at the outset.
Temporal Structure in Factory Video
Image datasets usually treat frames as independent, and many natural-scene video sets sample far enough apart to reduce frame-to-frame similarity. Production footage behaves differently.
A single class may remain visible for thousands of consecutive frames while moving only slightly. That high temporal correlation creates 2 annotation problems. Naive sampling fills the set with near duplicates, increasing cost without much new information. A test split drawn from the same time stream can also inflate metrics, because the model may memorize the layout instead of learning a detector that transfers.
Sample by operating state: SKU, lighting condition, time of day, and conveyor position. A useful factory dataset is not simply more footage hours. It is footage that represents the ways the environment actually changes.
What This Means for Annotation Strategy
Lighting variation, fixed-view bias, class mismatch, and temporal correlation point to one approach: treat factory annotation as domain-specific dataset engineering, not a race to increase label count.
Ask whether labels cover operating states, lighting, and object appearance, not only how many frames are marked. A 5,000-frame set spanning 6 lighting conditions and 4 SKU types should train better than 20,000 frames from one afternoon under one lighting setup.
Taxonomy design matters as much as volume. Factory objects often have no COCO counterpart, so class definitions must stay consistent across the variation in your footage. Splitting nearly identical views into "gripper" and "gripper with part," for example, can create label noise that more training frames will not repair.
Teams we work with sometimes gather a large video archive and label it quickly, while postponing taxonomy design and coverage analysis. The resulting dataset looks substantial but leaves predictable blind spots in conditions unlike the dominant recording session.
Pretraining Still Matters
We are not arguing for training factory models from scratch. COCO pretraining provides a useful start for low-level features and usually reduces labeled-data needs. For most projects, fine-tuning a pretrained backbone with factory labels remains a sensible baseline.
The domain gap is too large for factory annotation to be treated as a small fine-tuning task. Build coverage around real operating conditions, not natural-scene benchmark assumptions. One exception is a fixed, single-camera deployment with no planned viewpoint change, where extra diversity may add cost without helping. Check sampled clips before annotation.