A question in r/MachineLearning asks what the biggest challenges are in collecting high-quality speech and egocentric video datasets. It is worth answering carefully, because the obvious answer — volume — is the wrong one.

Volume is the easy part

Raw audio and raw first-person video are cheap to capture. A phone and a head mount will generate more hours in a week than most teams can process in a month.

The constraints are elsewhere.

Consent, and its long shadow

Both modalities capture bystanders by construction. A head-mounted camera records everyone in the room. An open microphone records every voice within range. Neither of those people agreed to anything.

This produces requirements that shape the entire collection design:

• Controlled environments where everyone present has consented — which immediately biases you toward staged settings.
• Face and voice redaction, which degrades exactly the signal you wanted.
• Revocable consent, meaning your dataset can shrink after you have trained on it.

The practical consequence is that the most useful data — unstaged, in real environments, with natural interaction — is the hardest to collect legitimately.

Annotation cost scales badly

For speech, careful transcription runs many times real-time once you include speaker turns, overlapping speech, disfluencies, and non-speech events. For egocentric video, dense temporal annotation of actions and objects is worse.

And the annotations that matter most are the expensive ones. Anyone can label "person walking." Labelling the moment an action begins, or which of two overlapping speakers is being addressed, requires trained annotators and produces genuine disagreement.

The long tail is where the value is

Models trained on clean data fail on:

• Speech: heavy accents, code-switching, low-SNR rooms, overlapping speakers, children, elderly voices, speech disorders.
• Egocentric: rapid head motion, low light, occlusion by the wearer's own hands, unusual environments.

Every one of those is expensive and slow to collect deliberately, and none of them appears in sufficient quantity in opportunistic collection. So datasets grow along the axis that is cheap — more hours of the conditions you already had — while the failure modes stay unaddressed.

That is why dataset size correlates so weakly with real-world robustness in both modalities.

What actually helps

• Stratify before you collect. Decide the conditions you need and target them, rather than collecting broadly and hoping for coverage.
• Pay for the hard annotations only where they matter. Dense labels on a small, well-chosen subset beat sparse labels everywhere.
• Report the distribution, not just the hours. A dataset card that says "1,000 hours" without a condition breakdown tells you very little.
• Treat consent architecture as a design problem from the start, not as a compliance step afterwards.

The teams that do well on these modalities are usually not the ones with the most data. They are the ones who decided in advance which hundred hours would matter.