AI for Indian languagesMiddleLesson 2416 min read

Collecting language data ethically

Language data comes from people. Learn how to collect text and speech with real consent, fair pay, privacy and fair representation, and how to give back.

✦ Ask the tutor🧪 Lab · 40 min📝 Lesson test
Lesson in motion

In 60 seconds

Collecting language data ethically

Language data comes from people. Learn how to collect text and speech with real consent, fair pay, privacy and fair representation, and how to give back.

1/5
In simple words
If you want to learn a grandmother's recipe, you ask her, you thank her, and you do not sell it under your own name. Recording people's words and voices for AI is the same. You ask first, you pay or thank them fairly, and you respect what they agreed to.
Every sentence and every voice clip in a dataset came from a person. For many Indian languages, especially smaller ones, the data comes from a small community who trust you with their words, stories and voices. Doing this well is both the right thing and the way to build better data.

Six principles

PrincipleWhat it means in practice
Informed consentExplain in the person's own language, in simple words, what you collect, why, who will use it, for how long, and whether it will be public. Let them ask questions. Record their agreement.
Freedom to say noNo pressure from employers, teachers or village leaders. Saying no, or stopping halfway, has no penalty.
Fair payPay fairly for people's time and skill, on time, in a way that works for them. Transcribers and reviewers are skilled workers.
PrivacyCollect only what you need. Remove names, phone numbers, addresses and ID numbers. Store data securely and limit who can see it.
RepresentationInclude different regions, dialects, genders, ages and backgrounds, and say who is missing.
Community benefitShare results back, credit contributors, and where possible release data and tools the community can use.

A consent process that works

  1. 1

    Plan with the community

    Talk to local groups, teachers or language organisations before you start. Ask what they want out of it.
  2. 2

    Explain clearly

    Use the local language, speak it aloud as well as writing it, and use examples. Give people time.
  3. 3

    Ask for each use

    Separate choices: use for research, use in a product, public release, voice cloning. People may say yes to some and no to others.
  4. 4

    Record and store consent

    Keep a record of who agreed to what and when, linked to their data by an ID, not by their name in the file.
  5. 5

    Honour withdrawal

    Give a simple way to withdraw, and a real process to remove their data from future versions.
Watch out
Some things need extra care: recordings of children, health information, caste, religion, political views and other sensitive details, and communities that have been harmed by research before. India's Digital Personal Data Protection Act and its rules apply to personal data, and rules for children's data are stricter. For a real project, get advice from a legal expert and, for research, an ethics committee.

Dialects and fairness

If you collect only from cities, from one region, or only from people who read and write easily, your model will work best for them and worse for everyone else. Plan who you collect from, write it down, and report it honestly in a dataset card.
Extracting data
  • Scraping community content without asking
  • Paying volunteers nothing because "it is for a good cause"
  • One consent form in English, signed in a hurry
  • Keeping everything, forever, just in case
Building with a community
  • Asking first and agreeing the purpose together
  • Paying fairly and on time, and crediting contributors
  • Consent explained in the local language, with separate choices
  • Collecting only what is needed and deleting what is not

Giving back

  • Share a summary of what you built, in the community's language.
  • Credit contributors and partner organisations, if they want to be credited.
  • Release data or models under an open licence where contributors agreed to it.
  • Make sure the product actually serves the people whose language it learned from.
Note
Open data and privacy can pull in different directions. A public speech dataset is useful to many researchers, but voices can identify people and can be misused for cloning. Decide the release level together with contributors, and give them a real choice.
Try this. Think of a language project you would like to do. Who would you collect from? Who might be left out? Who in the community would you talk to first?

Watch and read more

Lab

A one-page consent form and a short collection plan for a small speech or text project.

~40 min

The problem

Imagine you will collect 100 short voice recordings of everyday sentences in one Indian language, to test speech recognition for a farmers' helpline.

  1. Write a one-page consent form in simple English (it would be translated and read aloud). Include: who you are, why you collect, what you collect, how it is stored, who can use it, how long you keep it, payment, and how to withdraw.
  2. Add separate tick boxes for: use in testing, use in training, public release, and voice cloning (you may decide not to offer this one; say why).
  3. Write a short plan: which districts, dialects, genders and age groups; how you will reach them; how you will pay.
  4. List three risks to contributors and how you reduce each one.

Do not collect any real data for this lab. It is a design exercise.

You are done when

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1A funder wants your speech dataset released publicly, but several contributors did not agree to public release. What do you do?Reveal

Respect the consent. Release only recordings from people who agreed to public release, and keep the others under the terms they agreed to. Explain this to the funder, and plan consent choices before collection in future. Changing the terms afterwards without asking again would break trust and may break the law.

Q2Why can paying contributors much more than local wages also be a problem?Reveal

Very high pay can make it hard for people to say no or to withdraw, which weakens free consent. It can also cause conflict in the community. Pay should be fair and respectful of people's time and skill, agreed with community partners, and never used as pressure.

Please sign in to continue.

Questions people ask

Is a signature on a form enough for consent?

Not on its own. Consent must be informed: the person must understand what they agree to. Explain aloud in their language, allow questions, and give separate choices for different uses. The form records the agreement; it does not create understanding.

How much should I pay contributors?

There is no single rule. Consider local wages, the time and skill involved, and travel or data costs. Ask community partners what is fair. Do not pay so much that people feel they cannot say no, and never pay so little that it is exploitation.

Can I use WhatsApp group messages as language data?

Not without the consent of the people who wrote them. Private messages contain personal data and were not shared for AI training. Ask for data directly, with clear consent, instead.

What if someone withdraws after I trained a model?

Remove their data from your dataset and from future training runs, and say in your consent form what you can and cannot undo. Removing data from an already trained model is hard, so be honest about this before people agree.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Sign in to take the test and save your progress