Capstone projectsMiddleLesson 3346 min read

Test it with real people

A small pilot with a few real users teaches more than weeks of building alone. Do it with consent, collect only what you need, measure, and improve.

✦ Ask the tutor 🧪 Lab · 25 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Test it with real people

A small pilot with a few real users teaches more than weeks of building alone. Do it with consent, collect only what you need, measure, and improve.

1/4

In simple words

Before a big school function, you do a rehearsal with a few friends watching. They tell you the microphone is too soft and one song is too long. You fix it before the real day. A pilot is a rehearsal for your project, with a few real users.

You built your project and tested it yourself. Now you need real users. People will type questions you never thought of, misunderstand buttons, and use the tool in noisy places on slow phones. A pilot is a small, short test with a few people, done carefully, so you learn this before sharing widely.

Consent first

Before anyone tests your project, tell them clearly what it is, what you will record, and what you will do with it. Let them say no, or stop at any time, with no problem. Write down that they agreed.

Explain in simple words, in their language: "This is a student project. It may make mistakes. I will note whether it answered your question and your comments. I will not keep your name or number in the results."
For anyone under 18, get permission from a parent or guardian, and follow your school's or college's rules.
Ask your teacher or mentor whether your institution has its own rules for projects with people.
Never test with real money, real medical decisions or real legal matters.

Collect as little as possible

Privacy is easiest when you never collect the data in the first place. Give each tester a code like T01 instead of using names. Do not ask for phone numbers, addresses or ID numbers unless you truly need them. If someone types personal details into a comment, remove them before you share any results. Laws on personal data, such as India's Digital Personal Data Protection Act, 2023, apply to many projects that handle personal data. Check the current rules and ask your mentor if you are unsure.

A simple pilot plan

1
Pick 5–10 testers
Real users from your plan: parents, shopkeepers, classmates, family. A few is enough to find the biggest problems.
2
Give them real tasks
"Find out when the library closes on Saturday." Do not explain how to use the tool first. Watch what they do.
3
Record the same things for everyone
Tester code, consent yes or no, task done or not, a 1–5 rating, and one comment.
4
Summarise
Success rate, average rating, and the top three problems, with personal details removed.
5
Fix and test again
Change the biggest problem, then run the same tasks again to see if the number moved.

A short feedback form

Question	Type
Did you get what you needed?	Yes / Partly / No
How easy was it?	1 (very hard) to 5 (very easy)
What was confusing or wrong?	Short free text
Would you use it again?	Yes / Maybe / No
Anything else?	Short free text, optional

Note

With five or ten testers, numbers move a lot by chance. One person more or less can change a success rate by 10 or 20 points. Use the numbers to spot big problems, and read every comment. Do not claim precise results from a tiny pilot.

Python: comparing two pilot rounds honestlypython

round1 = {"testers": 8, "tasks_done": 5}
round2 = {"testers": 8, "tasks_done": 7}

for name, r in [("Round 1", round1), ("Round 2", round2)]:
    rate = r["tasks_done"] / r["testers"]
    print(f"{name}: {r['tasks_done']}/{r['testers']} tasks done ({rate:.0%})")

print("With only 8 testers, one person changes the rate by", f"{1 / 8:.0%}.")
print("Report the counts, not just the percentage, and read the comments.")

Real example

A group in Madurai tested their Tamil FAQ bot with six shoppers at a small store. Four of them typed Tamil in English letters, which the group had not tested. The bot said "not found" to most of them. The group added common spellings to their word list and tested again the next day. The simple fix made a big difference.

Watch out

If a tester is upset, confused about what is real, or at risk (for example, they think a scam test message is real), stop, explain, and help. People matter more than your results.

Try this. Write the one-paragraph consent message you will read to testers, in their language. Read it to a friend and ask: "Did you understand what I will record?"

Watch and read more

Lab

A pilot summary function that uses only consented responses and removes names and phone numbers.

~25 min

The problem

The responses are made up. Each has a tester code, a name, a phone number, consent, whether the task was done, a 1–5 rating and a comment.

Run the starter. Its summary uses every response, even from people who did not consent, and it leaks names.
Change summarise(responses) so that it uses only responses where consent is True.
Return a dictionary with: used (number of consented responses), excluded (the rest), success_rate (a fraction from 0 to 1), avg_rating, and comments (a list of strings).
The summary must not contain any tester's name or phone number. Remove any long run of digits from comments, for example by replacing it with "[number removed]".

Practice lab

Edit the code and run it. Python runs in your browser — nothing is sent to our server.

Ctrl + Enter to run

import re

RESPONSES = [
    {"tester": "T01", "name": "Asha",   "phone": "90000 00001", "consent": True,  "done": True,  "rating": 4, "comment": "Quick answer about timings."},
    {"tester": "T02", "name": "Ravi",   "phone": "90000 00002", "consent": True,  "done": False, "rating": 2, "comment": "It did not understand my Hinglish question."},
    {"tester": "T03", "name": "Meena",  "phone": "90000 00003", "consent": False, "done": True,  "rating": 5, "comment": "Nice."},
    {"tester": "T04", "name": "Imran",  "phone": "90000 00004", "consent": True,  "done": True,  "rating": 5, "comment": "Good. Call me on 90000 00004 if you need more."},
    {"tester": "T05", "name": "Kavya",  "phone": "90000 00005", "consent": True,  "done": True,  "rating": 4, "comment": "Show the date of the notice please."},
    {"tester": "T06", "name": "Suresh", "phone": "90000 00006", "consent": False, "done": False, "rating": 1, "comment": "Did not try."},
    {"tester": "T07", "name": "Lata",   "phone": "90000 00007", "consent": True,  "done": False, "rating": 3, "comment": "Letters were too small on my phone."},
    {"tester": "T08", "name": "Joseph", "phone": "90000 00008", "consent": True,  "done": True,  "rating": 4, "comment": "Useful, but it said not found for fees."},
]

def summarise(responses):
    # TODO: use only consented responses, and remove personal details
    done = sum(r["done"] for r in responses)
    return {
        "used": len(responses),
        "excluded": 0,
        "success_rate": done / len(responses),
        "avg_rating": sum(r["rating"] for r in responses) / len(responses),
        "comments": [r["name"] + ": " + r["comment"] for r in responses],
    }

summary = summarise(RESPONSES)
for key, value in summary.items():
    print(key, "=", value)

Output

No output. Add print(...) to see values.

You are done when

Only consented responses are counted
No names or phone numbers appear anywhere in the summary
The success rate and average rating match the consented responses
You listed the top problems from the comments and one change you would make

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1Your summary shows 5 of 6 tasks done. A teammate wants to write "83% success rate" on the project poster. What would you write instead, and why?Reveal

Write "5 of 6 testers completed the task in our small pilot", and say who the testers were. With six people, one person changes the rate by about 17 points, so a percentage alone suggests more precision than you have. Counts plus context are honest and still useful.

Q2A tester's comment says: "My son Rahul in Class 7 also wants this." Is removing phone numbers enough?Reveal

No. A name plus a class can identify a child, especially in a small school. Remove or generalise such details ("a family member also wants this") before saving or sharing. Phone numbers are only one kind of personal data; names, places, classes and ID numbers need care too.

Please sign in to continue.

Questions people ask

How many testers do I need?

For a capstone pilot, five to ten is a common, practical number. It will not give precise statistics, but it usually shows the biggest problems. If you want stronger numbers later, you need more people and a more careful design.

Can I test with my family and friends?

Yes, if they are real users of the kind in your plan, but know that they may be too kind. Ask them to be honest, watch what they do rather than only asking what they think, and try to include at least a few people who do not know you well.

What if testers type personal details into the tool?

Tell them beforehand not to. If it still happens, remove those details from your notes and logs before saving or sharing. Do not keep logs longer than you need.

Should I change the tool during the pilot?

Try not to change it in the middle of a round, or you will not know what caused the difference. Finish the round, make changes, then run the same tasks again.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.

Test it with real people

Test it with real people

Consent first

Collect as little as possible

A simple pilot plan

Pick 5–10 testers

Give them real tasks

Record the same things for everyone

Summarise

Fix and test again

A short feedback form

Watch and read more

The problem

Practice lab

You are done when

Hard questions

Questions people ask