How to do AI researchMiddleLesson 3277 min read

Compute on a budget

You can do real research without a big GPU cluster. Start small, use free and academic resources wisely, write efficient code, and know when you truly need more.

✦ Ask the tutor 🧪 Lab · 30 min 📝 Lesson test

Speed

Lesson in motion

In 60 seconds

Compute on a budget

You can do real research without a big GPU cluster. Start small, use free and academic resources wisely, write efficient code, and know when you truly need more.

1/5

In simple words

Before a big wedding, a cook first makes one small pot to taste the recipe. Only when it tastes right does he cook for five hundred guests. If he started with the giant pot and got the salt wrong, all the food and money would be wasted. Research compute works the same way.

Compute means the processing power you use to train and run models, usually GPU time. It is often the scarcest resource for students and independent researchers. The good news: many important research questions can be studied with small models, small data and careful planning. Budget limits also push you towards clearer, simpler experiments.

Small models first

Almost every idea can be tested first on a small model and a small dataset. If the idea does not work at small scale, you save a lot of compute. If it does work, you have a clean result to build on.

Small models: many open models come in several sizes. Use the smallest one that can show the effect you care about.
Subsets: debug on 1% of the data. Only use the full dataset when the pipeline works.
Short runs: check that loss goes down for a few hundred steps before launching a long run.
Scaling checks: try two or three small sizes. If the trend is steady, you have evidence about what may happen at larger scale. It is a hint, not a guarantee.
Adapt instead of train: fine-tuning with LoRA or QLoRA, or using a model as it is, is far cheaper than training from scratch.

Where to find compute

Source	What it is	Things to know
Your own laptop	CPU, sometimes a small GPU	Good for small models, data work, evaluation and debugging. Free and always available.
Free notebook tiers	Some cloud notebook services give free, limited GPU time	Limits, GPU types and rules change often. Sessions can stop without warning, so save checkpoints. Check the current terms.
University computers	Shared GPU servers or clusters in many colleges, including IITs and IISc	Ask your department or a professor's lab. You usually need a faculty sponsor and must share fairly.
Research credit programmes	Some cloud providers and AI companies offer credits or free access for academic research	They open and close over time. Search for current ones and read the conditions. A clear one-page proposal helps.
Collaborations	A lab or company partner runs the large experiments	Agree early on data use, authorship and what you can publish.

Watch out

Never leave a paid cloud GPU running when you are not using it. Set a budget alert, stop machines when a run ends, and do not put cloud keys or passwords in code or notebooks you share.

Efficient code: get more from the same GPU

Measure first: time one training step. Find out if the GPU is busy or waiting for data.
Prepare data once: tokenise or preprocess once and save it, instead of redoing it every run.
Mixed precision: most modern frameworks support lower-precision training, which often uses less memory and runs faster. Check your framework's docs.
Gradient accumulation: if a big batch does not fit, add up gradients over several small batches.
Checkpoints: save regularly so a stopped session does not lose hours of work.
Reuse results: cache model outputs you evaluate many times, and log every run so you never rerun something by accident.

Estimate before you run

A simple estimate saves a lot of pain. Measure a short run, then multiply.

Back-of-the-envelope compute plantext

Measured: 1 training run on the small setting = about 40 minutes
Runs needed:
  methods (baseline, mine)            2
  ablations (no A, no B)            + 2
                                    = 4 settings
  seeds per setting                 x 3
                                    = 12 runs
Total: 12 x 40 min = 8 hours of GPU time
Add about 50% for debugging and reruns = about 12 hours
Fits a free tier over a few days?  Maybe, with checkpoints.
Plan B: fewer ablations first, add them only if the main result holds.

Note

The numbers above are made up to show the method. Your own measured time per run is what matters.

When do you really need more compute?

You probably don't need more yet

The pipeline still has bugs
You haven't tried a smaller model
Each run is slow because data loading is slow
You are unsure what the hypothesis is

You may truly need more

The effect only appears above a certain model size, and you have evidence for that
Small runs work and the next question is about scale
You need enough seeds for a trustworthy claim
A reviewer or mentor asks for a larger check of a clear result

Do this

Many strong research areas need little compute: careful evaluation, data quality, interpretability of small models, efficient methods, and work on Indian languages with open models. Choosing a question that fits your compute is a skill, not a compromise.

Try this. Take the experiment you want to run next. Time one short run. Then write the plan above for it: settings, seeds, total hours, and a plan B.

Watch and read more

Lab

A written compute plan for one experiment, with a measured or estimated run time, total hours and a plan B.

~30 min

The problem

Pick an experiment you want to run: your reproduction target, or an idea of your own. You can do this on paper.

List every setting you need: methods, baselines, ablations.
Choose the number of seeds per setting and say why.
Time one short run if you can (even on a laptop with a tiny subset). If not, write an honest guess and mark it as a guess.
Multiply to get total GPU hours, then add a margin for debugging.
Say where you will run it (laptop, free tier, college server) and what could stop you.
Write a plan B that fits in half the budget: which runs you would drop first, and why.

You are done when

Every setting (methods, baselines, ablations) and the seed count are listed
The total hours come from a measured or clearly labelled estimated run time
A margin for debugging and reruns is included
Plan B keeps the most important comparison and drops the least important runs first

Hard questions

Try to answer before you reveal. If you can answer these, you understood the lesson.

Q1To save compute, a friend suggests running the baseline with 1 seed and your method with 5. Is that fair?Reveal

No. The comparison is then uneven: you know your method's spread but not the baseline's, and one baseline run could be lucky or unlucky. Use the same number of seeds, and the same seeds, for every method you compare. Cut ablations or settings before you cut fairness.

Q2An idea works on the two smallest model sizes you can afford. Can you claim it works for large models?Reveal

Not directly. You can say it held at the sizes you tested, and show the trend. Some effects shrink, grow or reverse with scale. State the sizes clearly in your limitations, and invite others with more compute to test larger models.

Please sign in to continue.

Questions people ask

Can I do AI research with only a laptop?

Yes, for many questions. You can run small models, study evaluation, analyse datasets, and do interpretability work on small networks. Use free or shared GPUs for the parts that need them. Start with questions that fit what you have.

Which free GPU service should I use?

Offers change often, so we don't name one. Look at what free notebook services and your college currently offer, compare their limits, and read the terms. Save checkpoints often, because free sessions can end without warning.

Are results on small models taken seriously?

Often yes, if the question fits the scale and the experiment is clean. Be honest in your limitations that larger models may behave differently. A clean small result is more useful than a messy large one.

How do I apply for research credits?

Search for current research credit or academic access programmes from cloud providers and AI companies. Most ask for a short proposal: your question, why it matters, what compute you need and why. A faculty supporter helps. Read the terms on data use and publication.

Lesson test

5 questions. Get 3 right (60%) to pass and complete this lesson.