TL;DR
This post uses one real systematic review as a research methodology example. The age cutoff, the diagnosis method, and the comorbidity exclusions are justified properly — a reason is stated, and the citation backs it up. The search strategy, the grey literature section, and the choice of systematic review itself are not — the citation is doing the explaining, with no reason of the student's own on the page. You'll see both, side by side, so you can spot the difference in your own chapter.
A student showed me a systematic review a few years ago that compared aquatic exercise to land-based exercise for osteoarthritis of the hip and knee. The search strategy was really robust — a full PICO table, Boolean logic, column-wise terms. It's a good systematic review methodology example especially because it shows both sides: the parts that are adequately explained, and the parts where the reasoning could not be put into writing.
Five columns, each with multiple related terms — not just “aquatic” but hydrotherapy, water-based, aqua aerobics sitting next to it.
Then I came to the sentence explaining why it was structured that way, and the reasoning wasn't theirs. It was borrowed from a reference that explained what Boolean operators typically do.
That distinction — between doing something well and explaining why you did it — is what is actually decided in the methods chapter. Not what was done. Why it was done that way, specifically for this study.
It appears more than once in the same chapter. Sometimes the reasoning is there and the quote supports it properly. Sometimes it doesn't, and the quote is left to carry a weight it was never meant to carry. This post goes through both, using the same review, so it's easy to spot the differences in your own work.
What Does a Justified Research Methodology Example Actually Look Like?
The search strategy itself was solid.
Five columns.
Population, condition, intervention, comparison, outcome.
Each joined with an “OR.” Then all five joined with an “AND.”
This is the real PICO structure.
Not a rough version of it.
The Reason That Wasn't a Reason
But then the reason for using it came up.
“Boolean operators were used to combine specific search terms, to further refine the search quality and limit unwanted results (Boland et al., 2017).”
A citation, not a reason. There was also its own citation for the method of combining columns — Bettany-Saltikov (2012).
Neither citation is wrong.
Both say exactly what they are credited with saying. But try this test: put the above citation on a different topic, in a different systematic review.
It still reads fine. That's the problem. Boland et al. did not construct this search. Bettany-Saltikov did not decide that this study needed five columns instead of three.
The citation is doing what the author's own reasoning should do.
Drop the sentence into a different systematic review, on a different topic. If it still reads fine, the reason isn't there.
A sentence was missing.
Something like this: The search remained purposefully broad, to capture aquatic exercise studies that use different terms — hydrotherapy, water-based, aqua aerobics — rather than narrowing down too quickly and missing half the field.
When Does a Citation Actually Support a Reason?
Not every part of this review got it wrong. Three choices got it right. The student gave the reason first. The quote supported it. That's the difference.
| Choice | The reason given | What the citation backed up |
|---|---|---|
| Age cutoff (35+) | Younger patients are more likely to develop OA from an injury, not from years of wear and tear. Setting the cut off at 35 excluded this different cause from the sample. | Research suggests that most OA patients are over 35 years old, while most younger patients develop it through injury rather than gradual wear and tear. |
| Diagnosis by imaging only | Confirming OA by X-ray or MRI, rather than by symptoms alone, reduces data inconsistency. | An imaging-based diagnostic tool that reduces variability between cases. |
| Comorbidity exclusions | Conditions such as serious cardiovascular problems increase the risk that a patient will not complete an exercise program, which would skew the results. | A tool linking certain health conditions to higher risk of quitting in exercise studies. |
Why Was Grey Literature Cited Without a Reason?
This pattern shows up in more than one research methodology example, and this review's grey literature section is one of them. Newsletters, government reports, and magazines—sources outside of peer-reviewed journals. The review identified these alongside database searches.
The Citations Behind It
“These types of sources can be useful for 'systematic reviews' (SRs) because they provide up-to-date, unbiased, and in-depth research-based analysis of a topic (Adams et al., 2016).”
This reference is accompanied by three more references. Mahood et al. (2014) on what 'grey literature' is. Benzies et al. (2006) on how to find it. Saleh et al. (2015) on what the implications of including it are.
Four references in total. But no specific reason was given for this review.
Each of these explains why grey literature is generally useful in systematic reviews. None of them says why this review — aquatic versus land-based exercise for OA — is needed. What gaps were expected? Missing trial data? Clinical guidelines not yet in journals? Practitioner reporting that was not formally published? The review does not say.
Same test as before.
Change this paragraph to another systematic review.
It still reads fine.
That's what it says.
When Were the Exclusion Filters Actually Applied?
The search started with 8,881 records. It ended with 9 studies.
I always wonder how the pool gets so small. Not just where it lands.
Two things happened in part to this.
The search wasn't filtered based on aquatic vs. terrestrial from day one. That was added later. After the pool was already down to 5,728. Once the date and language filters had already been run.
With the strength and pain of the desired results as well.
That also came later. After the interference filter, not before.
This kind of staged, transparent filtering isn't unique to this review — PRISMA's flow diagram guidance exists specifically because reviewers need to see how a search pool was narrowed, not just the final count. A methodology justification worksheet can help you track that same kind of reasoning stage by stage in your own chapter.
Why the Order Belongs in the Write-Up
Most students write exclusion criteria as a flat list. Applied all at once. As if it happened in one pass.
This one didn't. It was step-by-step. History and language first. Then interventions. Then results. Then manual reading of titles and abstracts. Then full text.
This order is worth keeping in writing.
Not very smooth.
It tells a reader something that a final list cannot. Which decision came first, which only meant that it happened when the earlier filters had already narrowed the field.
Check whether your own exclusion criteria show when each filter was added, not just what the final list contains.
Why Was the Choice of Systematic Review Never Justified?
Not every research methodology examples get the design choice right either.
‘Systematic review’ was not the only method being considered.
But the way in which this choice was explained does not make this clear.
The reason given was: the research by Chavez et al. (2016), which describes what systematic reviews typically provide.
Alternative methods were mentioned in just one line. Something like — other methods take longer and are harder to implement. No names, no details; just that they are “complicated”.
More complex than what? Meta-analysis? Scoping review? Or narrative review? The text does not explain this.
It is not necessary, because the citation itself makes it clear. Chavez et al. said that ‘systematic reviews’ are useful.
This is taken for granted.
But Chavez et al. did not decide that this study needed a ‘systematic review’ rather than some other method. They did not evaluate the timeline of this study, its scope, and the available literature against alternative methods. If there was a comparison, it was somewhere that did not make it into the text.
This is the same test as for the other sections. Just change the subject; the sentence still holds true. It has failed three times so far—three different sections of the chapter with the same structure, and the same methodology.
When Does “Unsatisfied” Need Its Own Reason?
The PEDro scale was used to assess one study included in the student's systematic review of swimming and cycling for osteoarthritis (OA) patients — Alcatan et al. (2016). The study received a score of 5 out of 10. This appraisal is its own methodology justification example — showing when a judgment call needs a reason and when it doesn't get one.
The PEDro scale itself scores 11 items — the official PEDro scale documentation confirms this — inclusion criteria, random allocation, concealed allocation, baseline similarity, blinding at three levels, follow-up completeness, intention-to-treat analysis, between-group comparisons, and variability reporting — with a total score out of 10, since the first item isn't counted toward the final tally.
| Choice | The reason given | What the citation backed up |
|---|---|---|
| Eligibility criteria specified | Satisfied | Diagnosis via Kellgren-Lawrence grading |
| Random allocation | Satisfied | Blinded investigator assigned groups |
| Baseline similarity | Unsatisfied | BMI and hip circumference differed |
| Therapist blinding | Unsatisfied | Same therapist ran both groups |
| Assessor blinding | Unsatisfied | Assessors knew the study's aims |
| Outcome data from 85%+ of subjects | Satisfied | 98% and 97% adherence |
| Point measures and variability | Unsatisfied | No variability reported |
Where the Marks Were Lost
Baseline matching was marked as unsatisfactory.
The swimming group had a higher BMI and a larger hip circumference than the cycling group before the study began.
Therapist blinding was marked as unsatisfactory. The same therapist administered both groups and knew which group was which.
These are two different criteria on the PEDro scale, one repeated. Blinding the therapist covers those who administered the exercise sessions. Blinding the assessor covers those who later measured the outcomes. Both failed here, but for related reasons — the same people were involved in the delivery and retesting, so neither line could be isolated from the other.
The blinding of the assessor was also rated as unsatisfactory.
The people measuring the outcomes were not blinded to the study objectives.
The reporting of point measures and variability was rated as unsatisfactory. No variability was reported for the outcomes tested.
Where the Marks Were Earned
Six other criteria were met. Eligibility was clearly defined—the methods confirmed the OA diagnosis by Kelgren-Lawrence grading with detailed inclusion and exclusion criteria.
Subjects were randomly assigned to groups by a blinded investigator.
Allocation was similarly concealed.
Outcome measures were recorded for more than 85% of participants—including 98% completion of swimming and 97% completion of cycling. Each participant received the same treatment as assigned.
Statistical comparisons between groups were also reported, with results presented in bar charts and discussed directly.
A study scoring 5/10 on PEDro isn't disqualified — but it does affect how confidently its findings get treated once you move into writing up your results.
What Was Actually Done Right Here
For each point (or mark) where the criteria were not met, a specific reason derived from the paper itself is given—not an estimate or a conventional mark. The BMI difference is mentioned. Details about the therapist are also given. The lack of ‘blinding’ is specifically pointed out, not simply a box ticked and the matter is closed.
This is the same standard that is adopted for all other parts of this chapter. Explain what comes up; do not leave any point or mark unexplained.
Checklist
Run these against your own chapter. These are the same patterns that show up across most research methodology examples. If you haven't mapped out your methodology chapter yet, a dissertation chapter outline is worth doing first — these checks matter most once that structure is already in place.
- Give reasons before citing the source, not after
- Reread each sentence in the methods and see if it applies to other studies
- Name specific alternative methods considered, not just “other methods” — this is where the ‘design-choice’ part of this post fell short
- Write down the order in which exclusion filters were applied, not just the final list
- In the quality appraisal, give reasons for each point that did not meet the criteria, not just mention the mark or mark
FAQ
What does it mean to justify a methodology choice instead of describing it?
This means giving a study-specific reason behind a choice, not just naming it or stating that the method exists.
Is citing a source enough to justify a decision?
No, a citation should back up a previously stated reason, not stand in for one that was never given — both the search strategy and gray literature sections in this post are examples of a citation.
What is the "because test"?
Reading the methodology sentence and asking if it still makes sense in a completely different study. If yes, then the reason is not there.
Why does the order of exclusion filters matter?
It shows which decisions came first and which decisions only made sense when earlier filters had already narrowed the pool.
Does a quality appraisal score need justification too?
Yes, every "unsatisfied" mark requires a stated reason from the paper, not just a checked box.
Final Thoughts
The methods section is not judged by what was done. It is judged by whether a reason was given for doing it. The systematic review used in this post — Comparing aquatic and land-based exercise for osteoarthritis of the hip and knee — had robust methods: a real search strategy, a real PRISMA funnel, a real assessment tool. But the writing didn't always explain why those methods were relevant to this particular study. The search strategy took a citation rather than explaining its rationale. The grey literature section did the same. In each case, the work wasn't problematic. What was missing was a sentence explaining why it was relevant here.
