Kurate by Kivira
Research quality assurance at portfolio scalePCORI already judges the rigor a study proposes. Now it can measure the rigor it delivered.
PCORI selects studies through merit review against its formal Methodology Standards — mandatory minimum requirements for sound science. Methodological rigor — whether a funded study used a valid comparator, held to its prespecified endpoints, and drew conclusions its design could support — is a distinct axis, and one no funder has been able to measure at scale after the fact, because appraising it took expert-hours per study. When Kurate graded a corpus of trials drawn from higher-quality domains, the spread across that axis was wide.
Even in a sample drawn from higher-quality domains, grades span the full range — the same wide spread found across the published literature generally, not a property of any one funder's choices. What's new isn't the spread; it's that it's now measurable. Once it is, a funder can see which work clears the bar for confident interpretation (A–C) and direct more of the portfolio toward it.
Kurate's premise doesn't rest on our numbers alone. A large, independent metascience literature has documented that questionable research practices are common — outcomes switched between registration and publication, selective reporting, underpowered designs, p-hacking and HARKing — leaving a public record in which a study can look authoritative and still not support its own conclusions. Public and philanthropic funding flows into that record. Kurate turns decades of this evidence into a check that runs at portfolio scale.
Kurate is built by researchers who publish in statistics, research methodology and metascience — including in Nature, NeurIPS and ICLR. Members of the team have separately built the peer-reviewed precursors to Kurate's core method — automated comparison of trial registrations against published results. That background shapes how the tool handles fragile claims, uncertain comparisons and noisy literature.
Research in causal inference, deep generative modelling and multimodal ML; 50+ peer-reviewed publications including ICLR and NeurIPS.
Two doctorates — Engineering (Vision, Speech & Signal Processing), University of Surrey; and Applied Mathematics for the Human & Social Sciences, University of Lausanne. Affiliations: University of Lausanne, University of Surrey, and the Sense Center for Innovation and Research.
Contributes to Kurate's evidence-evaluation methodology and clinical rubric development; 50+ peer-reviewed publications, including three metascience papers in Nature.
Expertise in research-integrity assessment and LLM-workflow evaluation; author of published registration-versus-report comparison tools.
How Kurate works
Kurate reconstructs each study's intended design from its own paper trail, then measures the published report against it — the work a methodologist and a statistician would do together, applied uniformly at scale. Two documents in; one auditable verdict out.
One rubric, applied identically to every study — reproducible, auditable, and fast enough to run across a whole portfolio.
Where Kurate fits
PCORI already scores rigor at merit review against its Methodology Standards, and peer-reviews every awardee's final report before release. The one question neither answers at scale: did the funded work deliver the rigor it proposed? Kurate is built for exactly that gap — complementing the mechanisms already in place, not replacing them.
Is the proposed science rigorous?
PCORI's Methodology Standards set formal, mandatory minimum requirements for sound science — covering design, conduct and reporting — and merit review scores each proposal against them, with methodologists and biostatisticians on the panel. Assessed on the proposal, before any data exists.
Is the final report sound and reported?
PCORI peer-reviews every awardee's draft final report before public release, and requires registration and results on ClinicalTrials.gov — with analytic and outcome changes documented. Thorough, but a manual review of the report as submitted, one study at a time.
Did the funded work deliver the rigor it proposed?
No portfolio-scale measure closes this loop today. Kurate compares each published study against its own pre-registration, across the whole portfolio — the delivered-rigor counterpart to the rigor PCORI scores at merit review, and reputation-blind by construction.
PCORI's own Methodology Standards already require studies registered on ClinicalTrials.gov with their outcomes and analysis plan, and every amendment documented — so the pre-specification is on file by design. What isn't done at scale is checking the published report against it. That comparison is exactly what Kurate performs — automatically, across the whole portfolio.
What it lets PCORI do
Across a comparative-effectiveness portfolio on the order of $0.5 billion a year, Kurate grades every study on one rubric in a single pass, turning rigor from something you could only sample by hand into a portfolio-wide layer alongside the merit review and Methodology Standards PCORI already applies. What that puts within reach:
See which schemes and calls actually deliver rigor
Rank funding schemes, programmes and calls by the methodological quality of the work they produce — so PCORI can direct more funding toward the ones that consistently deliver, and give the ones that lag the evidence to sharpen their guidance or evaluation criteria. The same lens resolves to the researcher level where that's useful — surfacing where rigor support would help most, not a blacklist. Grounded in method, not reputation, and defensible because every grade is auditable.
Track portfolio quality over time
Watch methodological quality across a funding scheme, a discipline, or a decade — and test whether a given call raises or lowers the rigor of the work it produces.
Strengthen the schemes that need it
See which funding schemes and evaluation panels produce the most rigorous work — and where adjusted guidance would raise quality the most.
Report defensible rigor metrics
Give PCORI's leadership, its Board of Governors, and the public a reproducible measure of research quality that complements the Methodology Standards instead of competing with them.
See a field's quality evolve
Track whether rigor in a domain is improving or eroding over years — the kind of question that shapes where a new call or initiative should be aimed.
How to read the rigor
- Registration-anchored. Quality is judged against the study's own pre-data-collection commitments, not a reviewer's taste.
- GRADE-like, without pool contamination. Comparable in spirit to a Cochrane/GRADE appraisal, but dealbreaker studies are excluded rather than downweighted — so one flawed study can't quietly dilute a synthesis.
- Cross-domain in one pass. Clinical, methodological and statistical checks applied uniformly — precisely where human review is scarce, slow, and inconsistent.
- Reproducible & auditable. The same rubric, applied the same way, every time — a property no distributed panel of reviewers can guarantee, and the basis for defensible portfolio metrics.
- Scope-honest. Strongest in medication, treatment and RCT-relevant domains, where the registration-versus-report comparison is most decisive.
See Kurate for yourself
The figures in this brief are illustrative. To see the real thing, ask us for a demo — we'll walk you through Kurate on live published research: the methodological grade, the dealbreaker flags, and the detailed analysis behind each grade.
About the figures. The grade distribution is Kurate's own result on a corpus drawn from higher-quality research domains; it is presented as a floor, not a random portfolio sample, and does not yet establish how PCORI-funded output is distributed. PCORI's funding scale (on the order of $0.5 billion a year in awards) is approximate, from PCORI reporting. PCORI mechanisms referenced — merit review against the PCORI Methodology Standards, peer review of awardee final reports before release, and required registration and results reporting on ClinicalTrials.gov with documented analytic changes — are per PCORI public documentation. The savings floor scales the annual award base by an illustratively small share and is a lower-bound sketch, not a projection. Kurate is decision-support for research quality assessment and does not replace clinical, regulatory, or institutional review. Kurate is built by Kivira Health — kivira.health. Kurate is built by Kivira Health — kivira.health.
Selected related publications
Methodological and meta-scientific work associated with the Kurate team.
- Vowels, M. J. (2023). Misspecification and unreliable interpretations in psychology and social science. Psychological Methods, 28(3), 507–526. DOI
- Vowels, M. J., Vowels, L. M., & Wood, N. D. (2023). Spectral and cross-spectral analysis: A tutorial for psychologists and social scientists. Psychological Methods, 28(3), 631–650. DOI
- Vowels, M. J. (2023). Prespecification of structure for the optimization of data collection and analysis. Collabra: Psychology, 9(1), Article 71300. DOI
- Vowels, M. J. (2024). Trying to outrun causality with machine learning: Limitations of model explainability techniques for exploratory research. Psychological Methods. DOI
- Vowels, M. J. (2025). A causal research pipeline and tutorial for psychologists and social scientists. Psychological Methods. DOI
- Vowels, M. J. (2024). Typical yet unlikely and normally abnormal: The intuition behind high-dimensional statistics. Statistics, Politics and Policy, 15(1), 87–113. DOI
- Aczel, B., Szaszi, B., Clelland, H. T., Kovacs, M., Holzmeister, F., et al. (2026). Investigating the analytical robustness of the social and behavioural sciences. Nature, 652(8108), 135–142. DOI
- Higgins, W. C., Clarke, B., Elson, M., & Cummins, J. (2026). Recommendations for incorporating LLMs into psychological research: A commentary on Austin and colleagues (2026). PsyArXiv. DOI
- Ahnström, L., Bruckner, T., Aspromonti, D. A., Caquelin, L., Cummins, J., et al. (2026). TrialScout links published results to trial registrations using a large language model. medRxiv. DOI
- Elson, M., Hussey, I., Clarke, B., Norwood, S. F., Grinschgl, S., Arslan, R. C., et al. (2026). Against anonymising meta-scientific data. PsyArXiv. DOI
- Cummins, J., Clarke, B., Hussey, I., & Elson, M. (2026). RegCheck: A tool for automating comparisons between study registrations and papers. arXiv. DOI
- Miske, O., Abatayo, A. L., Daley, M., Dirzo, M., Fox, N., Haber, N., Hahn, K. M., et al. (2026). Investigating the reproducibility of the social and behavioural sciences. Nature, 652(8108), 126–134. DOI
- Cummins, J. (2025, September 1). Psychology needs… an AI revolution. The Psychologist. Article
- Röseler, L., Kaiser, L., Doetsch, C., Klett, N., Seida, C., Schütz, A., Aczel, B., et al. (2024). The Replication Database: Documenting the replicability of psychological science. Journal of Open Psychology Data, 12(1), Article 8. DOI
- Tierney, W., Hardy, J. H., III, Ebersole, C. R., Leavitt, K., Viganola, D., Clemente, E. G., Gordon, M., Dreber, A., Johannesson, M., Pfeiffer, T., Hiring Decisions Forecasting Collaboration, & Uhlmann, E. L. (2020). Creative destruction in science. Organizational Behavior and Human Decision Processes, 161, 291–309. DOI
- Van Dessel, P., Cummins, J., Hughes, S. J., Kasran, S., Cathelyn, F., & Moran Yorovich, T. (2020). Reflecting on twenty-five years of research using implicit measures: Recommendations for their future use. Social Cognition, 38(Supplement), S223–S242. DOI