Should I vibe code
Turns academic papers into structured summaries and flashcards
Any model will summarise a paper. Getting a two-column PDF into the right reading order is the decade of work.
?
Their verdict, the Scholarcy Plus (free Article Summarizer tier below it) price and the build-time estimate come from their entry, MIT-licensed. Checked 2026-08-04.
?
Our verdict, the regret score and everything below it. Editorial and unsponsored — nobody can pay to be moved.
The honest answer
why the verdict is what it is
Summarising a paper stopped being the hard part somewhere around 2023. Hand any current model a clean methods section and it will give you a better précis than most reading groups. The work Scholarcy is actually doing sits one step earlier and is completely invisible in the output: turning a two-column PDF with running headers, ligatures, footnotes, figure captions wedged between paragraphs and a reference list in whatever style the journal fancied into ordered, labelled text with resolvable citations. That is what GROBID has spent a decade on, and it is why your version will fail in the one way you cannot catch — silently. Feed a raw text dump to a model and it will summarise whatever order the extractor emitted, confidently, in fluent prose, with no signal anywhere that the results paragraph it is describing was actually assembled from a figure caption on page four and the tail of the discussion. Then you turn that into flashcards and revise from them for a term. Build it for yourself, absolutely. Just never trust a summary you have not spot-checked against the PDF.
What actually breaks
not "if". the specific failures.
- Reading order, on any two-column layout, which is the failure that produces a fluent summary of a paper that does not exist
- Tables and figure captions, which extract as interleaved fragments and get summarised as if they were prose
- Reference parsing, where an inconsistent citation style turns into DOIs that resolve to the wrong paper — or resolve to nothing, which is at least visible
- Scanned PDFs, which need OCR, and OCR of a 1994 photocopy of a chemistry paper is its own project
- In-text citation linking, the feature that makes a summary genuinely useful, because it requires matching "(Smith et al., 2011)" to the right entry in a reference list that spells it four ways
- Numbers, which models paraphrase: an effect size of 0.34 becomes "a modest effect", and a p-value becomes "significant", and both are now unrecoverable from your flashcard
- The flashcards themselves, which persist, get studied, and carry any error forward into whatever you write next
- Publisher access, if the tool starts fetching PDFs on your behalf at any volume — that is the pattern institutions get letters about
Is that you?
the verdict is a default, not a law
- It is your own reading pile and every summary is a first pass before you read the paper properly
- The output always links back to the page and paragraph it came from, so checking takes one click
- The preprints are public and you are not shipping a colleague's manuscript under review to a third-party API
- You would be equally happy with a well-made outline and highlights rather than a polished narrative
- A summary is going to be cited, quoted or taught without anyone opening the original
- You are running it over papers you obtained by pointing a scraper at a publisher
- It handles manuscripts under confidential peer review, because those are somebody else's unpublished work leaving your machine
- The flashcards are the study material rather than a study aid — the errors are invisible and they compound
If you build it anyway
the checklist, then the prompt that enforces it
- Do not parse the PDF yourself. Run GROBID or an equivalent that produces structured TEI with real section labels, and give the model text that already knows what a methods section is.
- Keep provenance on every claim: page number, section, character offsets. A summary sentence that cannot be traced back to a span in the source is a sentence you cannot check, which is the entire risk of this tool.
- Detect multi-column layouts explicitly and refuse rather than guess. A loud "could not determine reading order for pages 3–7" is worth more than a confident wrong summary.
- Extract numbers, statistics and effect sizes verbatim into their own structured field and never let the model paraphrase them. Quote them back next to the prose.
- Resolve references through Crossref or OpenAlex rather than trusting the model's memory of a DOI, and mark anything that does not resolve as unresolved rather than dropping it.
- Show a diff view before anything becomes a flashcard: source paragraph on one side, generated card on the other, and make yourself click through the first fifty.
- Keep the original PDFs, filed by DOI, alongside everything derived from them. Summaries are cheap to regenerate; a library you cannot re-derive is the thing you lose.
- Respect where the papers came from. Downloading at machine speed from a publisher is how institutional access gets suspended for everyone on the network, not just for you.
I am building a tool that turns research PDFs into structured summaries and
flashcards, for my own reading. The dangerous failure is a fluent summary of
text that was extracted in the wrong order. Build defensively against that
first.
1. Do not write a PDF text extractor. Use GROBID or an equivalent that emits
structured TEI with labelled sections, and treat its output as the input to
everything else.
2. Before summarising anything, build a viewer that shows the extracted text
next to the rendered page. I need to be able to see extraction failures
with my own eyes.
3. Detect column count and reading-order confidence per page. Where confidence
is low, mark the pages and refuse to summarise them rather than guessing.
4. Every generated sentence carries provenance: page, section and character
range in the source. Drop any output you cannot attribute.
5. Pull numbers, sample sizes, effect sizes, confidence intervals and p-values
into a separate structured field, verbatim. The model may reference them
but must never restate them in its own words.
6. Resolve every reference against Crossref or OpenAlex. An unresolved citation
is shown as unresolved; never let the model supply a DOI from memory.
7. Flashcards require an explicit accept step showing the source paragraph and
the proposed card side by side. No bulk generation without review.
8. Store the original PDFs filed by DOI, with everything derived kept separate
and regenerable. The library is the asset; the summaries are cache.
9. Do not build a crawler that fetches PDFs from publisher domains. If I ask
for bulk import, restrict it to a local folder or an open repository API
and tell me why.
10. Warn me before any document leaves the machine for a model API, and let me
mark a document as confidential so it can only be processed locally.
11. Out of scope: a citation manager, a reference formatter and anything that
writes prose for me to publish. Zotero already exists and is free.That one keeps you out of trouble. For the prompt that actually builds it, canivibecodeit.com has one.
their build prompt ↗Or don’t build it
the boring option, and the way back out
Buy if you are reading at volume across an unpredictable range of journals and layouts, because that is precisely where a homemade extractor falls over and where a decade of tuning shows. Scholarcy also has a free tier that will tell you within an hour whether the format suits how you read, which is a cheaper experiment than a weekend. Build your own if your reading is narrow — one field, a handful of journals, mostly arXiv preprints with clean sources — because then the parsing problem is small and the tool can be shaped exactly to how you take notes.
Everything worth keeping is the PDFs and your own notes; the summaries are cache and should be treated that way. File the sources by DOI in a plain folder, export flashcards to Anki's format or plain CSV rather than leaving them in your app's schema, and keep the extracted structured text next to each paper so a future tool does not have to re-parse anything. Do that and abandoning the project costs you an afternoon of regeneration rather than a library.
Machine-learning extraction of structured text and references from scholarly PDFs; the hard half of this product, already solved.
Mature open-source reference manager with PDF storage, metadata lookup and annotation.
Questions
Is Scholarcy still going?
Yes. Unlike Lateral and Genei in this category, Scholarcy is operating normally — its public status page records ordinary incidents and recoveries through July 2026 across the library, the engine and three APIs. The only thing we could not establish is the price, because the pricing page is behind a bot challenge and the amounts are injected client-side in a region-dependent currency.
Why is the PDF parsing worth more than the summarisation?
Because summarisation is a commodity and parsing is not. Every model vendor will summarise clean text for a fraction of a cent, and they all do it well. Getting clean text out of the actual artefacts of academic publishing — two columns, rotated tables, ligature-mangled fonts, references in nineteen styles — is a research problem that GROBID has been working on since 2008. Your pipeline is only as good as its worst extraction, and you will never see the worst one, because the summary reads fine.
What will an agent get wrong?
It will reach for pdftotext or PyPDF, get plausible-looking output on the first paper it tries, and declare the extraction step finished. That first paper will be a single-column arXiv preprint. Nothing downstream will ever check reading order again, and no output will carry a page reference, so the moment a two-column journal PDF arrives there is no mechanism anywhere in the pipeline capable of noticing.
How is this different from the Afforai entry?
Afforai is about citation integrity — whether the page your retriever pointed at actually says the thing. This is one layer below that: whether the text your system read was ever in that order in the first place. You can have a perfectly honest citation to a passage that your extractor assembled out of two unrelated paragraphs.
Every week, someone ships something they shouldn’t have.
New verdicts, the worst thing that landed in the trap, and the occasional incident report. No other email, ever.
The vendor switched the lights off in 2025. Whatever you build is now the only version of this that exists.
A citation is a claim, not a formatting flourish. Your retriever cannot tell you whether the page actually says it.
Its premium feature is still called GPT3 Summaries. That is the entire build-versus-buy analysis.
last reviewed 2026-08-05 · verdict is editorial and unsponsored · shared entry data from canivibecodeit under MIT · not legal advice