arxiv:2607.04439

ResearchStudio-Idea: An Evidence-Grounded Research-Ideation Skill Suite from ML Conference Outcomes

Published on Jul 5

· Submitted by

Qihao Zhao on Jul 7

Microsoft

Upvote

Authors:

Qihao Zhao ,

Abstract

ResearchStudio-Idea provides a skill suite for effective research ideation that combines literature search, novelty checking, and pattern-guided generation to produce traceable research proposals.

Generated by Qwen/Qwen2.5-Coder-32B-Instruct

Large language models have made research ideation increasingly accessible, yet effective idea development requires more than generating candidate directions. Researchers must ground a problem in current literature, identify meaningful bottlenecks, differentiate from existing solutions, and evaluate risks before committing to implementation. We present ResearchStudio-Idea as a reusable skill suite for this first mile of research ideation. The suite includes Paper-Search, a standalone multi-source literature search skill; Scoop-Check, a standalone prior-art collision checker for novelty claims; and IdeaSpark, the end-to-end skill that composes evidence grounding, pattern-guided generation, collision retrieval, audit, and idea-card rendering into one workflow. IdeaSpark is constructed from a corpus of 1,947 machine learning conference papers collected from ICLR, ICML, and NeurIPS between 2021 and 2025, including Oral papers, a separately tracked high-citation subset, and rejected submissions. Analysis of these outcomes reveals 31 recurring ideation sub-patterns, consolidated into 15 reusable ideation patterns. Each pattern is operationalized as a structured card containing research contexts, bottleneck types, differentiation strategies, supporting precedents, and common failure modes. Given a research problem and an evidence bundle, IdeaSpark evaluates evidence readiness, reconstructs the surrounding research context, identifies unresolved bottlenecks, selects relevant patterns, instantiates one candidate direction, retrieves potentially conflicting prior work, and performs outcome-informed auditing. This workflow transforms reusable ideation patterns into traceable research proposals. Blind automated-judge evaluations show that IdeaSpark consistently produces stronger research proposals than no-skill and generic-skill baselines while maintaining competitive novelty.

View arXiv page View PDF Project page GitHub 307 Add to collection

Community

fistyyyy

Paper author Paper submitter 2 days ago

•

edited 2 days ago

From a research problem to a reviewer-defensible idea card — Automating the First Mile of Research Ideation.

ResearchStudio-Idea is a reusable research ideation skill suite that assists researchers in developing well-grounded research proposals. It combines three skills: Paper-Search for multi-source literature retrieval, Scoop-Check for prior-art collision detection, and IdeaSpark, an end-to-end workflow skill that grounds ideas in evidence, identifies research bottlenecks, applies reusable ideation patterns, audits risks, and generates structured research idea cards. The framework is built from an analysis of 1,947 machine learning papers published between 2021 and 2025, from which 15 reusable ideation patterns are distilled. Experiments show that it produces higher-quality research proposals than generic ideation baselines while maintaining competitive novelty.

librarian-bot

about 19 hours ago

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

fistyyyy

Paper author about 18 hours ago

Thanks @librarian-bot — useful list! A quick note on where ResearchStudio-Idea sits relative to these.

Our scope is deliberately narrow — the first mile of ideation only: from a research problem to one reviewer-defensible, implementable idea card (method + falsification + prior-art collision + audit). We explicitly do not run experiments, write full papers, or predict acceptance.

That framing separates us from most of the recommended work:

Full-lifecycle / autonomous systems — FARS, AI for Auto-Research: these automate idea → experiment → paper. We're complementary: an idea-stage quality gate such systems could sit on top of, motivated by the documented fragility of full autonomy.
Evaluation / benchmark studies — Measuring the Gap, RWGBench: these judge ideas or related-work positioning. We generate and audit; our blind quality/novelty study is an endpoint check, not the contribution. (Scoop-Check does claim-level prior-art collision, not related-work writing.)
Contamination / seed-exposure diagnosis — Read, Grep, and Synthesize: a failure mode we address by design — citation-free abstract recipes, collision retrieval, and faithful-by-construction provenance to avoid regurgitation.
Evidence-gated orchestration — ResearchLoop: closest in spirit; we share the evidence-gating philosophy but narrow it to a single ideation + audit node backed by an outcome-grounded pattern library.
LLM-as-judge critiques — On the Limits of LLM-as-Judge for Novelty: we agree — that's exactly why our novelty checks are retrieval-grounded rather than opinion-based; it also flags a real limitation of our own automated-judge evaluation.

Core differentiator: patterns are induced from Oral / High-Cited / Reject outcomes as contrastive cards (success ⊕ failure) — not accept-only or model-opinion — and packaged as a model-agnostic, retrieval-gated skill.