RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources
Abstract
Skills are a useful abstraction for software agents, turning human and agent experience into reusable procedural knowledge. Yet existing skill libraries are mostly hand-written, text-centric, or derived from agent traces, leaving tutorial videos and other multimodal human resources largely underused. We present RESOURCE2SKILL, a framework that distills multimodal resources, including tutorial videos, repositories, articles, and reference artifacts, into executable skills for software agents. RESOURCE2SKILL organizes these skills as a hierarchical multimodal Skill Wiki, where each entry combines structured text, code, visual examples, metadata, and provenance. This design preserves complementary signals from different resources: videos capture temporal operations and visual effects, code captures executable tool patterns, and articles or artifacts provide conceptual and stylistic grounding. At inference time, agents retrieve and compose relevant skills from the wiki; when coverage is insufficient, the same construction operator can acquire new skills online. Across seven practical authoring domains, RESOURCE2SKILL improves average overall score by +11.9 percentage points over no-skill agents and outperforms strong harness baselines in 26 of 28 main-aggregate model-domain cells. Ablations confirm the value of multimodal skill format, hierarchical organization, source diversity, selection strategy, and online acquisition.
Community
š We are excited to release Resource2Skill.
The most exciting result is not only that skills improve agentsābut that they work remarkably well across a surprisingly broad range of real software environments:
ā¢ PowerPoint
ā¢ Excel
ā¢ Web Generation
ā¢ Blender
ā¢ CAD
ā¢ UI/UX
ā¢ Music Production
To our knowledge, this is the first framework to demonstrate strong and consistent skill-driven gains across such diverse domains.
Across seven domains and four model backends, Resource2Skill improves the average score by +11.9 percentage points over no-skill agents, improves performance in 28/28 modelādomain settings, and outperforms strong agentic harnesses in 26/28 settings.
These domains involve very different knowledge, tools, workflows, and forms of reasoning. The breadth of the results shows that Skill can serve as a general interface for adapting foundation models to new frameworks, domains, and software environments.
The internet already contains enormous amounts of procedural knowledgeāin tutorials, code repositories, documentation, articles, and high-quality examples.
Resource2Skill turns these scattered multimodal resources into a hierarchical Skill Wiki: reusable, inspectable, and executable knowledge that agents can retrieve, compose, and apply.
This suggests a much broader opportunity: many valuable agent skills may already exist implicitly across the internet, waiting to be extracted and organized.
š https://aka.ms/Resource2Skill
š» https://github.com/microsoft/Resource2Skill
š https://arxiv.org/abs/2606.29538
Get this paper in your agent:
hf papers read 2606.29538 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper