rwkv 0.4b is actually fast :) please use https://github.com/BlinkDL/Albatross
BlinkDL PRO
BlinkDL
AI & ML interests
RWKV is all you need
Recent Activity
updated a Space about 3 hours ago
BlinkDL/RWKV-Gradio-4 updated a model about 4 hours ago
BlinkDL/temp-latest-training-models updated a Space 5 days ago
BlinkDL/RWKV-Gradio-1Organizations
posted an update over 1 year ago
Post
14813
RWKV-7 "Goose" 0.4B trained w/ ctx4k automatically extrapolates to ctx32k+, and perfectly solves NIAH ctx16k ๐คฏ 100% RNN and attention-free. Only trained on the Pile. No finetuning. Replicable training runs. tested by our community: https://github.com/Jellyfish042/LongMamba
posted an update over 1 year ago
Post
13842
RWKV-6-world-v3 (+3.1T tokens) is our best multilingual 7B model as of now: BlinkDL/rwkv-6-world
It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).
RWKV-7-world-v4 soon :)
It's 100% RNN and attention-free. MMLU 54.2% (previous world-v2.1 = 47.9%. note: without eval-boosting tricks such as annealing).
RWKV-7-world-v4 soon :)
- 1 reply
posted an update almost 2 years ago
Post
5870
RWKV-7 "Goose" preview rc2 => Peak RNN architecture?๐Will try to squeeze more performance for the final release. Preview code & model: https://github.com/BlinkDL/RWKV-LM/tree/main/RWKV-v7
- 2 replies