Add working directory note to AMD download script docstring

Add FineWeb download script for AMD/US environment (direct HuggingFace, no mirror)

Add deepseek run_lite and fineweb extract/tokenize scripts

The changes update the download plan script to handle .tar.gz files (for Llama 3 models) in addition to the previously supported .tar files. The code checks if the target filename ends with .tar.gz and adjusts the extraction command accordingly by using tar xfzf format gzip files. This ensures compatibility with Llama 3 models which use the .tar.gz format.

feat(download): track shard progress in progress.json for resumability

refactor(download): hardcode hf-mirror endpoint for China access

chore: add ruff pre-commit hooks and apply formatting

feat(download): add hardcoded 100B-token GPT-3 ablation downloader

feat(download): add token-budgeted FineWeb shard planner/downloader

refactor: reorganize project structure