feiyang-k/curation-bench/alpaca-a-strong-replicable-instruction-following-model-stanford-alpaca-2023

Alpaca: A Strong, Replicable Instruction-Following Model

One-line decision

Use this skill when you want to distill instruction-following capability from a strong LLM to a weaker one using 52K generated instruction samples. Avoid it when you can directly use the strong LLM or have human-written instructions.

Skill metadata

• Skill type: instruction-data-distillation
• Paper kind: operational-method
• Actionability: high
• Evidence quality: full_paper

Goal

Distill instruction-following capability from GPT-3.5 to LLaMA-7B by generating 52K instruction-following samples using the Self-Instruct approach.

Problem signature

• Modality: text instruction-response pairs generated by GPT-3.5.
• Data state: 52K instruction samples generated by GPT-3.5 from 175 seeds.
• Scale regime: 52K instruction-response pairs.
• Model requirement: GPT-3.5 for generation; LLaMA-7B for fine-tuning.

Use when

• You want to create instruction data cheaply using a strong LLM.
• You need to fine-tune a smaller model for instruction following.
• You want a cost-effective alternative to human annotation.

Do not use when

• You can use the strong LLM directly.
• You have abundant human-annotated instruction data.
• You need domain expertise the LLM lacks.

Required inputs

• strong_llm: GPT-3.5 or equivalent for generating instruction data.
• seed_instructions: 175 seed instruction tasks.
• base_model: LLaMA-7B or equivalent for fine-tuning.

Optional inputs

• quality_filter: Filter for removing low-quality generated instructions.

Outputs

• alpaca_data: 52K instruction-response pairs from GPT-3.5.
• alpaca_model: LLaMA-7B fine-tuned on the generated data.

Assumptions and prerequisites

• Strong LLM outputs provide effective supervision for smaller models.
• 52K samples are sufficient for basic instruction following.
• The Self-Instruct approach scales to production quality.

Procedure

1. Generate instructions with GPT-3.5

Action: Use GPT-3.5 to generate 52K instruction-response pairs following Self-Instruct. Why: GPT-3.5 produces reasonable quality instruction data. Note: See paper for details.

1. Quality filter

Action: Remove obviously low-quality or malformed samples. Why: Basic filtering improves data quality. Note: See paper for details.

1. Fine-tune LLaMA-7B

Action: Fine-tune the base LLaMA model on the 52K samples. Why: Creates an instruction-following model. Note: See paper for details.

1. Evaluate instruction following

Action: Test on instruction-following benchmarks. Why: Validates the distillation quality. Note: See paper for details.

Parameters to set

• generation_cost — Role: API cost for generating data. How to set: ~$500 for 52K samples with GPT-3.5. Default/range: ~$500. Effect: Very cost-effective compared to human annotation.
• sample_count — Role: Number of instruction samples. How to set: 52K for basic instruction following. Default/range: 52K. Effect: More samples improve instruction diversity.

Validation checks

• The fine-tuned model should follow diverse instructions.
• Quality should approach that of the teacher model on simple tasks.
• The approach should be reproducible at low cost.

Failure modes

• Generated data inherits biases from GPT-3.5.
• 52K samples may not cover complex instruction types.
• The quality gap between teacher and student may be significant for hard tasks.

Adaptation notes for VLM training

• The Alpaca recipe has been widely replicated for VLM instruction data.
• Replace GPT-3.5 with Claude for data generation.
• Scale to more data for improved instruction following.

Implementation notes

• Use the Alpaca data generation code for reproducibility.
• Monitor per-task quality during fine-tuning.
• Compare against the teacher model on diverse tasks.

Evidence from the paper

• Alpaca creates an instruction-following LLaMA-7B for ~$500 in API costs.
• 52K GPT-3.5-generated samples enable reasonable instruction following.
• The approach demonstrates cost-effective LLM distillation.
• Alpaca has inspired numerous open-source instruction-tuning efforts.

Source paper

• Title: Alpaca: A Strong, Replicable Instruction-Following Model
• Year: 2023
• Venue: Stanford
• Paper ID: stanford-alpaca-2023
• URL: https://crfm.stanford.edu/2023/03/13/alpaca.html
• arXiv ID: N/A