Mistral 7B: an efficient open 7B language model

A 7-billion-parameter base language model from Mistral AI that showed a small, well-trained model can compete with much larger ones. It is a popular starting point for fine-tuning.

• Decoder-only transformer with grouped-query attention for faster, lighter decoding
• Sliding-window attention keeps long inputs cheaper than full attention
• Byte-fallback BPE tokenizer, so unusual characters never become unknown tokens
• Base model with no built-in moderation; add your own safety layer for production

Summary written by SolxLab. Weights and code belong to their authors and are distributed under the license shown; always check the official page before use.