v2.0.4: SynapseAI v1.17.0
SynapseAI v1.17.0
The codebase is validated with SynapseAI 1.17.0 and optimum-habana 1.13.1.
Tested models and configurations
| Model | BF16 | FP8 | Single Card | Multi-Cards |
|---|---|---|---|---|
| Llama2-7B | ✔ | ✔ | ✔ | ✔ |
| Llama2-70B | ✔ | ✔ | ✔ | |
| Llama3-8B | ✔ | ✔ | ✔ | ✔ |
| Llama3-70B | ✔ | ✔ | ✔ | |
| Llama3.1-8B | ✔ | ✔ | ✔ | ✔ |
| Llama3.1-70B | ✔ | ✔ | ✔ | |
| CodeLlama-13B | ✔ | ✔ | ✔ | |
| Mixtral-8x7B | ✔ | ✔ | ✔ | ✔ |
| Mistral-7B | ✔ | ✔ | ✔ | ✔ |
| Llava-v1.6-Mistral-7B | ✔ | ✔ | ✔ |
Highlights
- Added support for vision-language models
Full Changelog: v2.0.1...v2.0.4