Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
liuliu
38 days ago
|
parent
|
context
|
favorite
| on:
Bonsai 27B: A 27B-Class model that runs on a phone
The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases.
liuliu
38 days ago
[–]
You also need to pay close attention to BFCLv3 multi-turn result, that helps you to get a sense how frequently these quants will be in a doom loop.
h14h
38 days ago
|
parent
[–]
I'm curious what kind of results one could get from combining the clever quantization PrismML is doing here with something like LiquidAI's antidoom:
https://github.com/Liquid4All/antidoom
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search: