HackerTrans
TopNewTrendsCommentsPastAskShowJobs

orost

no profile record

comments

orost
·2 ปีที่แล้ว·discuss
Let me clarify.

Mixtral-8x22B-v0.1 was released a couple days ago. The "mixtral:8x22b" tag on ollama currently refers to it, so it's what you got when you did "ollama run mixtral:8x22b". It's a base model only capable of text completion, not any other tasks, which is why you got a terrible result when you gave it instructions.

Mixtral-8x22B-Instruct-v0.1 is an instruction-following model based on Mixtral-8x22B-v0.1. It was released two hours ago and it's what this post is about.

(The last updated 44 minutes ago refers to the entire "mixtral" collection.)
orost
·2 ปีที่แล้ว·discuss
Considering "mixtral:8x22b" on ollama was last updated yesterday, and Mixtral-8x22B-Instruct-v0.1 (the topic of this post) was released about 2 hours ago, they are not the same model.
orost
·2 ปีที่แล้ว·discuss
That's not the model this post is about. You used the base model, not trained for tasks. (The instruct model is probably not on ollama yet.)
orost
·2 ปีที่แล้ว·discuss
Mistral 7B Instruct v0.2 and Mistral 7B v0.2 are different models. Judging by the title, I suspect OP meant to post about the latter, which was released a few days ago, but accidentally linked to the former instead.
orost
·2 ปีที่แล้ว·discuss
A rocket on a typical orbital launch profile spends less than 60 seconds in air dense enough for jet engines to have good performance, so there is little to gain.

Pegasus is an orbital rocket launched from an aircraft, but it doesn't exactly impress with performance or cost-effectiveness. Just doesn't make much sense to operate a huge aircraft and design your system around it just to improve on the least important 10% of the flight.
orost
·2 ปีที่แล้ว·discuss
An air-breathing jet engine doesn't need to carry oxidizer, which in a rocket is most of the propellant weight. It also has access to unlimited reaction mass, so it can be much more energy-efficient in producing thrust (it is more efficient to produce thrust by accelerating a lot of mass by a little, than by accelerating a little mass by a lot, but a rocket can't take advantage of this because it would need to carry all that extra mass. A plane can use ambient air for this purpose)

This all adds up to a plane needing to carry many times less mass to gain the same altitude and speed as a rocket, at least within relatively dense atmosphere.
orost
·3 ปีที่แล้ว·discuss
You can partially offload with some backends (e.g. llama.cpp and derivatives) but speed gains from that don't come in until it's mostly offloaded. I have 8GB VRAM and it's not enough to get any boost on mixtral in Q8. 16GB might do better or it might not.

The speed is quite good even on CPU only though, I get 3.5 tokens per second with 6 cores and DDR5-6000. For comparison llama2-70B is less than 1 t/s on the same hardware in Q4. And, subjectively, Mixtral performs better.
orost
·3 ปีที่แล้ว·discuss
A reactor that has never been turned on isn't a significant radiation hazard. It's the fission products that are hazardous, not the fuel, if it's never gone critical there are no fission products yet.