• 0 Posts
  • 12 Comments
Joined před 2 roky
cake
Cake day: 8. července 2023

help-circle
  • Yes. But you won’t be prosecuted, if you simply won’t spread propaganda. They could’ve refuse posting this article and would’ve been okay anyway.

    I recall there where a story, from nazi germany soldier side, where it was explained how no one really was forced to do all that nazi stuff in the first place and like, officers just commanded “who want to rape&kill locals — go ahead, rest can stay, no one will be prosecuted”. And like, almost no one stayed. Similar story happens in russia currently, especially in army, but in civil life too. No one forcing russians to hate Ukrainians and yet my grandparents said something across the lines “we should gas them”.

    So yeah, nah.







  • is there a general term for the setting that offloads the model into RAM? I’d love to be able to load larger models.

    Ollama does that by default, but prioritizes gpu above regular ram and cpu. In fact, it’s other feature that often doesn’t work, cause they can’t fix the damn bug that we reported a year ago - mmap. That feature allows you to load and use model directly from disk (alto, incredibly slow, but allows to run something like deepseek that weight ~700gb with at least 1-3 token\s).

    num_gpu allows you to specify how much to load into GPU vram, the rest will be swapped to regular RAM.