• 1 Post
  • 25 Comments
Joined 3 years ago
cake
Cake day: June 16th, 2023

help-circle
  • pech@lemmy.worldtolinuxmemes@lemmy.worldJust Draw
    link
    fedilink
    English
    arrow-up
    1
    ·
    14 days ago

    honestly, systemd isn’t that bad. I’m finishing some free labs for the CompTIA Linux+ sysadmin course and learning how to compose bash scripts into services is going to be great for my homelab.

    I completely ditched windows at home in early 2020 and used fedora until last year when I took the plunge into NixOS. I knew how to do things in Linux, but I wanted to understand the underlying system structure. NixOS helped pull back the veil a bit.






  • That seems like a very balanced approach beyond what I was even considering; I like it.

    The guys channel is full of mad science slapdash stuff to make things work that otherwise, just shouldn’t lol. I take that approach to my homelab (with some caveats, since I’m not trying to lose all my data lol), and just kind of throw things together with some minimal understanding of how they fit conceptually.













  • Currently I’m running a Q6K quant of Hermes 4 14B with a 32K context window via llama.cpp that works pretty well. Generation output is a comfy ~50tok/sec. These v100s are 16GB each, but there are 32GB versions available too.

    I’m running everything via NixOS and have to do package overrides to get inference engines to build with the right CUDA versions.

    My goal is to get a cohesive environment set up for Hermes Agent to learn my system/lab/network and help my grow it over time.

    Overall, I’m happy with them. The mezzanine board is good quality, I’m using PTM sheets under those massive heatsinks and some arctic p9 fans to keep them at around 60C under load.