Personally, I’d really like the option of running LLMs locally, but the hardware requirements make it hard. Small models run okay on CPU or low-end GPUs, but anything approaching the complexity and usefulness of GPT4 or DeepSeek requires a hefty GPU setup. Considering how much even old hardware like the P40 has gone up in price, it’s hard to justify the cost.
Personally, I’d really like the option of running LLMs locally, but the hardware requirements make it hard. Small models run okay on CPU or low-end GPUs, but anything approaching the complexity and usefulness of GPT4 or DeepSeek requires a hefty GPU setup. Considering how much even old hardware like the P40 has gone up in price, it’s hard to justify the cost.