DeepSeek R1 running locally
A Test Drive with Unsloth’s 1.58-bit Compression
I’ve checked how DeepSeek R1 (not the distilled versions) runs locally on my RTX-4090 (24GB VRAM) with 64GB RAM, using the amazing Unsloth team’s Dynamic 1.58-bit version which achieved an 80% reduction in size on the 671B parameter model. The quantized size is around 131GB.
A fresh build of Llama.cpp was used for inference, and I've offloaded 10 layers into VRAM, with the rest swapping between RAM and virtual memory.
./llama-cli --model DeepSeek-R1-UD-IQ1_S-00001-of-00003.gguf --cache-type-k q4_0 --threads 12 --prio 2 --temp 0.6 --ctx-size 4096 --n-gpu-layers 10As a test, I asked the model to generate a Breakout game.
Although inference was extremely slow on my dev-PC (0.33 tokens/sec), it's a small miracle that Unsloth was able to compress the full model without radically degrading its performance.
An extra 64GB RAM might speed things up a bit, but I'm keeping an eye on Nvidia’s Project Digits, which could arrive in Q2 this year.
DeepSeek R1 has developed a fully functional game with only one typo: it used pygame's key.getPressed function instead of key.get_pressed:
After correcting the problem, the game worked:
For comparison, I also ran the same prompt on the DeepSeek-R1-Distill-Qwen-32B model what is a Qwen 2.5 base distilled with R1. Although it is very fast on my setup, and generated a code without errors, the final product lacks basic functionality: the ball does not move, and displaying the points and lives are missing. The block colors did not follow my instructions either.
Here is the full prompt:
Create a Breakout game in Python. You must include these features:
You must use pygame.
The background color should be randomly chosen and should be a light shade. Start with a light blue color.
The paddle should be at the bottom of the screen, move left and right with the arrow keys, and should have a randomly chosen dark color.
The ball should be a small circle, with a randomly chosen dark color, and should start moving in a random direction.
The bricks should be arranged in multiple rows at the top of the screen, with a fixed number of columns. Each row should have bricks of a different color, chosen from dark green, light brown, or dark gray shades.
The score should be displayed at the top right of the screen and should increase when a brick is broken.
When the ball hits a brick, the brick should disappear, and the ball should bounce.
If the ball falls below the paddle, the player loses a life. Display the remaining lives at the top left.
If all bricks are broken, the player wins and the game restarts.
When you lose all lives, show the best score on the screen.
Pressing Q or Esc will quit the game.
Restarting the game after losing should be possible by pressing SPACE.
What a time to be alive!
