Within these pages are recorded my attempts to wield the highest arcane art and conjure minds that play the game of CodeCraft.
Continue readingConjuring a CodeCraft Mind
Leave a reply
Author Archives: clemenswinter
My Reinforcement Learning Learnings
I spent a good chunk of my time over the last two years applying deep reinforcement learning techniques to create an AI that can play the CodeCraft real-time strategy game. My primary motivation was to learn how to tackle nontrivial problems with machine learning and become proficient with modern auto-differentiation frameworks. Thousands of experiment runs and hundreds of commits later I have much to learn still but like to think that I have picked up a trick or two. This blogpost gives an overview of the workflows and intuitions I adopted over the course of working on CodeCraft in the hope that they will prove useful to anyone else looking to pursue similar work. For a very different take on the same material that provides motivating examples for many of the ideas summarized here, check out my dark fantasy machine learning poem “Conjuring a CodeCraft Mind“.
Continue reading
Mastering Real-Time Strategy Games with Deep Reinforcement Learning: Mere Mortal Edition
The capabilities of game-playing AIs have grown rapidly over the last few years. This trend has culminated in the defeat of top human players in the complex real-time strategy (RTS) games of DoTA 2 [1] and StarCraft II [2] in 2019. Alas, the exorbitant engineering and compute resources employed by these projects has made their replication difficult. As a result, the application of deep reinforcement learning methods to RTS games has remained disappointingly rare. In an attempt to remedy this sad state of affairs, this article demonstrates how you can use deep reinforcement learning to train your very own sociopaths for a nontrivial RTS game within hours on a single GPU. We achieve this by employing an array of techniques that includes a novel form of automatic domain randomization, curricula, canonicalization of spatial features, an omniscient value function, and a network architecture designed to encode task-specific invariants.
Continue reading
Removing Algae from my Water Loop
This post documents the eradication of algae from my water cooled 4 GPU build.
Continue reading
Building the Ultimate Deep Learning Workstation
This blogpost documents the addition of a full custom water loop and two GTX 2080 Ti GPUs to my previous build. This work was done back in November/December of 2019 but didn’t get around to doing the write up until now.
Continue reading
32 Core Threadripper + Dual 2080 Ti Build Log
This post documents my recent desktop PC build featuring a 32 core threadripper processor and two 2080 Ti graphics cards. It is specced to allow for another upgrade to four 2080 Ti graphics cards with a full custom water loop. I plan to use it mostly for machine learning and simulating reinforcement learning environments. As an added bonus, it can also run Crysis at 20FPS.
How to Read 100s of Millions of Records per Second from a Single Disk
This article gives an overview of the implementation and performance of two recent additions to LocustDB, an extremely fast open-source analytics database built in Rust. The first addition is support for persistent storage, the second is an lz4 compression pass for data stored on disk or cached in memory. Benchmarking LocustDB on a dataset of 1.46 billion taxi rides demonstrates excellent performance for cold queries, reaching > 95% of sequential read speed on SSD and > 70% of sequential read speed on HDD. Comparing to ClickHouse, queries with similar data volume are as fast or faster, and disk space usage is reduced by 40%.
How to Analyze Billions of Records per Second on a Single Desktop PC
Introduction
This article gives an overview of LocustDB [1], a new and extremely fast open-source analytics database built in Rust. Part 1 gives some background on analytical query systems and my goals for LocustDB. Part 2 presents benchmark results produced on a data set of 1.46 billion taxi rides and demonstrates median speedups of 3.6x over ClickHouse and 2.1x over tuned kdb+. Part 3 is an architecture deep dive that follows two SQL statements through the guts of the query engine. Part 4 concludes with random learnings, thoughts on Rust, and other incoherent ramblings. Continue reading
New roadmap after finishing multiplayer
I now have a working implementation for a multiplayer server that allows for running a game across separate machines that can run both the JavaScript and the JVM version (link to demo). It’s too slow to be really usable yet, but all the core algorithms are in place. Multiplayer was the last difficult technical problems that I wanted to tackle, so from now on I intend to focus mostly on improving existing features and getting the word out about CodeCraft. For this post, I have compiled an overview of what I want to do on the technical side of things over the next few weeks. If you have any feedback or suggestions, let me know in the comments or on twitter. Continue reading
Multiplayer
Ever since I began working on CodeCraft, I wanted to implement a multiplayer mode which would allow players anywhere in the world to pit their AIs against each other. With most other parts of CodeCraft taking shape, I now spent some time working on that aspect of the game. In this post I want to talk about some of the design challenges and outline my plans for multiplayer. Continue reading