+ AeroMogul Module
I have used LLMs in many of their capacities for personal projects or just to test functionality. However, I primarily only use LLMs for what it actually are: search engines that predict what I am looking for. Why? If you’re as old as me, you remember an age when you could actually get the results you were looking for on Google. Some time in the last 10-15 years, they changed their algorithm to keep on the page more, thus breaking the efficiency of search. I believe LLMs have become a better product in this respect. Though, still not perfect.
But you may see me list other uses.
-Language referencing: By language, I am talking about programming languages as well as APIs and low-level hardware concepts. In the past, outside of the most popular languages or well documented APIs, you’d have to dig through manuals, headers, or deal with SEO spam on Google to get the results you were looking for. That said, the more obscure the knowledge, the more likely the LLM is incorrect, which happens quite frequently even today.
-Web search: Mentioned at the top, web searching is horrendous these days. I can type an exact quote from someone on Google, with their user name and the web site name and not get the result. Again, LLMs have many flaws in this area. Sometimes their references are good. Many times they are not or they don’t include any. But even if it’s wrong much of the time, it’s better than Google or Bing.
-Static code analysis: This is where we have a local LLM read through the code and report back any issues that the code may have. The LLM is not allowed to write code, suggest fixes beyond a line or two, or anything else. It’s simply a supplemental, too similar to GCC, Clang, VC++ analysis systems or CPPCheck. As of this writing, we’re at 65% false positives but a 2% real bug rate that the four other analyses fail to catch. Many of those false positives would have broken the game in ways that no one would catch until a few thousand people played the game and noticed things were wrong. So, it’s a dangerous tool in the toolbox but also useful.
-Grammar Check: LLMs are used for spell-check. Originally I used aspell. But then I switched to Grammarly. When Grammarly switched to LLMs their quality declined considerably. Now I am bouncing around several services looking for one that will just tell me that I am using passive voice, so that ~I~ can fix it. I don’t use grammar checking enough, as you probably can tell. I haven't fired my editor yet. But I only use him for really important stuff.
-Debuging: This plays into two things I already mentioned. LLMs are very good at scanning logs and plucking out information like a search engine. They’re also fairly good at analyzing a chunk of code for a specific problem you’re having. And telling you what might be wrong. For me, this is generally a weapon of last resort. I often try to use my local models because they’re much slower, giving me more time to figure out major issues myself. (After creating a prompt, it often becomes clear to me what the issue is.) And because I don’t want to train the LLMs on my work more. One key thing is to not let the LLM generate code to fix bugs. They'll often break many more things than they fix if you let them run free in your code.
We do not use LLM powered code completion. All of our IDEs do not have LLM functionality, with the exception of VSCode for AeroMogul's client. However, we disable all VSCode LLM functionality. Additionally, our Windows machines are heavily internet restricted. Only licensing servers are whitelisted.)
In general, no. We made one attempt at it in the GPT4 era to address seven long standing issues with our Copyfree third party libraries in the Beardgine 2.5 engine. Of the seven, 1 issue was fixed for 2 lines of code. Another issue required an extreme amount of hand holding and manual intervention to fix, which was a net loss of time. Another issue required an extreme amount of hand holding and ended up with the same results as when I attempted to fix it myself a few years previously. The other 4 attempts were failures. I have no further plans to use it on production software after that experience.
Before anyone tells me, "Eric, it's much better now. " I have used and continue to make use of agentic coding for personal software that doesn't leave my intranet. It is not up to my standards for distribution. It's barely up to my standards for home use. But it's software that would never get made otherwise. If I won't put my name on it, I'm not going to sell or distribute it to you.
Before OpenGPT was released, the original plan was to use ethically sourced (aka licensed) generated headshots. The provider of these became obsolete quickly. So, the plan shifted to use frontier model generated headshots on an opt-in basis, either as a DLC or bundled with the game. This would not displace a human made paperdoll style headshot system. However, constant model change, backlash against generative AI, and my views on the matter made me postpone the idea indefinitely.
Currently there are some placeholder generative AI NPC headshots. These will be removed before prototype testing. If generative AI headshots make it into the game, it'll be via a local model when consumer backlash against GenAI dies down. That said, if consumer backlash against GenAI dies down, I'm likely out of business anyway just from the sheer flood of GenAI games.
All art subcontractors are now required to sign an AI Non-Use Agreement. These will be renewed yearly. I will also have some artists make proof of work, which can also be used for marketing materials. I haven't had to hire any programmers yet, but if I did, they would likely have to sign an AI Limited-Use Agreement. Additionally, they'll be limited to LLMs we provide and monitor.
Yes, I’m pretty negative about the future of gaming. Democratization of game development has brought a race to the bottom. Both money and skills required to make a game are getting close to 0. On the surface this seems to be a good thing. But when the supply and demand equilibrium goes too far out of balance in favor of supply, the system fails. Large companies become much more conservative and wall in users. And opportunity costs cull small businesses with high costs. And which small businesses have high costs in the gaming industry? People and studios who make a living without making a hit. This primary leaves hobbyists and the low skilled filling much of the industry. This is likely to create a talent void, and quality will fall as a result. And when quality falls, demand will decrease.
Observation and data already prove this to be true. Observationally, over the last ~15 years we’ve been racing to the bottom. Large companies have become much less willing to take risks. They rely on formulaic, long-standing IP. More recently, they have moved to live services to keep you playing their games forever. Data-wise, New Release sales on Steam are down YoY, and revenue growth is from back catalog sales. Source. We’ll see how well this continues to play out. There has been a 3664% increase in games since GearCity launched in 2014. But only a 50% growth of consumers… And the number of games is increasing exponentially.
I expect niche gaming oversaturation to kill businesses in this space. Including my own. The industry landscape will be filled with fully AI generated games. With the occasional heavily AI-assisted hobbyist developer making a passion game. If they’re able to produce better games than those that they displaced, for consumers it will be a good thing. Maybe some people will have extremely big hits. But as an industry, I’m not seeing a sustainable future in it anymore.
Being human might be the only selling point in the future. We're not far off from the day when Clankers can clone your multi-year work in minutes with a couple prompts and a handful of tokens.
Anyway, gaming is one slice of the pie. I am more worried about the broader economic ramifications as this technology improves.
We used a waveform text to speech system for some tutorials and foreign voice overs. These predate modern LLM based speech synthesis. So, technically it is not an LLM, but it also isn't human.
If the future is prompting an LLM and having it regurgitate stolen copyright and copyleft code, artwork, and data into a bowl of vomit soup, which I then sell to you with a fake smile and say, "But all the design decisions were my own!" Then it's not a future I want to be a part of. I make games because I want to make something. I don't want other people's vomit. Nor do I think you should pay me for my vomit. If you insist on eating vomit. Give it a few weeks, and you could just ask the Clanker to vomit something like my own game into your own bowl for free.