Am I Still a Hacker if I Use an LLM?
by Jeff Barron (jeffbarron@protonmail.com)
"Yes, I am a criminal. My crime is that of curiosity. My crime is that of judging people by what they say and think, not what they look like." - The Mentor, "The Conscience of a Hacker"
When I was 12, I built a Red Box and used it to call my friends on payphones. It was the only way I could talk with them and my parents moved around a lot. A Red Box could be created with an electronic tone dialer that was easily accessible at an electronics store called RadioShack. You could modify the tone dialer by soldering a crystal into it. This would make it so that when you pressed #, it would emit a coin tone for a nickel. The payphone would believe you had dropped a nickel into it. Just press it five times and call whoever you want. This was illegal, but it was the only way I had to stay connected with the few friends I had. It was the tool I used with the resources I had available to me. That was a long time ago.
I work on the offensive security team of a very large corporation. I've recently gotten into Maldev and I've been learning the Nim programming language. I got my start with Nim with the wonderful OffensiveNim project by byt3bl33d3r. My first project is bazzy, a shellcode loader.
The initial version takes a payload generated by MSFvenom that pops calc. I encode it into Base64 and embed it in the loader, and it executes without detection on the latest Windows Pro version as of this writing. I tried it with a reverse shell and that also glides right by Defender. I added the code to GitHub and wrote a lazy README, making sure I credited OffensiveNim.
The README looked terrible and then I had the idea that I could let a Large Language Model (LLM) write it. I copy pasted my source code into it and told it to generate a README.md. It did a great job. I did have to reword a couple of things, but it was really easy. But it left me with two questions: Did I need to put a citation in for the LLM and am I a noob and a phony for using it?
I think it's O.K. to use it for generating a README. But would it be O.K. for writing this article? I think that answer is hell no. I don't want to read LLM-generated content and I don't think I'm alone. We get a lot of it. I think it lacks authenticity and that's super important for both the writer and the reader. LLM-generated writing sucks.
I've been reading a lot of resumes recently, and many of them have quite obviously been generated by an LLM. It makes them very tedious to read and everyone is doing it. A resume is similar to a README in that I don't expect original thinking and voice in either. So I feel like it's totally fine to use an LLM for this. But it does make the resume a boring read.
Since I was able to get my shellcode loader working, I decided to try to implement my own shell in Nim. It would be a good first step to writing a beacon in Nim for a C2. The problem was I didn't really know where to start, so I asked the LLM. I was using Claude and I had to constantly reassure it that I was a security professional, so I switched over to ChatGPT.
It generated a simple script that used sockets to connect and PowerShell to execute commands. I told the LLM that we can't just use a socket for our C2 since anything other than HTTPS would likely arouse suspicion in the network security logs. I also told the LLM that PowerShell wasn't good enough that we'd need to implement the functionality that we needed from the OS package of Nim. The LLM made the changes and sent me the code and this is where the debugging began. It didn't work. I started copying and pasting error messages into it and copying back the "fixed" code. I turned on some drum and bass in my earbuds and spent the next two hours going back and forth with the LLM.
After the two hours, we finally had a 200-lines-of-code monstrosity with plenty of debugging information. It still didn't work. I looked at the code and noticed a small issue that the LLM wouldn't spot: a typo in the IP address. I changed the IP and it worked. Am I a noob and a phony for using an LLM to help me code? This one feels different than the README. Am I still a hacker if I use an LLM?
Google search is dead. It was a great tool to find things, but LLMs can retrieve that information so much faster. I never once felt like less than a hacker for using a search engine. What is it that gives me pause when I use an LLM to help me code that doesn't when I use a search engine to find exploits or regex? Is it too easy? When the LLM generates the code, there isn't some poor coder with a two liter of Mountain Dew celebrating the success at 3 am. The sense of accomplishment is not there, but I think the only thing that matters for code is that it works.
When I was a kid, I used a modded tone dialer, but that didn't make me a hacker. Solving problems with the tools that I had made me a hacker. I think using an LLM to help you code is valid, although possibly script-kiddie territory depending on how you use it. It's a tool like any other. Hacking has always been about using and abusing available tools whether that's a Red Box or an LLM.
References
The Mentor. The Conscience of a Hacker, Phrack, Issue #7