Experts believe that artificial intelligence will become an invaluable tool for auditing smart contracts and enhancing cybersecurity – but it is not there yet.
Auditing Smart Contracts Using AI
Auditing smart contracts and identifying cybersecurity vulnerabilities is viewed as one of the main uses of artificial intelligence in the future of cryptocurrency. However, there is just one problem – currently, GPT-4 does not perform well in this area.
Coinbase experimented with the capabilities of ChatGPT in automatically reviewing token security earlier this year, and in 25% of cases, it incorrectly classified high-risk tokens as low-risk tokens. James Edwards, the chief supervisor for the cybersecurity investigator Librehash, believes that OpenAI is not interested in having the bot used for such tasks.
“I firmly believe that OpenAI has downplayed some of the bot’s capabilities when it comes to smart contracts in order not to have it explicitly relied upon to draft a deployable smart contract,” he says, indicating that OpenAI may not want to be liable for any vulnerabilities or exploits.
AI Capabilities in Smart Contract Auditing
However, this does not mean that AI lacks capabilities regarding smart contracts. AI Eye spoke with digital artist Rich Mankind in Melbourne last May. He knew nothing about creating smart contracts, but through trial and error and repeated rewriting, he managed to use ChatGPT to create a meme coin called Turbo, which reached a market cap of $100 million.
However, Kang Li, the security director at the blockchain security firm CertiK, points out that while you may be able to get something functional with ChatGPT’s help, it is likely to be riddled with logical code errors and potential exploit vulnerabilities. “You write something, and ChatGPT helps you build it, but due to all these design flaws, it may fail dramatically when attackers start appearing,” he says.
Challenges in Auditing Smart Contracts Using AI
Currently, AI faces challenges in auditing smart contracts, as the training data for GPT-4 is very general and not specialized in smart contracts. According to Richard Ma from the blockchain security firm Quantstamp, “Since ChatGPT is trained on many servers and there is very little data on smart contracts, it performs better in hacking servers rather than smart contracts.”
Therefore, there is now a race to train models using years of smart contract exploitation and hacking data until they learn to recognize them. There are new models that allow you to input your own data, and that is partially what we’ve done,” he says.
Developing AI Models for Auditing Smart Contracts
Edwards is working on a similar project and is nearly finished building the open-source AI model WizardCoder that integrates the Mando Project repository for smart contract vulnerabilities. He also uses the pretrained programming language model CodeBert from Microsoft to help identify problems.
According to Edwards, AI has been able to “audit contracts with unparalleled accuracy far exceeding what can be expected from GPT-4.” A custom dataset has been configured for smart contract exploits to pinpoint vulnerabilities down to the responsible lines of code. The next big step is to train the model to recognize patterns and similarities.
Although he admits it is not yet on the level of a human auditor, it can already perform a strong preliminary review to expedite the auditor’s work and make it more comprehensive. “It helps similarly to what LexisNexis does for lawyers, but more effectively,” he says.
Challenges
Auditing Smart Contracts Using Artificial Intelligence
Near founder, Ilya Polushkin, highlights the fact that exploits of smart contracts are often rare, bizarre cases — a one in a billion opportunity leading to unexpected behavior in a smart contract.
However, Polushkin states that long-memory language models, which rely on predicting the next word, approach the problem from the opposite direction. “Current models try to find the most statistically plausible outcome, right? And when you think of smart contracts or protocol engineering, you should think of all the rare cases,” he explains.
Polushkin mentions that his background in competitive programming means that when Near was focusing on artificial intelligence, the team developed procedures to try to identify these rare cases. “The formal search procedures were about code outcomes. So I don’t think it’s completely impossible, and there are now startups really investing in working with the code and its correctness,” he says.
However, Polushkin does not believe that AI will reach human levels in auditing for “the next few years. It will take a little longer.”
Leave a Reply