AIBlade Podcast

Article - https://www.aiblade.net/p/4030b68a-2ab6-452e-9a67-530f91a801f9 Notion Free Trial - https://affiliate.notion.so/pqesm7yjddbc AI Jailbreaking has been around since the dawn of consumer-grade LLMs. Defined by Microsoft as “a technique that can cause the failure of guardrails” , jailbreaking still poses a huge problem to LLM providers in 2025, since people can leverage it to easily break ter...

Article - https://www.aiblade.net/p/is-github-copilot-poisoned-part-2 In my previous post, I looked at how code generation models could potentially be poisoned. The impacts could be devastating, and I created a small script to find evidence of this at play. However, my code was too slow, and I didn’t find any meaningful results. In this post, I seek to improve upon my last experiment. I’ll investi...

Article - https://www.aiblade.net/p/a2b8dbe2-ff30-4dd5-9c60-2781f07fea9a DeepSeek AI is taking the world by storm; their new R1 model provides ChatGPT-like capabilities at a fraction of the cost. But how secure really is it? In this post, we’ll take a look at three key areas: the shady origins of DeepSeek AI, a critical vulnerability allowing full database access, and targeted account compromise....

In my last post , I looked at the feasibility of poisoning AI models. While the task would be challenging, the payoff would be huge, allowing threat actors to inject critical vulnerabilities into production codebases. So… have code suggestion models already been poisoned ? In this post, we’ll develop a script to test Copilot for poisoning, evaluate its results, and suggest improvements for future...

Article - https://www.aiblade.net/p/ai-poisoning-is-it-really-a-threat AI Training Data Poisoning is a hot topic, with OWASP citing it as the third most critical security risk faced by LLM Applications. But have these attacks ever occurred, and are they feasible for threat actors to use? In this post, I will scrutinize cutting-edge research and use my cybersecurity knowledge to conclude how impact...

Article - https://www.aiblade.net/p/ai-pentesting-with-vulnhuntr For years, CISOs have been fantasizing about truly automated penetration testing, allowing them to quickly find critical bugs in key applications. While this dream isn’t fully here yet, VulnHuntr offers an LLM-based code analysis package that promises to “find and explain complex, multistep vulnerabilities”. In this post, we’ll look...

Article: https://www.aiblade.net/p/ai-bug-bounty-guide-2024 Bug Bounty has long been an established source of income in the cybersecurity industry. As insecure AI/ML-based applications enter the market in 2024, new bounty programs with low-hanging fruit are opening up. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.aibl...

Article: https://www.aiblade.net/p/claude-computer-use-prompt-injection On 22nd October 2024, Claude Computer Use was released to the world. While Computer Use is an incredible tool, it is also insecure by default. In this blog post, we’ll look at how Johann Rehberger from Embrace The Red was able to completely compromise a Claude-controlled machine via an ingenious Indirect Prompt Injection. This...

Article: https://www.aiblade.net/p/hacking-the-ai-goat The AI Goat is a deliberately vulnerable AI architecture hosted on AWS. Created by Orca Security , it serves as a resource to train the next generation of ethical hackers. In this post, I will hack the Goat, discuss what I like about it, and suggest improvements to make it even better. This is a public episode. If you would like to discuss thi...

After exploiting several Indirect Prompt Injection vulnerabilities, I decided to author my first white paper: The Practical Application of Indirect Prompt Injection Attacks . In this post, I will present my Indirect Prompt Injection Methodology from the paper, discuss the outcomes of my research, and consider its significance in the future of AI Security. This is a public episode. If you would lik...

Generative AI now features in the production environments of several large organizations, yet very little research has been done surrounding its security. Orca Security seeks to change this with their “ 2024 - State of AI Security Report ” . In this post, I will summarize the report’s key findings, analyze their relevance, and consider the future of AI Security. This is a public episode. If you wo...

Chester Wisniewski is the Global Field CTO at Sophos, with a wealth of technical knowledge and over 25 years of experience in the cybersecurity industry. In this episode, we sit down and discuss a range of topics, including: -          Whether ChatGPT was released too soon to the world -          AI Security as the next big security skillset -          If Apple Intelligence will live up to all its...

Article - https://www.aiblade.net/p/chatgpt-delete-my-code AskTheCode is a GPT that allows users to “Provide a GitHub repository URL and ask about any aspect of the code”. With over 100k conversations and 1000 ratings on ChatGPT, software developers widely use this tool to improve their efficiency. …But is it really secure to give an AI access to your codebase? In this post, I will showcase how I...

Article: https://www.aiblade.net/p/how-secure-will-apple-intelligence-be On 10/06/24, Apple announced its long-awaited “Apple Intelligence” to the world. Apple Intelligence is a suite of AI tools integrated into existing functionality to let users “get things done effortlessly” . As always, Apple has gone to great lengths to make this technology high-quality and watertight. But will it be 100% sec...

Article: https://www.aiblade.net/p/chatgpt-send-me-someones-calendar OpenAI recently introduced GPTs to premium users, allowing people to interact with third-party web services via a Large Language Model. But is this safe when AI is so easy to trick ? In this post, I will present my novel research: exploiting a personal assistant GPT, causing it to unwittingly email the contents of someone’s calen...

Article: https://www.aiblade.net/p/how-hugging-face-was-ethically-hacked In this episode, we will look at how security researchers at Wiz were able to achieve Remote Code Execution on Hugging Face and escalate their privileges to read other people’s data. We will examine the consequences of the attack, and then consider countermeasures to prevent it from happening in the future. This is a public e...

Article: https://www.aiblade.net/p/ai-phone-scams Several companies have begun offering free AI phone call services , featuring large language models linked to AI voice generators. The technology is undeniably cool… but the consequences are potentially catastrophic. In this episode, we will look at how AI phone assistants work, demo some real examples of them being exploited to perform social engi...

Article: https://www.aiblade.net/p/backdoors-in-ml New machine learning models are an exciting field to research. Hugging Face is the leader in this space, allowing people to upload and download open source ML projects. At the time of writing, over half a million open source models are available on Hugging Face. But innovative threat actors are using the hype around AI as a guise to hack victim co...

Article: https://www.aiblade.net/p/unjailbreakable-large-language-models Since the beginning of the AI gold rush, people have used large language models for malicious intent . Drug recipes, explicit output, and discriminatory behaviour have all been elicited, with often hilarious results. These techniques are known as “prompt injections” or “jailbreaks” - getting the LLM to perform actions outside...

Article: https://www.aiblade.net/p/how-ai-threatens-critical-infrastructure On April 26th, 2024, the Department of Homeland Security released a 28-page document outlining AI security guidelines for critical infrastructure owners. While it is a step in the right direction, the whitepaper is vague, dry, and unhelpful. In this podcast, I will summarize the paper to save you from reading it, give my t...

Article: https://www.aiblade.net/p/indirect-prompt-injection Since ChatGPT was released in November 2022, big tech has been racing to integrate LLM technology into everything. Music, YouTube videos, and hotel bookings are just a few examples. But as of writing, any LLM which can read data from external sources is inherently insecure. In this article, we will take a deep dive into indirect prompt i...