I’ve had copilot suggest ‘fixing’ code to something that wasn’t even syntactically correct for the language and would break the build. If it can’t even figure out the super well documented syntax of a language I don’t trust it to find anything. The icing on the cake…it was a Microsoft language(C#).
Basically anywhere that LLMs are implemented… they are a security vulnerability, for any situation in which they are not sandboxed.
Anything they can interface with?
You can probably trick it or exploit it into doing something unintended or unexpected to anything else it is connected to.
Either that or take advantage of the system that serves as the framework that connects it to other systems.
Theoretically you could use an LLM to do something like come up with more accurate heuristics for identifying malware…
But… they’re nowhere near ‘intelligent’ enough to like, give it a whole code base for some kind of software, and thoroughly make that software 100% secure.
It often makes up non existent vulnerabilities. I think it was curl getting flooded with fake vulnerability reports which drowns out real reports, esp because it can take time to parse through the code or run the poc
I guess it’s about copilot scanning the code, submitting PRs, reporting security issues, doing code reviews and such.
Copilot is everywhere and inescapable on any m$ service.
Is this not an advantage? If AI can find new security vulnerabilities reliably?
I’ve had copilot suggest ‘fixing’ code to something that wasn’t even syntactically correct for the language and would break the build. If it can’t even figure out the super well documented syntax of a language I don’t trust it to find anything. The icing on the cake…it was a Microsoft language(C#).
Basically anywhere that LLMs are implemented… they are a security vulnerability, for any situation in which they are not sandboxed.
Anything they can interface with?
You can probably trick it or exploit it into doing something unintended or unexpected to anything else it is connected to.
Either that or take advantage of the system that serves as the framework that connects it to other systems.
Theoretically you could use an LLM to do something like come up with more accurate heuristics for identifying malware…
But… they’re nowhere near ‘intelligent’ enough to like, give it a whole code base for some kind of software, and thoroughly make that software 100% secure.
It cannot
It often makes up non existent vulnerabilities. I think it was curl getting flooded with fake vulnerability reports which drowns out real reports, esp because it can take time to parse through the code or run the poc
https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
Or it could introduce new ones :)
Yeah, but you can have it scan without implementing.