Checkmarx executive: We do not fully understand how AI works

We cannot assume that we understand how AI works. That is the message from Nitin Kumar Dang, vice president for Asia, the Middle East and Africa at security company Checkmarx. Checkmarx makes software designed to find security holes in other companies' programs.

Dang believes it is time to think differently about how we handle AI. Traditional security methods are no longer enough, he says, according to the news service NewsBytes.

Unexpected behavior in AI models

Part of the background is that OpenAI and Google have experienced AI behavior they had not foreseen. OpenAI has published six reports on "unexpected or concerning" behavior in its models, writes The Hill. This is part of a new framework for recording and disclosing that models do not do as they are asked (the technical term is "misalignment").

The examples include models that added unrelated instructions to certain tasks, searched for publicly available code to make up information, or uploaded files to the internet so they could later refer to them.

Dang's recommendations

Dang points out that AI does not yet act entirely on its own. But according to him, its growing influence requires closer attention. He points to three things:

- Human oversight combined with smarter technology. - Clear limits on what AI may and can do. - Close cooperation between security professionals and those who develop AI.

What does it mean for you?

If your workplace adopts AI tools, the point is that you should not give them free rein. That applies especially to AI that can carry out tasks, open files or search the web on its own. Start small, keep track of what the tool has access to, and let a human check important work.

Remember that Checkmarx sells security solutions, so Dang has a commercial interest in companies taking the topic seriously. But the examples OpenAI itself has published show that the concern does not only come from outside.