Introducing Qwiet AI AutoFix! Reduce the time to secure code by 95% Read More

The internet has been on fire since the launch of ChatGPT. This AI powered chatbot was released late November and people wasted no time in finding humorous, thought provoking, and potentially dangerous uses for it.

At the core, any AI is only as good as the information and prompts you feed into it and this lead to a lot of discussion amongst the ShiftLeft team as to the potential issues that go along with an AI with what appears to be very few guardrails.

Following is our team’s take on the technology which ranges from ending the college essay to being overly vulnerable to hacks.

1. People Might Feel Nervous Because Language Is the Most Human Thing about Us.

Like with all GPT models, ChatGPT has taken in an astronomical amount of data about our world and has been trained to recall the right pieces of information and stitch them together from relatively little input.

The defining capability of ChatGPT to continue a dialogue comes from the use of Proximal Policy Optimization (PPO) to train the GPT3.5 model. PPO is quite a few years old now, so ChatGPT’s advancement doesn’t come from a new breakthrough at the math level, but rather a breakthrough at the human-computer interaction level.

From a forward-thinking perspective, the ability to maintain a dialogue suggests the ability to monitor a data source in an active manner. If this capability could be expanded to multiple data sources at once, including output of other models or computer systems, we could see ChatGPT or similar serving as an interface between humans and the computer world. A translator of sorts, or perhaps more of an assistant.


2. Confidence in ChatGPT responses in the age of fake news

We think one should treat ChatGPT like a search engine that offers an extremely good interface for humans, so, any responses it gives should be understood as ‘fragments’ of things on the internet related to what we just said.

What we find problematic is that the system speaks more like an expert. It confidently presents answers that are completely wrong in an all-knowing tone. If it also provided pointers to the data that it took to provide its answers, it would help humans fact check the answers and see whether they come from a reliable source.

3. ChatGPT applied to code?

What we have been describing is known as a ‘poisoning attack,’ or ‘adversarial machine learning.’ Frankly, that danger always exists when attackers can influence training data.

Fortunately, secure machine learning can protect training data from such incursions. The Berryville Institute for Machine Learning has an informative paper on the subject, which has been summarized over at Dark Reading.

For our immediate concerns with ChatGPT, the focus should be on designing around risks, just as we do with any system, AI or otherwise.

A potential approach would be to say that a system like that could allow us to map code to hypotheses, but that those are just that: unproven hypotheses. We could then use static analysis to verify these hypotheses. The gain is that we can concentrate the analysis on small regions of code, so, we get around the intense computation required for static analysis, while at the same time fact checking results produced by the AI.

The downside is that their models, transformers and exclusions are unfortunately not shared so we could end up swimming in hypothesis land – in principle, this would be the right thing to do to benefit from a combin