Research scientist Andrey Barbu gives his opinion on the LLM design

Today I heard from MIT research scientist Andrei Barbu about his work at the LLM and how to prevent certain types of problems related to data leaks.

“We are particularly interested in studying language,” he said of his research, noting some of the team’s goals in this area.

Thinking about the duality of human and computer cognition, he pointed out some differences. For example, he said, humans teach each other. Another thing humans can do is keep secrets, which may be difficult for digital humans.

“The problem is with LLM programs,” he said, explaining the potential for leaks. “They can’t keep secrets.”

In explaining how to identify the problem, Barbu cited prompt injection attacks as a prime example, something similar to what we heard shortly before from Adam Chipala, who mentioned the verifiable software principle as a potential solution.

But Barbu pointed out something that I found interesting, which is kind of a dilemma for systems that aren’t very good at containing data.

“A model is only as sensitive as the most sensitive data stored within it,” he explained. “Humans can inspect the model. … (But) the model is only as vulnerable to attack as the least sensitive data stored within it.”

In some cases, he suggested, people could contaminate models quite easily.

Solution: Barbu explained their customized model with some tweaks and how it works.

Specifically, he was talking about something called Low Rank Adaptation, or LORA, a fine-tuning method developed at Microsoft in 2021.

Upon investigation, experts point out two things that LORA does uniquely: it tracks weight changes rather than updating them directly, and it separates parameters by splitting a large matrix of weight changes into smaller matrices.

In discussing some of the techniques involved, Barbu spoke about extracting what you need from a library of components and explained that there are many ways to approach this. He used a Venn diagram to illustrate the differences between, for example, adaptive and selective techniques.

He suggested that strategies such as translating English into SQL could be considered to solve the problem and expedite the solution. But at the end of the day, the challenge remains.

“Security is a binary thing,” he noted. “You either succeed or you fail.”

Now, this part of the presentation caught my attention. Barbu talked about some AI tools that could significantly reduce the effort of information security. He described a situation where the tools would sit on the network, look for sensitive information, and actually provide input.

This, he suggested, could solve some of the pervasive HIPAA issues surrounding the leaking of protected health data.

“Right now, we don’t have a good solution for this,” he said. But with the right tools, maybe it can be done!

Another good idea I heard from his talk was to label “informed puzzles” and “uninformed puzzles” to see where the problems lie.

This would also improve LLM’s ability to maintain confidentiality, he said.

“We can build a secure LLM,” he added. “We can build a model that is completely resistant to any kind of attack by just not connecting any parameters that the user should have access to. So there is literally nothing the user can do…”

There was a lot more to the demo, including a very detailed, but very brief, example from Barbu of a scenario where data about alien landings is stored in a database. This was also worth noting, and you’ll need to tune into the video to see it all in detail. However, we’ll continue covering the highlights from the conference in near real time.