🎙️Podcast Review: 20VC Contextual AI

Enterprise adoption of Generative AI is still early days

Overview

Harry speaks w/Douwe Kiela (Founder of contextual.ai) about large-language models (LLMs) and generative AI. They discuss that we are still in the early days of generative AI even though there is a ton of hype in the AI community. There is a huge opportunity for generative AI in the enterprise, however, there are outstanding questions we don’t have the answers to yet that will limit adoption. There are concerns about the role regulations will play in the widespread adoption of generative AI. These systems are achieving the capabilities to do human tasks and will create economic displacement in the next 5-10 years.

Highly recommended for anyone interested in the enterprise market for generative AI https://www.thetwentyminutevc.com/douwe-kiela/

Contextual AI Value Proposition

Contextual AI is decoupling the retrieval from the generative aspects of LLMs. They are targeting B2B AI use cases with contextual language models. Contextual AI differentiates from other “frontier”, general-purpose LLMs (e.g. ChatGPT and Anthropic) by introducing something called retrieval augmented generation. Their goal is to create a cleaner and more loosely coupled architecture that enables companies to adopt generative AI on their terms. Key points on contextual’s architecture:

• Retrieval augmented generation decouples the memory of the LLM from the LLM generation to add and remove attributions on the fly. This enables organizations to train the model and rely on what it finds leading to stronger attribution

• This “grounds” the generations on what has been retrieved, making it more appealing to enterprises for use cases that require greater predictability 

• Their LLM architecture calls for a cleaner separation of the “data plane” from the “model plane” to improve organization concerns and controls on data privacy 

LLM Considerations

There is an argument that “hallucinations” are a feature, not a bug. However, Douwe says there is a spectrum of generated output from LLMs from “grounded” vs. “hallucinations”. For enterprise use cases, the “grounded” side of the spectrum is more appealing. For inspirational and imaginative use cases, the “hallucination” side of the spectrum might be more appealing. Key points on LLMs:

• Douwe argues that the size of the data matters more than the size of the model

• Their research has shown that training a smaller model on more data over a longer period yields better results than using a large model with lots of parameters

• If you have a huge model running on 1000s of GPUs with limited data, the outcome is going to be ok. 

• The LLaMA model was trained on open data. An emerging GPT-4 use case is to use it to generate data for use with cheaper models (e.g. OpenLLaMA)

• Building GPT-4 consists of 3 basic steps: 

  • Starting with pre-trained data

  • Creating the core model

  • Reinforced learning from human feedback, supervised fine-tuning of the model

Generative AI Market

A Google researcher has said OpenAI has “no moat”, but Douwe argues this is incorrect. OpenAI has a giant moat because generative AI is all about the data. OpenAI has a deep understanding of how to use language models, and giant economies of scale, and can serve up models cheaply because of demand. 

LLM Evaluation

The reality is that we don’t know how to evaluate LLMs. There is a huge market opportunity for LLM evaluations and benchmarks. Nothing exists that describes how to evaluate the quality of these LLMs. For instance, GTP-4 looks amazing, but it’s only trained on things it’s being evaluated on, it is not appropriate for the model to evaluate the quality of itself.

• Stanford HELM project is an example of providing a holistic evaluation of LLMs, however, it is too static

• Researchers need to evaluate models by trying to break them

Enterprise adoption

Enterprise adoption is already happening. Adoption is taking place in the enterprise but in small R&D pockets. There will be a tidal wave next year, however, there are lots of hurdles to overcome around data privacy and security. Data contamination and quality are big issues. There are no good tools for data contamination and health checking of models. This is required for enterprise adoption. There are three categories of models, enterprises will gravitate towards mid-size:

• Frontier models: GPT-4 and Anthropic, strong in Artificial General Intelligence (AGI)

• Mid-size models: Less expensive, Artificial specialized intelligence. Most likely what enterprises will use

• bottom tier models: low barrier to entry, anyone can use, Meta gave away OpenLLaMA

Deployment Trade-offs

The architecture and deployment of your generative AI setup require trade-offs to determine where the model plane goes. For instance, if you put it in the VPC, you have privacy but no feedback, and learning is limited. Contextual is solving for this by decoupling retrieval and generative aspects of AI models.

Regulations

Regulations are becoming more prevalent. There are concerns about the existential threat of AI. There is a non-zero chance that AI takes over the world. For example, the paperclip maximizer scenario: Give an intelligent system a command to create paper clips and it will turn the entire universe into paper clips. However, there is a very slim chance this happens. The people pushing to regulate AI are the same ones that will benefit from it. EU will try to over-regulate AI and kill innovation, hope this doesn’t happen in the US. The only ones who benefit from over-regulation are the big incumbents, it potentially crushes startups. 

Conclusion

While enterprise companies are tinkering with generative AI and there are pockets of production deployments, this is a very early market. There are barriers around data privacy, security, compliance, and separation of concerns that still need to be addressed. Companies are resistant to moving highly sensitive data (e.g. transactions) into a model that lives outside of their four walls, and this is where the architecture and deployment models of generative AI need to be reimagined for enterprise use cases and requirements.

My Ask

Thank you for reading this article. I would be very grateful if you complete one or all of the three actions below. Thank you!

1. Like this article by using the ♥︎ button at the top or bottom of this page.

2. Share this article with others.

3. Subscribe to the elastic tier newsletter! *note* please check your junk mail if you cannot find it