In a dynamic conversation with Harry Stebbings on "20vc," Alex Lebrac, co-founder and CEO of Nabla, delves into the AI landscape, sharing insights from his extensive experience, including leading engineering at Facebook AI Research and founding AI-driven companies like Wit AI and Virtuos. Lebrac critiques new European AI regulations, deeming them impractical and potentially stifling innovation by making current AI training methods illegal. He emphasizes the importance of timing and involvement of practitioners in regulation. Lebrac also envisions AI transforming healthcare by providing AI assistants to clinicians, improving efficiency, and patient care. However, he notes the challenges in healthcare startups, particularly around who pays for innovation. Lebrac predicts that in ten years, AI will play a significant role in high-level decision-making in healthcare and expresses his ambition to build a comprehensive, data-driven healthcare system.
"So the new regulation is a disaster. It means 100% of the LMS that were trained these last three years would be illegal in Europe."
The quote from Speaker A emphasizes the perceived negative impact of new European regulations on the legality of recently trained language model systems, highlighting a disconnect between regulatory intent and practical implications for AI development.
"Welcome back to 20 vc with me, Harry Stebings, and today we continue the deep dive into the world of AI."
Harry Stebings opens the podcast, setting the stage for a conversation about AI advancements and introducing his guest, Alice Labrac, who has a significant background in AI and startups.
"But before we dive into the show today, this episode is brought to you by Tigas, the Goto research destination for bold investing."
Speaker C introduces the sponsors of the episode, detailing their services and how they contribute to the fields of investment research, security and privacy compliance, and global HR management.
"So, 22 years ago, I fell in love with a chatbot. Her name was Sibel."
Alice shares the initial inspiration behind his career in AI, recounting his fascination with chatbots and how it led to his first company.
"When we started, I thought chatbots would become very, very intelligent after three, four years."
This quote reflects Alice's early expectations about the rapid advancement of AI and chatbots, and his subsequent realization of the complexities involved in achieving such intelligence.
"I was sitting at Terrace at Facebook headquarters in Menlo park."
Alice describes the moment that led to the founding of Nabla, influenced by his experiences at Facebook and a desire to return to the entrepreneurial arena.
"So first thing that amazed me when I arrived at Facebook is I thought all big companies were slow."
Alice shares his initial surprise at Facebook's efficiency and the lessons he learned about operating effectively within a large, fast-moving organization.
"One thing I like, he spent not a lot of time outside the company."
Alice discusses Zuckerberg's focused and data-driven approach to meetings, which he found to be a valuable lesson in leadership and decision-making.
"It's not getting easier. You learn some lessons, you don't do the same mistakes again, hopefully, but get new dangers, new issues."
Alice reflects on the continuous challenges of entrepreneurship, highlighting that while some lessons are learned, new obstacles always arise.
"The solution first is, I think, is with external people, even my investors, who are too close to me."
Alice outlines his strategy for avoiding the entourage trap, which involves seeking candid feedback from external sources and fostering a culture of challenge within his team.
"For the general public, it looks like a very big step function with huge advancements every ten years."
Alice comments on the discrepancy between the AI community's view of gradual progress and the public's perception of sudden breakthroughs in AI technology.
"From our standpoint, it's really ridiculous."
The quote captures Alice's view on the cyclical and often exaggerated excitement that venture capitalists exhibit towards AI, based on his extensive experience in the field.
"Today, I don't think it's fair."
Alice argues against the notion that generative AI applications are merely superficial layers on top of foundational models, suggesting that there is significant value and complexity in developing these applications.## LLM Output and Hallucination
"I mean, by construction, the LLM has to output something, and so if there is nothing to say, it'll make something up that looks natural. So this is probably what Emad meant by design."
This quote explains that the nature of LLMs is to produce content regardless of the input's substance, which can result in fabricated or nonsensical responses. This is a fundamental aspect of how LLMs are constructed and relates to their design as per Emad's comments.
"Absolutely agree with that. Will you drive the car you drive today in ten years? I don't think so."
This quote emphasizes the speaker's agreement with the idea that the rapid pace of progress in LLMs will render current models obsolete within a year, similar to how cars are replaced over a decade.
"I think this is always one train late. So having a lot of proprietary data was very, very important for the last cycle, five years ago, maybe it's less and less true."
This quote suggests that while proprietary data was once essential for AI development, its importance is diminishing as new methods allow for effective training with less data.
"So a large language model is trained in two phases. First phase is unsupervised pretraining... And then the second phase is fine tuning..."
This quote describes the two-stage training process for LLMs, with the initial phase focusing on general language comprehension and the second phase refining the model's responses to specific tasks or instructions.
"This lima paper shows that with only 1000 question and answer examples, they get something that performs better than GPT-3 and almost at the level of GPT-4 with only 1000 QA for fine tuning."
The quote highlights a recent study indicating that high-quality fine-tuning can be achieved with considerably smaller datasets, challenging the notion that vast amounts of data are always necessary for effective model training.
"We probably have most of the existing companies, maybe a few will be started."
This quote acknowledges the possibility of new companies entering the foundational model space, though it suggests that many current players are already established.
"So incumbents have a huge advantage through distribution... But I think the best incumbents can benefit from AI to be competitive in their existing markets, but I don't think they will invent totally disruptive things."
The quote points out that established companies can leverage AI to improve their market position, but they are unlikely to pioneer completely disruptive AI applications, which could be an opportunity for startups.
"So obviously I think the foundational model that will win will be open."
This quote reflects the speaker's belief in the dominance of open AI models, although they note that "open" does not guarantee complete clarity into the model's functioning.
"I'm not sure about national data sets... But even if you feed an LLM with curated data, it's not guaranteed that the output will be good, will be perfect, that you can trust the output."
This quote expresses doubt regarding the utility of national data sets for AI and emphasizes that curated inputs do not ensure accurate or trustworthy outputs from LLMs.
"I fully agree with Janin. I think if you really understand how machine learning works and you're not looking for free publicity, I don't see why you would say something like that."
The speaker concurs with Jan's dismissal of the notion that AI's increasing intelligence would lead to a desire to dominate, attributing such claims to a lack of understanding or a desire for attention.
"It's weird. At the same time, he said that I know he was trying to build a team to compete with OpenAI."
This quote highlights the perceived inconsistency between public statements advocating for a halt in AI development and private actions that suggest a desire to advance in the field.
"Finally, it's here. When I see people using Chat GPT and learning to do their job differently with the help of Chat GPT, I really feel we are on the verge of finally having a huge impact with chatbots."
The quote expresses the speaker's view that we are on the cusp of a transformative period in AI adoption, particularly with the integration of tools like Chat GPT into various professions.
"AI will not replace doctors, but doctors who use AI would replace doctors who don't."
This quote suggests that AI's role in healthcare will be augmentative rather than substitutive, providing a competitive edge to doctors who leverage AI technologies.## Burnout Symptoms and Administrative Work
The biggest work they have to do is clinical documentation.
This quote highlights the primary source of administrative workload for healthcare providers, emphasizing the time-consuming nature of clinical documentation.
You need to have a very solid file with all that. Otherwise the insurance will take the first opportunity, the first pretext not to pay your claim.
This quote explains the financial necessity of thorough documentation for insurance reimbursement purposes in the U.S. healthcare system.
Your best protection is to document this in the right way.
This quote underscores the importance of proper documentation as a legal safeguard against medical malpractice lawsuits.
It will bring an AI assistant to every doctor, to every clinician.
This quote describes the envisioned role of AI in healthcare, where it serves as a support tool for clinicians, streamlining their workflow.
We don't store it, we just capture it and then drop it.
The quote clarifies that while AI assistants would capture audio for context, they would not retain this data, addressing privacy concerns.
They have an API, it's easy to integrate.
This quote indicates that despite their outdated nature, modern EHR systems are designed to allow integration with other applications, like AI tools, via APIs.
We are missing 18 million clinicians by 2030.
This quote from the World Health Organization emphasizes the dire shortage of healthcare workers, suggesting that AI could help bridge this gap.
The health systems are collapsing everywhere.
The quote reflects the global crisis in healthcare, with systems under strain and the need for innovative solutions like AI becoming more urgent.
We are trying to go too fast to a patient facing perfect system.
This quote identifies a common mistake made by startups, which is attempting to revolutionize patient care without first addressing the needs and processes of clinicians.
It would be so easy to do with what we do at Nabla and what we know how to do with AI today.
This quote suggests that current AI technology is capable of significantly improving the documentation process in emergency services.
Who is paying for the product?
This quote emphasizes the importance of understanding the payment structure in healthcare, which is critical for a startup's success and scalability.
It's impossible to go to market with something too ambitious.
The quote advises against overly ambitious projects that do not align with the current healthcare system's structure, as they are likely to fail in the market.
It used to be true, I think that you needed to be in the valet with having these kind of people around you. Now I think it's less true because everything is distributed.
This quote reflects the changing landscape of AI talent distribution, suggesting that location is less critical than it once was for success in AI.
We produce lots of good engineers, but we
This incomplete quote hints at France's ability to produce skilled engineers, but suggests that there may be challenges in leveraging this talent to its full potential within the startup ecosystem.## Company Growth Challenges
"We are not bad at that. I don't know why. Maybe we lack discipline. Maybe we." "So if you are normal, it's hard to refuse a $1 billion offer if you haven't done 10 million before." "Hopefully it's changing. We have very good scale ups in France now, but it takes some time too."
These quotes highlight the speaker's self-reflection on why they may not excel at growing companies, the temptation to accept large buyout offers, and the evolving startup ecosystem in Europe.
"So Europe is probably ten years relates compared to us and China." "So the new regulation is a disaster." "If the regulation, if it's really like this and nobody can challenge it, startups like us may have to move."
These quotes express the speaker's concern about Europe's position in AI development and the potentially detrimental impact of new regulations on AI startups.
"Good regulation is about good timing and involving the right people who are actually doing stuff in this domain." "First, I wait a little bit, because it's too early."
The speaker suggests that regulators should wait and involve practitioners in the field before enacting AI regulations, emphasizing the importance of timing and real-world experience.
"They have a key advantage that there is no GDPR or very few regulation internally." "They probably have everything about every individual in China."
These quotes underline China's competitive edge in AI due to less restrictive data regulations and extensive data availability.
"That will keep Europe behind and we are already behind." "Immigration laws may be something that will be a problem eventually in the US."
The speaker analyzes the factors that could influence the success of different regions in AI, highlighting Europe's regulatory challenges and potential issues with US immigration policy.
"AI will enable a new generation of players in every industry that will kill the incumbents eventually." "Existing services companies like consulting companies are embracing AI."
These quotes predict the transformative impact of AI on various industries and the potential for AI to create new market leaders.
"Make it more diverse." "Somebody discovered that the model...didn't learn to find the tennis ball. It learned the context of a tennis ball, like a racket."
The speaker emphasizes the importance of diversity in identifying and correcting biases within AI models, using a real-world example to illustrate the point.
"I don't think AI will kill investigative journalism." "Now, the generation part at the output probably will be disrupted a lot by AI."
The speaker believes that while AI will change certain aspects of media, it will not eliminate the need for human-driven investigative journalism.
"That fundamentally people don't really change." "You can help people to grow... But I think you have to accept that some of the fundamental characteristics... won't change."
The speaker reflects on the nature of personal growth and the limits of change, suggesting that fundamental characteristics are stable over time.
"It's that they think it's conscious." "In 1966, when the mother of all chat bots was released at the MIT Eliza... people around the world thought that AI was solved."
The speaker addresses the misconception of AI consciousness and cites historical examples of people being deceived by the superficial appearance of intelligence in AI systems.
"It's very hard because when we change belief, we tend to forget that we have the contrary belief before." "If patients love your healthcare product enough and you prove the health benefits of your product, then payers will pay for it."
These quotes discuss the evolution of beliefs and the speaker's personal learning experience regarding value and payment in the healthcare industry.
"There is nothing about financial industry finance in being a VC a little bit, of course, but it's mostly about entrepreneurship." "But for my first startups, I did some strategic mistakes, go to market mistakes, and then I wish my vcs back then would have been more involved with me and coach me."
The speaker contrasts the backgrounds of European and US venture capitalists and shares personal experiences with VC support during different stages of company growth.
"Every physician has their AI assistant doing all a lot of stuff for them and helping them to be ten times more efficient." "So my dream is to build a healthcare system from scratch without any of all these limitations and constraints we mentioned."
The speaker outlines a vision for the future of healthcare, emphasizing the role of AI in augmenting medical professionals and optimizing healthcare systems.