AI and The Next Computing Platforms With Jensen Huang and Mark Zuckerberg

Mark Zuckerberg joined a discussion at Siggraph, emphasizing Meta's advancements in AI and virtual reality. He highlighted Meta's work on generative AI, including its application in recommendation systems and content creation. Zuckerberg also discussed the potential of AI-enhanced smart glasses and the company's open-source approach with projects like Llama, which democratizes AI development. The conversation touched on the transformative impact of generative AI across various fields and industries, and the ongoing evolution of computing platforms. The session concluded with a light-hearted jersey swap between Zuckerberg and the host.

• The host introduces Mark Zuckerberg, highlighting his achievements.
• Zuckerberg is recognized for starting a company that touched billions of lives, leading it to over $1 trillion in value, and being a college dropout.

"Ladies and gentlemen, please help me welcome Mark Zuckerberg."

• Introduction of Mark Zuckerberg to the audience.

• Siggraph is a notable event for computer graphics, image processing, artificial intelligence, and robotics.
• Companies like Disney, Pixar, Adobe, Epic Games, and NVIDIA have showcased innovations at Siggraph.
• NVIDIA introduced 20 papers on AI and simulation, including differentiable physics and synthetic data generation.

"This is the show of computer graphics, image processing, artificial intelligence and robotics combined."

• Highlights the significance and interdisciplinary nature of Siggraph.

• Meta has been involved in AI for several years, notably through FAIR (Facebook AI Research).
• Meta's work includes PyTorch, computer vision, language models, and real-time translation.

"Meta has done a lot of work and has been at Siggraph for, you know, eight years."

• Emphasizes Meta's long-term commitment to AI research and development.

• Generative AI is transforming Meta's products, including feed and recommendation systems.
• AI enhances content recommendation by managing millions of pieces of content.
• Future AI tools will create content on the fly or synthesize existing content.

"With generative AI, I think we're going to quickly move into the zone where not only is the majority of the content that you see today on Instagram just recommended to you from the kind of stuff that's out there in the world that matches your interests."

• Discusses the integration and future potential of generative AI in content creation and recommendation.

• Recommender systems have evolved from simple friend-based models to complex, general models.
• Unified AI models for all content types improve recommendation quality.
• Future models may integrate various content types and objectives.

"I kind of dream of one day, you can almost imagine all of Facebook or Instagram being, you know, like a single AI model that is unified, all these different content types and systems together."

• Envisions a future where a single AI model manages diverse content and user interactions.

• Generative AI enhances user experience in applications like WhatsApp by generating images and other content in real-time.
• AI assistants will evolve from simple chatbots to more complex systems that can handle long-term tasks and intents.

"I mean, a lot of the gen AI stuff is going to, on the one hand, I think, be this big upgrade for all of the workflows and products that we've had for a long time."

• Highlights the transformative potential of generative AI in enhancing user interactions and creating new experiences.

• AI assistants will become more sophisticated, capable of handling complex tasks and long-term projects.
• Future AI models will move beyond turn-based interactions to more dynamic and continuous engagement.

"I think it's going to pretty quickly evolve to, you give it an intent, and it actually can go away on multiple time frames."

• Predicts the evolution of AI assistants to handle more complex and extended tasks.

• Meta's AI Studio aims to empower creators and small businesses to build AI versions of themselves.
• These AI agents will help creators engage with their communities and manage interactions more efficiently.
• AI Studio will allow creators to train AI on their material to represent them authentically.

"We want to empower all the people who use our products to basically create agents for themselves."

• Describes the vision of AI Studio to enable personalized and scalable AI interactions for creators and businesses.

• Individuals will create their own AI agents for various purposes, including utility tasks, entertainment, and personalized interactions.
• Custom AI agents offer a judgment-free zone for role-playing difficult social situations, providing support and feedback.
• People prefer creating their own AI agents rather than using universal ones like Meta AI or ChatGPT.

"One of the top use cases for Meta AI already is people basically using it to role-play difficult social situations that they're going to be in."

• AI agents are being used as tools for practicing and preparing for challenging conversations.

"A lot of people, they don't just want to interact with the same agent, whether it's Meta AI or ChatGPT or whatever it is that everyone else is using, they want to kind of create their own thing."

• There is a strong desire for personalized AI agents tailored to individual needs and preferences.

• Future businesses will likely have AI agents that interface directly with customers, similar to how they currently use email and social media.
• AI can unify customer support and sales, providing a seamless experience for customers.

"I think in the future, every business is going to have an AI agent that interfaces with their customers."

• AI agents will become as essential as email addresses and social media accounts for businesses.

"As a customer, you don't really care. You know, you don't want to have a different route when you're trying to buy something versus if you're having an issue with something that you bought."

• Customers prefer a unified approach to interacting with businesses, regardless of their needs.

• AI Studio allows users to fine-tune AI models with their own data, such as images and written content.
• Creators can use AI Studio to develop custom AI agents that remember past interactions and continue conversations seamlessly.

"So the business version of this is-that I think has a little more integration and we're still in a pretty early alpha with that."

• AI Studio is in early development but shows promise for extensive customization and integration.

"And then I could, could I give it, load it with all the things that I've written, use it as my RAG?"

• AI Studio supports loading personal data to create highly personalized AI agents.

• Despite rapid progress in AI foundation models, there is still significant room for product innovation.
• The industry needs to figure out the most effective ways to utilize the current advancements in AI.

"Even if the progress on the foundation models kind of stopped now, which I don't think it will, I think we'd have like five years of product innovation for the industry to basically figure out how to most effectively use all the stuff that's gotten built so far."

• The potential for product innovation remains vast, even if foundational AI research slows down.

"I actually just think the kind of foundation models and the progress on the fundamental research is accelerating. So, that it's, a pretty wild time."

• Continuous acceleration in foundational AI research suggests an exciting future for AI development.

• Meta has a history of open-sourcing technology to benefit the broader ecosystem, such as Open Compute and PyTorch.
• Open-sourcing Llama models has democratized AI development, enabling companies and researchers to build on a strong foundation.

"We had a bunch of projects like that. I think the biggest one was probably Open Compute where we took our server designs, the network designs, and eventually the data center designs and published all of that."

• Open Compute has set industry standards and saved billions of dollars by organizing supply chains.

"When that came out, it activated every company, every enterprise and every industry. All of a sudden, every health care company was building AI. Every company was building AI, every large company, small companies, startups were building AIs."

• The release of Llama models has spurred widespread AI development across various industries.

• The next generation of computing may see a return to open ecosystems, similar to the PC era with Windows.
• An open ecosystem can provide significant advantages over closed systems, fostering innovation and collaboration.

"But it's not always like that where if you go back a generation, you know, Apple was doing their kind of closed thing. But Microsoft, which as you know, it obviously wasn't like this perfectly open company, but, you know, compared to Apple with Windows running on all the different OEMs and different software, different hardware it was a much more open ecosystem and Windows was the leading ecosystem."

• Historical context shows that open ecosystems have previously led the industry, and there is hope for a similar trend in the future.

"For us specifically I just want to make sure that we have access to-I mean, this is sort of selfish, but, you know, after building this company for a while, one of my things for the next 10 or 15 years is like, I just want to make sure that we can build the fundamental technology that we're going to be."

• Ensuring access to fundamental technology is a key motivation for advocating open ecosystems.

• Discussion on the limitations of closed platforms and the desire to build open platforms.
• Importance of having the freedom to build without restrictions from platform providers.

"There have just been too many things that I've tried to build and then have just been told, nah, you can't really build that by the platform provider."

• Frustration with closed platforms limiting innovation and development.

"For the next generation, like, we're going to go build all the way down and make sure that that..."

• Commitment to creating more open and flexible platforms for future development.

• The benefits of having both open and closed AI services.
• Open-source AI can lead to a robust ecosystem that enhances overall development.

"The idea that you could have great services, incredible services as well as open service. Open ability. Then we basically have the entire spectrum."

• The concept of using larger models to teach smaller models and the importance of synthetic data generation.

"You could use it for synthetic data generation, use the larger models to essentially teach the smaller models."

• The role of Llama Guard in providing guardrails and ensuring transparency and safety in AI development.

"It's built in a transparent way. You dedicated- You've got a world-class safety team. World-class ethics team."

• The importance of creating an ecosystem around open-source projects like PyTorch.
• Contributions from various companies, including NVIDIA, to enhance the performance and scalability of open-source projects.

"NVIDIA alone, we probably have a couple of hundred people just dedicated to making PyTorch better and scalable and, you know, more performant."

• Industry standards help optimize systems and silicon to run these open-source projects efficiently.

"All of the silicon in the systems will end up being optimized to run this thing really well, which will benefit everyone."

• Introduction of AI Foundry to help companies build and own their AI systems.
• The importance of owning AI to maintain control over institutional knowledge and data flywheels.

"We created this thing called AI Foundry. We provide the tooling, we provide the expertise, Llama technology, we have the ability to help them turn this whole thing into an AI service."

• The output of AI Foundry, known as NIM (Neuro Micro NVIDIA Inference Microservice), allows companies to run AI services on-prem or anywhere they choose.

"The output of it is what we call a NIM. And this NIM, this neuro micro NVIDIA Inference Microservice, they just download it, they take it, they run it anywhere they like, including on-prem."

• The necessity of having custom AI models tailored to specific functions within a company.
• Fine-tuning AI models to ensure they are focused on relevant tasks and not distracted by unrelated topics.

"We have software AI that understands our bugs database and knows how to help us triage bugs and sends it to the right engineers."

• The potential for a vast proliferation of different AI models tailored to specific tasks and uses.

"I would bet that they're going to be just a vast proliferation of different models."

• The importance of using high-performance AI models to maximize the efficiency of engineers.
• The role of NVLink in ensuring seamless performance of large AI models across multiple GPUs.

"We have every one of our GPUs connected by this non-blocking switch called NVLink switch."

• The cost-effectiveness of using high-performance models despite their higher initial costs due to the significant time savings for engineers.

"We pay the engineers a lot of money. And so to us, a few dollars an hour, amplifies the capabilities of somebody that's really valuable."

• The development and application of the Segment Anything model for video and computer vision tasks.
• Practical applications of the model in various industries, such as robotics and industrial digitalization.

"We're now training AI models on video so that we can understand the world model. Our use case is for robotics and industrial digitalization."

• The ability of the Segment Anything model to recognize and track objects in video, aiding in tasks like warehouse management and safety monitoring.

"Having a video understanding model, a video language model is really, really powerful for all of these interesting applications."

• The potential of smart glasses as the next computing platform, with widespread adoption anticipated.
• The distinction between smart glasses and VR/MR headsets and their respective use cases.

"Pretty much everyone who's wearing a pair of glasses today will end up that'll get upgraded to smart glasses. And that's like more than a billion people in the world."

• The belief that both smart glasses and VR/MR headsets will coexist, serving different needs and preferences.

"The VR MR headsets, I think some people find it interesting for gaming or different uses. Some don't yet. Yet my view is that they're going to be both in the world."

• Mixed Reality Headsets: Compared to smart glasses, these are more like workstations or game consoles, designed for immersive sessions requiring significant computing power.
• Smart Glasses: These are envisioned as always-on, mobile computing platforms with constraints due to their small form factor, much like mobile phones compared to desktop computers.

"Glasses are going to be sort of the mobile phone, kind of always-on version of the next computing platform, and the mixed reality headsets are going to be more like your workstation or your game console."

• Explanation: Differentiates the roles of smart glasses and mixed reality headsets in future computing, with glasses being more portable and headsets offering more immersive experiences.

• Ideal Holographic AR Glasses: Involves custom silicon and display stack work to achieve advanced holographic capabilities.
• Stylish, Functional Glasses: Partnership with Essilor Luxottica to create aesthetically pleasing glasses with integrated technology like cameras, microphones, and speakers.

"We've been building what we think is sort of the technology that you need for the kind of ideal holographic AR glasses and we're doing all the custom silicon work, all the custom display stack work."

• Explanation: Describes the technical efforts to develop advanced holographic AR glasses.

"By partnering with the best glasses maker in the world, Essilor Luxottica... let's constrain the form factor to just something that looks great. And within that, let's put in as much technology as we can."

• Explanation: Strategy to create attractive, functional glasses by collaborating with a leading eyewear company.

• Current Features: Includes camera sensors for photos, videos, livestreaming, video calls, microphones, and open-ear speakers for music and calls.
• Future Developments: Anticipate more advanced features and broader adoption, with potential for display-less AI glasses at lower price points.

"At this point we have camera sensors, so you can take photos and videos. You can actually livestream to Instagram. You can take video calls on WhatsApp and stream to the other person what you're seeing."

• Explanation: Lists the current technological capabilities of the glasses.

"I would guess that display-less AI glasses at like a $300 price point are going to be a really big product that, like tens of millions of people or hundreds of millions of people eventually are going to have."

• Explanation: Predicts the widespread adoption of affordable, AI-integrated glasses.

• Unexpected AI Advancements: Generative AI has progressed faster than expected, influencing the development of smart glasses and other technologies.
• Integration with Smart Glasses: AI capabilities like real-time translation and visual language understanding are integrated into smart glasses, enhancing their functionality.

"This breakthrough happened with LLMs. And it turned out that we have sort of really high-quality AI now and getting better at a really fast rate before you have holographic AR."

• Explanation: Highlights the rapid advancement of AI technologies compared to holographic AR.

"You have visual language understanding that you just showed, you have real-time translation. You could talk to me in one language, I hear in another language."

• Explanation: Describes the integration of advanced AI features into smart glasses.

• Virtual Meetings: Envisions future virtual meetings where participants interact as holograms, enhancing the sense of physical presence and collaboration.
• Holographic Glasses: Although full holographic glasses in a thin form factor are still far off, chunkier, stylish frames with holographic capabilities are closer to reality.

"We're not that many years away from being able to have a virtual meeting where, like, you know, it's like, I'm not here physically. It's just my hologram."

• Explanation: Predicts the future of virtual meetings with holographic technology.

"I think having it in a pair of stylish, kind of chunkier framed glasses is not that far off."

• Explanation: Suggests that holographic glasses with a chunkier, stylish design will be available soon.

• Importance of Style: Emphasizes the need for diverse styles and form factors in smart glasses, as people prefer unique, personalized eyewear.
• Open Ecosystem: The diversity in styles will lead to an open ecosystem for smart glasses, accommodating various preferences and needs.

"People really do not want to all look the same. Right? And it's like, so I do think that it's, you know, it's a, it's a platform that I think is going to lend itself... towards being an open ecosystem."

• Explanation: Highlights the necessity for personalized and diverse styles in smart glasses to cater to individual preferences.

• Cross-Industry Influence: Generative AI is rapidly influencing various fields, from climate tech to biotech, and transforming how we approach computing and software development.
• Personal AI Assistants: Envisions personal AI assistants tailored to individual preferences, providing non-judgmental, helpful interactions.

"Generative AI is right in the middle of that, fundamental transition. And in addition to that, the things that you're talking about, generative AI is going to make a profound impact in society."

• Explanation: Discusses the extensive impact of generative AI across multiple industries and its societal implications.

"Well, that's exactly the creative AI you were talking about... where we just build our own AIs and I, I load it up with all of the things that I've written and I fine tune it with the way I answer questions."

• Explanation: Describes the concept of personal AI assistants that learn and adapt to individual preferences and styles.

• Company Evolution: Both companies have evolved through various technological shifts, from desktop to mobile, VR, and AI, demonstrating resilience and adaptability.
• Large-Scale Computing: Emphasizes the importance of large, powerful computing systems, like data centers with GPUs, to support advanced AI and other technologies.

"You pivoted yours from desktop to mobile to VR to AI, all these devices, it's really, really, really extraordinary to watch."

• Explanation: Acknowledges the adaptive journey of the company through different technological eras.

"Instead of building smaller and smaller devices, we made computers the size of warehouses."

• Explanation: Highlights the shift towards large-scale computing systems to support advanced technological needs.

• Style Influencing: The speaker humorously discusses their attempts to become a style influencer in anticipation of the future market for stylish smart glasses.
• Jersey Swap Tradition: Ends with a light-hearted moment of swapping jackets, symbolizing camaraderie and mutual respect.

"I'm trying to make my way into becoming a style influencer. So I can, like, influence this before, you know, before the glasses come to the market."

• Explanation: Playfully expresses the speaker's interest in influencing fashion trends for future smart glasses.

"I think we oughta jersey swap again. All right. Well- This one’s yours. I mean, this is worth more because it's used."

• Explanation: Concludes with a symbolic gesture of swapping jackets, reinforcing the collaborative spirit between the speakers.