Yann LeCun - Gen AI Winter School, Objective Driven AI.

The talk explores the pursuit of human-level AI, emphasizing the need for systems that can learn, remember, reason, plan, and possess common sense, unlike current AI models. The speaker critiques existing machine learning techniques, highlighting the limitations of supervised learning and the potential of self-supervised learning. They propose "objective-driven AI" as a solution, advocating for models that can plan and execute tasks with safety and efficiency, using joint embedding architectures over generative models. The speaker underscores the importance of open-source AI to ensure diversity and prevent control by a few entities, warning against the dangers of closed AI systems.

• The goal is to build AI systems reaching human-level capabilities: learning, memory, reasoning, planning, common sense, steerability, and safety.
• Current AI systems fall short in these areas, requiring advancements to achieve human-level intelligence.
• The future envisions AI mediating interactions with digital content, necessitating AI with human-level intelligence to avoid user frustration.

"We need systems that have essentially can reach human level AI systems that can learn, they can remember, they can reason, can plan, that have common sense, and as well as are steerable and safe."

• Highlights the essential capabilities required for AI to reach human-level intelligence.
• Emphasizes the gap between current AI systems and the desired human-level AI.

"There is a future in which basically all of our interaction with the digital world will be mediated by AI assistance."

• Predicts a future where AI assists in digital interactions, making human-level intelligence crucial.
• Underlines the importance of developing AI systems capable of seamless digital mediation.

• Current machine learning, particularly supervised learning, is limited in real-world applications.
• Self-supervised learning has been a recent breakthrough, showing potential for real-world applicability.
• Human and animal learning models are driven by objectives and goals, which current AI lacks.

"Machine learning really sucks when we compare it to what humans and animals can do."

• Acknowledges the limitations of current machine learning compared to human and animal capabilities.
• Suggests the need for AI systems to emulate human and animal learning processes.

"What really has revolutionized machine learning in the last five or six years is self-supervised learning."

• Identifies self-supervised learning as a significant advancement in machine learning.
• Points to self-supervised learning as a potential path toward more human-like AI capabilities.

• Generative models like transformers have shown impressive results but have limitations in reasoning and common sense.
• These models often make factual errors and lack memory and planning capabilities.
• The current paradigm may require radical changes to address these issues.

"Generative models really are an application of this, so in a generative language model, the particular architecture of course of the system is a Transformer."

• Describes the architecture of generative models, specifically transformers.
• Highlights the application of generative models in language processing.

"They really make stupid mistakes, they make factual errors, logical errors, inconsistent, they have reasoning abilities, they can be toxic."

• Critiques the limitations and errors present in current generative models.
• Emphasizes the need for improvements in reasoning and factual accuracy.

• AI has significantly impacted natural language understanding and translation.
• Advanced systems can translate multiple languages and preserve the speaker's voice and expression.
• Despite advancements, AI's understanding of the world through language is limited.

"AI of course also is images and that is making progress, this is the system from meta which I advantage described in a paper."

• Acknowledges the advancements in AI for image processing alongside language.
• Highlights ongoing progress in AI's ability to handle complex tasks like translation.

"There's a version of it that can that is real time with two-second latency and preserves the voice and the expression of the speaker."

• Describes the capabilities of advanced translation systems in maintaining speaker characteristics.
• Emphasizes the potential of AI to break language barriers in real-time communication.

• AI struggles with planning and generating novel plans outside training data.
• Language-based AI models have limitations in capturing the full scope of human knowledge.
• Future AI research must address these limitations to achieve human-level intelligence.

"Really cannot plan, they can produce a plan whenever the plan performs to something seen in the training data but they can't really invent."

• Highlights AI's limitations in generating original plans beyond training data.
• Suggests the need for AI to develop genuine planning abilities.

"If we only train AI with language, we're never going to get them to reach human level intelligence because there's just not that much knowledge in language."

• Critiques the reliance on language as the sole source of knowledge for AI.
• Points to the necessity of incorporating other forms of sensory input for AI development.

• Achieving human-level intelligence requires AI systems to learn from sensory inputs beyond language.
• The current amount of text data is insufficient; AI must leverage sensory input for comprehensive learning.
• Future AI systems must integrate multiple modules and world models to understand and interact with the world effectively.

"The only way we'll ever get to human level intelligence or to animal level intelligence is if we get systems to learn from sensory inputs."

• Advocates for the inclusion of sensory inputs in AI learning to reach human-level intelligence.
• Suggests that expanding beyond text data is crucial for AI development.

"The idea is to build a system that has multiple modules which are trained simultaneously."

• Proposes a modular approach to AI system design for comprehensive learning.
• Emphasizes the integration of various learning modules to enhance AI capabilities.

• The world model is an architecture that predicts the resulting state of the world from an initial state after a sequence of actions.
• It involves setting objectives to measure task accomplishment and guardrails to ensure safety and appropriateness of actions.
• Optimization is used to find the sequence of actions that minimize objectives, akin to a cost function, to achieve desired outcomes without causing damage.

"Imagine a sequence of actions into your world model, and from the initial state of the world, the world model predicts what the resulting state of the world will be after you've taken these actions."

• This quote explains the fundamental function of the world model, which is to predict outcomes from a sequence of actions.

"The system operates through optimization, trying to find a sequence of actions that will minimize those objectives."

• The system uses optimization to ensure actions taken minimize objectives, balancing task achievement and adherence to safety guardrails.

• Hierarchical planning involves breaking down a complex task into smaller, manageable sub-tasks.
• This method allows for planning at different levels of abstraction, from high-level goals to specific actions.
• The challenge lies in training systems to learn multiple levels of abstraction for effective hierarchical planning.

"Hierarchical planning is a very important thing to be able to do if you want the intelligence system to accomplish complicated tasks."

• Hierarchical planning is crucial for intelligent systems to handle complex tasks by breaking them into smaller tasks manageable at various abstraction levels.

"We have absolutely no idea how to do this other than this very vague idea that is presented here."

• Despite its importance, the methodology for achieving hierarchical planning in AI remains largely undeveloped.

• Humans and animals learn about the world through observation and interaction, forming a basis for world model learning.
• Generative AI is a natural approach for world model learning, predicting future states from observed data.
• Current generative architectures for video prediction struggle with accuracy due to the unpredictable nature of the world.

"Many humans and animals learn how the world works in the first few months of life by basically observing the world initially and then interacting in the world."

• This highlights the natural learning process, which serves as a model for developing AI world models.

"Generative architectures for images and video do not work well if you want to use them to learn representations of the world."

• Current generative models fail to accurately predict future states, indicating a need for improved methods in world model learning.

• Joint Embedding Architectures (JEAs) aim to improve prediction accuracy by focusing on representations rather than raw data.
• JEAs use an encoder to transform inputs into a representation space where predictions are made.
• This approach helps in eliminating unpredictable noise and constructing abstract representations necessary for hierarchical world models.

"The difference between a JEA and a generative architecture is that you also run Y through an encoder."

• JEAs improve upon generative architectures by encoding inputs, allowing for more accurate predictions in a representation space.

"It extracts the idea of the state of the world and then, using an action, predicts what the next state of the world is going to be."

• JEAs focus on extracting core representations of the world state to facilitate accurate predictions of future states.

• Energy-based models capture dependencies between variables by assigning low energy to compatible variable pairs and high energy otherwise.
• These models are used to understand relationships in data, such as video continuations, by evaluating compatibility through energy levels.

"Imagine we have two variables X and Y, and you want to capture the dependency between X and Y."

• Energy-based models assess the dependency between variables, which is crucial for predicting outcomes based on initial states.

"What this function should do is produce a single scalar value that is small if Y is a good continuation for X and large if Y is not."

• The function of energy-based models is to evaluate the compatibility of variable pairs, aiding in accurate prediction and modeling.

• Energy-based models focus on shaping the energy function to give low energy to observed data and high energy to unobserved data.
• Two main classes of methods: contrastive methods and regularized methods.
• Contrastive methods involve generating negative samples to shape the energy landscape, but they struggle in high-dimensional spaces.
• Regularized methods minimize the volume of space that can take low energy, ensuring that reducing energy in some regions increases it elsewhere.

"The training... is going to be to shape this energy function so that it gives low energy to stuff we observe and high energy to stuff we don't observe."

• This describes the fundamental goal of training energy-based models, focusing on discriminating between observed and unobserved data.

"The problem with contrasting method is that they kind of break down in high dimension."

• Highlights a limitation of contrastive methods, particularly in high-dimensional representation spaces.

• Suggests moving away from generative models and probabilistic modeling in favor of energy-based models.
• Advocates for regularized methods over contrastive methods due to practical advantages.
• Criticizes reinforcement learning for its inefficiency in many systems.

"Abandon generative models in favor of Jing architectures... abandon probabilistic modeling in favor of those energy-based models."

• Encourages the adoption of energy-based models over traditional generative and probabilistic approaches for building AI systems.

• Non-contrastive methods aim to maximize the information content from the encoder while minimizing prediction error.
• The goal is to balance carrying maximum information without including non-predictable information.
• Variance-Invariance Co-variance Regularization (VICReg) prevents collapse by ensuring variability and decorrelation among components.

"One class that works by maximizing some measure of information... to prevent collapse."

• Describes a class of non-contrastive methods focused on maximizing information to maintain effective representations.

"Variance-Invariance Co-variance Regularization... tries to decorrelate the components."

• Explains a method to prevent collapse by maintaining variability and decorrelation in the encoded representations.

• Distillation methods involve using two encoders with shared weights, often using exponential moving average for stability.
• These methods are efficient, fast, and do not collapse, despite a lack of full understanding of why they work.
• DINO (Distillation with No Labels) is highlighted as a successful self-supervised method for image feature extraction.

"The encoder of Y uses weights that are basically exponential moving average... it doesn't collapse."

• Describes the structure and stability of distillation methods, emphasizing their efficiency despite theoretical uncertainties.

"DINO really works really well if you want a self-contained image feature extraction."

• Highlights the effectiveness of DINO for self-supervised image feature extraction across various applications.

• Distillation methods are used in various applications, including object recognition, feature extraction, and environmental monitoring.
• These methods perform well in transfer tasks and segmentation, with specific adaptations for local feature matching.
• Recent advancements include video versions of distillation methods, showing strong performance in video classification.

"Extracting features from video... system that extracts an estimate of the height of the canopy in forests."

• Illustrates the application of distillation methods in environmental monitoring, specifically for estimating canopy height from satellite images.

"Video Jepa... works really well for learning teachers to classify videos."

• Describes the application of distillation methods in video classification, emphasizing their effectiveness in understanding dynamic content.

• Masked autoencoders are generative architectures that predict missing parts of input data, such as images or videos.
• These methods are efficient and perform well compared to other reconstruction-based or supervised methods.
• They offer good performance even with limited supervision, making them suitable for various data regimes.

"The idea there is you take an image... train a predictor to predict the missing pieces."

• Explains the concept of masked autoencoders, focusing on their ability to predict missing parts of input data efficiently.

"You get good performance also on sort of the data regime where you only train supervised on 1% of ImageNet."

• Highlights the robustness of masked autoencoders, achieving strong performance with minimal supervised training data.

• Developing a complete AI agent requires solving numerous problems related to architecture and planning.
• Current AI systems need enhancements to handle uncertainties and inaccuracies in world models effectively.
• The focus is on self-supervised learning (SSL) from video and learning modules that can reason, plan, and work hierarchically.

"We still have a lot of problems to solve to make this work in terms of a complete agent according to the architecture object-driven architecture I showed you earlier."

• The speaker emphasizes the complexity and the multitude of challenges involved in developing a complete AI agent.

• Open source AI platforms are crucial to prevent control by a small number of entities.
• Diversity in AI systems is essential for maintaining a diversity of thought and information sources.
• Governments are considering regulations that could make open source AI illegal, which poses a threat to innovation and democracy.

"Those platforms must be open source because we cannot have a small number of AI assistance controlling the entire digital diet of every citizen across the world."

• The speaker advocates for open source AI to ensure a diverse and democratic digital ecosystem.

• There is no such thing as AGI because human intelligence is specialized, and AI should be seen as a collection of skills.
• Machines will eventually surpass human intelligence in specific domains, which should be viewed as a benefit.

"There is no such thing as AGI artificial general intelligence because even human intelligence is very specialized."

• The speaker clarifies that AGI is a misconception and highlights the specialization inherent in both human and machine intelligence.

• Current reinforcement learning (RL) approaches in language models are limited and not fully representative of traditional RL.
• The inefficiency of RL, especially in real-world scenarios, suggests minimizing its use.

"The type of reinforcement learning that is used in LLM which is RHF is not really reinforcement learning; it's really learning an objective function."

• The speaker explains the limitations of current RL implementations in language models and their inefficiency.

• Progress is being made in developing models with fewer parameters that maintain performance.
• Future language models will likely incorporate world models to better understand and respond to user queries.

"I think the future of LLM will be systems that can basically do what I described in the talk, so systems that build a representation of the question then compute a representation of the answer."

• The speaker envisions future language models that use world models to generate more context-aware and accurate responses.

• A single, configurable world model could handle various tasks by adapting to specific situations.
• This approach mirrors human cognitive processes, where we focus on one conscious task at a time.

"The role of this configurator is basically to configure the world model for the situation at hand."

• The speaker suggests using a configurable world model to efficiently manage different tasks, akin to human cognitive strategies.

• There is a need for exploring a wide range of architectural components to advance AI capabilities.
• Current architectures, like convolution and self-attention, have limitations that need addressing.

"I think there is a need for very wide exploration of architectures to do this right."

• The speaker calls for innovation in AI architectures to overcome existing limitations and enhance performance.