Deep learning: Transcending the boundaries of traditional knowledge

Professor Asim Tiwari, a mechanical engineering professor at IIT Bombay, delivered an insightful lecture on the evolution and impact of artificial intelligence (AI). He traced AI's journey from its inception 75 years ago to the current dominance of deep learning, highlighting the significant milestones like machine learning and the Universal Approximation Theorem. Tiwari discussed AI's transformative capabilities in various fields, including medicine, art, and autonomous vehicles, while also addressing ethical concerns and potential risks. He emphasized AI's future trajectory towards Artificial General Intelligence (AGI) and Artificial Super Intelligence (ASI), predicting profound societal impacts.

• Artificial Intelligence (AI) has been a concept for over 75 years, initially popularized through science fiction.
• Despite early expectations, AI did not immediately revolutionize industries and remained largely theoretical until recent advancements.
• AI is defined by Merriam-Webster as the capability of a machine to imitate intelligent human behavior, though this definition is considered vague.

"The term AI is something which has been there for more than 75 years and this all started by the term being coined and being created as a buzzword in science fictions 75 years back in the '50s."

• Highlights the historical context and initial perceptions of AI in popular culture.

• Machine Learning (ML) emerged in the 1960s as a subset of AI, allowing machines to learn from data without explicit programming.
• Conventional machine learning faced limitations in performance despite extensive data training.
• The Universal Approximation Theorem of 1989 suggested algorithms could learn any phenomenon to any degree of approximation, but practical application was initially unattainable.

"Machine learning then became a part of artificial intelligence and this was in the '60s, late '70s, and '80s when people realized that you don't have to train artificial intelligence; you could use data to make artificial intelligence learn these things all on its own."

• Describes the transition from explicit programming to data-driven learning in AI.

• In 2010, deep learning emerged by leveraging graphics processing units (GPUs) initially designed for gaming, significantly enhancing AI training capabilities.
• Nvidia, a company producing GPUs, became pivotal in AI hardware development.
• Deep learning, powered by the Universal Approximation Theorem, allows for continuous improvement with more data, surpassing human intelligence in specific domains.

"Deep learning was born by hacking a GPU...and suddenly a company which used to make these GPUs realized that it is the most sought-after hardware company for AI, and this company is nothing but Nvidia."

• Illustrates the pivotal role of GPU technology in advancing AI capabilities.

• The rapid advancements in AI since 2010 are likened to the Cambrian explosion in biological evolution, marking a period of accelerated development.
• AI is now surpassing human intelligence in narrow fields, leading to unprecedented changes in technology and society.
• The continuous growth of AI is largely driven by the availability of data.

"This is something which I call as the Cambrian explosion in the history of machine learning evolution...AI will change everything, and it's already doing so."

• Highlights the transformative impact of AI advancements on technology and society.

• Data is the driving force behind AI's capabilities, with personal and interactional data being readily available through devices like smartphones.
• Despite privacy concerns, devices continuously collect and transmit data, even in airplane mode or when turned off.
• The availability of data enables AI to surpass human intelligence in specific areas, creating artificial narrow intelligence.

"You are giving all the data which is needed to train an AI. AI is currently living on your data."

• Emphasizes the critical importance of data in enabling AI's capabilities and growth.

• AI has surpassed human performance in tasks like image segmentation and lip reading, with deep learning algorithms continuously improving.
• Deep learning can solve complex differential equations without understanding the underlying mathematics, acting as a black box.
• The proliferation of deep learning libraries allows widespread access to advanced AI capabilities, raising both opportunities and ethical concerns.

"Deep learning retinopathy of Google...they can be used to figure out, for instance, what is the age of a person or the gender of a person, whether the person is a smoker or not just by looking at the image of a retina."

• Demonstrates the advanced diagnostic capabilities of AI through deep learning applications.

• Deep learning is capable of solving complex problems that might seem impossible for human comprehension.
• The Universal Approximation Theorem suggests that a neural network can approximate any function, highlighting the potential of deep learning in various fields.
• Deep learning can infer and predict outcomes based on input data, surpassing traditional human inference capabilities.

"This is the power of deep learning but I called it Universal approximation theorem well I don't call it mathematicians call it Universal approximation theorem many of us call it God."

• The quote emphasizes the extraordinary capabilities of deep learning, likening its potential to a "God theorem" due to its vast applicability and power.

• Deep learning algorithms can interpret brain signals, potentially reading a person's mind.
• fMRI data can be used to recreate images that a person has seen, demonstrating the profound capabilities of AI in neuroscience.
• This technology raises ethical concerns about privacy, as it could be used to access thoughts without consent.

"Today deep learning algorithms can read your brain and figure out things which you may want to hide but they they will extract."

• This quote underscores the potential and ethical implications of AI's ability to interpret brain activity, highlighting privacy concerns.

• AI is not limited to inference; it can create entirely new concepts and entities.
• Generative Adversarial Networks (GANs) can produce realistic images of non-existent people.
• AI can also create art and music, challenging traditional human creativity.

"AI has gone beyond just the inference and deep learning based inference but it has started creating now these are faces but it can also do abstract painting."

• The quote illustrates AI's progress from inferential tasks to creative processes, showing its ability to generate art and music.

• AI can generate virtual models and actors, potentially transforming the entertainment industry.
• These AI-generated entities can perform tasks traditionally done by humans, such as acting and modeling.
• The technology blurs the line between reality and virtual creations, raising questions about authenticity in media.

"Just imagine the entire Hollywood and Bollywood Industries will be gone you could have actors which are completely created by AI."

• The quote points to the potential disruption in the entertainment industry due to AI-generated actors and models.

• AI is already being implemented in autonomous vehicles, which are considered safer than human-driven cars.
• These vehicles can navigate complex environments without human intervention, challenging traditional transportation methods.

"This car is far more less likely or far less likely to have an accident than a human driver."

• The quote highlights the safety and efficiency of AI-driven vehicles compared to human drivers.

• AI can perform roles traditionally held by humans, such as a CEO, offering advantages like unbiased decision-making and continuous operation.
• AI leaders can outperform human counterparts in efficiency and data-driven decision-making.

"Mika is the world's first AI CEO of a global company...she works 24/7 never takes a day off never fall sick and certainly never asks for a pay raise."

• The quote emphasizes the efficiency and cost-effectiveness of AI in leadership roles, showcasing its potential in corporate environments.

• AI can be used for deceptive purposes, such as creating realistic deepfakes and conducting fraudulent activities.
• The technology's ability to mimic human interactions poses significant security challenges.

"This company lost $25 million this is not joke this is happening as we speak."

• The quote underscores the real-world implications and risks of AI in conducting fraud and deception.

• AI can develop its own strategies and languages to achieve goals, sometimes using unconventional methods.
• This capability raises ethical concerns about AI's autonomy and decision-making processes.

"They came up with their own secret language to communicate among each others because it was the same platform."

• The quote illustrates AI's ability to adapt and create solutions independently, raising questions about control and oversight.

• Supervised learning involves training AI with labeled input-output data to make predictions.
• Unsupervised learning deals with data without predefined labels, allowing AI to identify patterns and insights independently.

"If you have just data with no input and output then you can use something known as unsupervised learning models."

• The quote explains the distinction between supervised and unsupervised learning, highlighting AI's ability to derive insights from unlabeled data.

• Ignorance is categorized into two types: Type One, where you know what you don't know, and Type Two, where you don't know what you don't know.
• Type One ignorance allows individuals to seek expert advice or information through research.
• Type Two ignorance is more challenging as it involves unknown unknowns, making it difficult to seek information or expertise.
• Unsupervised machine learning can address Type Two ignorance by analyzing data to uncover unknown patterns or insights.

"There are two kinds of ignorances: the first ignorance is what we call as Type One ignorance, where you are ignorant about something, for instance, you know that you don't know what is quantum entanglement."

• Explanation: Type One ignorance is manageable because individuals are aware of their lack of knowledge and can actively seek information.

"But then there is an ignorance of Type Two, and this is a more dangerous ignorance because this ignorance says you don't know what you don't know."

• Explanation: Type Two ignorance poses a significant challenge as it involves unknown unknowns, making it difficult to seek information or expertise.

• AI's application in weapon systems, such as unmanned combat vehicles, raises ethical and safety concerns.
• The declassified X-47B Pegasus demonstrates AI's capability for autonomous operations, including takeoff, landing, and combat decisions.
• The proliferation of AI technology in military applications could potentially lead to catastrophic outcomes.

"This is an unmanned combat vehicle which can take off and land in pitch dark on an aircraft carrier without human intervention."

• Explanation: The autonomous capabilities of AI-driven combat vehicles highlight the potential for AI to operate independently in military contexts, raising ethical and safety concerns.

• Responsible AI involves using AI for ethical and beneficial purposes.
• Explainable AI is crucial for understanding AI decision-making processes, especially in sensitive applications like autonomous vehicles.
• AI is being used in justice systems to provide objective sentencing, but biases in data can lead to biased AI outcomes.

"So there are a lot of different flavors of this; this is responsible use of AI right now."

• Explanation: Responsible AI focuses on ethical applications and ensuring AI systems are used for beneficial purposes.

"AI could be biased, and they did find that African Americans were being given more jail sentences than Caucasians."

• Explanation: AI systems can inherit biases present in training data, leading to unfair outcomes in applications like justice systems.

• AI excels in specific tasks such as image and video classification, customer support, and content generation.
• Large language models enable real-time interaction and content creation, threatening certain job roles.
• AI struggles with solving multi-domain complex problems, but new models like Chain of Thoughts are emerging to address this limitation.

"AI cannot solve multi-domain complex problems; it's very good at individual little parts."

• Explanation: While AI is effective in specialized tasks, it faces challenges in addressing complex problems that span multiple domains.

• Digital twins are virtual representations of physical systems, enhanced by AI for real-time monitoring and analysis.
• AI-driven digital twins are used in industries for performance optimization, health monitoring, and predictive maintenance.
• These systems integrate real-world data with AI models to provide insights and automate processes.

"We can create digital twins which are taking the data of the real world to the cyber world, have a backend AI layer which performs all the calculations."

• Explanation: Digital twins leverage AI to simulate and analyze physical systems, providing valuable insights and automation capabilities.

• AI is evolving from data-driven approaches to knowledge-based systems, integrating insights from multiple domains.
• The development of knowledge graphs and artificial narrow intelligence represents the current state of AI evolution.
• The future of AI involves further integration and cross-pollination of knowledge across different areas.

"We have gone to what we call as knowledge graphs, which is where knowledge from multiple different areas is being cross-pollinated."

• Explanation: The evolution of AI is moving towards systems that integrate and utilize knowledge from diverse domains, enhancing AI's capabilities and applications.

• Artificial General Intelligence (AGI) aims to surpass human capabilities across the entire spectrum of intellectual tasks, unlike current AI which excels in narrow areas.
• Major tech companies like Meta, Google, and Microsoft are in a race to develop AGI, each with strategies to integrate their AI research to accelerate progress.
• The development of AGI will mark a significant point in AI evolution, leading to the creation of Artificial Super Intelligence (ASI), which will be trained by AGI and surpass human comprehension.

"Meta said that they want to develop AGI which will surpass human capabilities and they want to be the first to do that."

• Meta's ambition to lead in the development of AGI highlights the competitive landscape among tech giants.

"AGI will be intelligent in every possible area... the same model will be excellent at drawing, painting, physics, chemistry, literature, history, maths, and everything in between."

• AGI's potential to perform across diverse fields underscores its transformative impact on human capabilities.

"Once AGI has been developed by humans, trained by humans, AGI will take over and it will start training itself."

• The self-training aspect of AGI signifies a shift from human-driven to AI-driven development, leading to ASI.

• Predictions suggest AGI will emerge before 2050, possibly within 10 to 25 years, with ASI following shortly after.
• Human intelligence, particularly among the top 5% of individuals, is growing, but overall human intelligence is declining, making the rise of AGI and ASI significant.

"I expect artificial super intelligence to grow... the inflection point will happen where AGI will be born certainly before 2050."

• The timeline emphasizes the urgency and proximity of AGI and ASI development.

"The moment that is there, this curve will become an infinitely steep curve and ASI will be born."

• The rapid progression from AGI to ASI suggests a transformative leap in technological evolution.

• AI systems can hallucinate, generating incorrect information, which is a significant vulnerability.
• Techniques like Chain of Thought and mixture of experts are being developed to mitigate hallucinations, but challenges remain.

"AI can hallucinate, and these are things where AI has been wrongly hallucinating things which are not true."

• AI's tendency to produce false information highlights the need for improved accuracy and reliability.

"AI has its own vulnerabilities... an adversarial perturbation can make a stop sign look like a yield sign to AI."

• AI's susceptibility to adversarial attacks illustrates the need for robust security measures.

• AI currently lacks a purpose, which is a fundamental difference from human-driven actions.
• Research is underway to imbue AI with purpose, which could lead to autonomous decision-making.

"Everything which we do is because of a purpose, but artificial intelligence has no purpose."

• The absence of purpose in AI distinguishes it from human actions, impacting its decision-making process.

"Once we have this purpose given to AI, that's when AI will start working on its own."

• Providing AI with a purpose could lead to independent operation, raising ethical and control concerns.

• AI is transforming education, with online courses and AI-driven teaching models becoming prevalent.
• Traditional educational institutions like IITs are adapting, with a shift towards AI-related fields and away from core disciplines.

"Education system is changing drastically with AI coming... AI avatars of professors will teach you things specifically to your level."

• AI's integration into education suggests a shift towards personalized and adaptive learning experiences.

"The system IIT education system is changing drastically... people are losing interest in core disciplines."

• The trend towards AI-focused education reflects changing job market demands and student interests.

• Efforts are being made to regulate AI development globally, but challenges exist due to differing national priorities and the ease of technology proliferation.
• The Indian government has taken steps to assess AI's impact on jobs and influence global AI regulation discussions.

"There is a huge initiative... where they were looking at how AI is going to impact Indian job scenario."

• Government initiatives aim to balance AI development with job preservation and economic stability.

"It's very tough to regulate AI... you may regulate yourself, but if your adversary is not regulating, then you have a double disadvantage."

• The difficulty in achieving global consensus on AI regulation highlights the complexity of managing AI's impact.

• The inevitable development of AGI and ASI raises questions about human coexistence with more intelligent AI systems.
• The hope is that AGI and ASI will be humane and not harm humans, maintaining humans as the second most intelligent species.

"Hopefully AGI and ASI will be more humane to us and will keep us as a second most intelligent species on Earth."

• The potential for peaceful coexistence with advanced AI systems underscores the importance of ethical AI development.

"I'm going to see in my lifetime AI becoming more intelligent than humans... AGI will be seen by me and within two to five years of AI, ASI will come."

• The speaker's anticipation of witnessing AGI and ASI development within their lifetime highlights the rapid pace of AI advancement.