{"id":51375,"date":"2025-03-13T12:26:23","date_gmt":"2025-03-13T06:56:23","guid":{"rendered":"http:\/\/officechai.com\/?p=51375"},"modified":"2025-03-13T12:26:24","modified_gmt":"2025-03-13T06:56:24","slug":"yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai","status":"publish","type":"post","link":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/","title":{"rendered":"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI"},"content":{"rendered":"\n

There have been some concerns that pre-training on text is hitting a wall in AI model development, and there might be structural reasons why this could be happening.<\/p>\n\n\n\n

Recently, Yann LeCun, the Chief AI Scientist at Meta and a Turing Award winner, provided a compelling argument for why text data alone is insufficient for achieving human-level AI. His insightful calculation, comparing the data ingested by large language models (LLMs) with the sensory input a child receives, highlights a fundamental limitation of current AI approaches. His analogy emphasizes the sheer volume of visual data processed by a child in their formative years, suggesting that replicating this experience digitally is crucial for achieving true artificial intelligence.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

\u201cLet me give you a very simple calculation,” LeCun says<\/a>. “A typical large language model is trained with something on the order of 20 trillion tokens\u2014 20 thousand billion tokens. The token is like a word, more or less. The token typically is represented in three bytes. So 20 or 30 trillion tokens, each in three bytes\u2014that\u2019s about 10 to the 14 bytes; one with 14 zeros behind it. This is the totality of all the texts available publicly on the internet. It would take any of us several hundred thousand years to read through that material. Okay, so it\u2019s an enormous amount of information.\u201d<\/p>\n\n\n\n

He continues, building his comparison: \u201cBut then you compare this with the amount of information that gets to our brains through the visual system in the first four years of life, and it\u2019s about the same amount. In four years, a young child has been awake a total of about 16,000 hours. The amount of information getting to the brain through the optic nerve is about 2 megabytes per second. Do the calculation and that\u2019s about 10 to the 14 bytes. It\u2019s about the same. In four years a young child has seen as much information or data as the biggest LLMs.\u201d<\/p>\n\n\n\n

And he concludes with his takeaway: \u201cWith these two, is that we\u2019re never going to get to human-level AI by just training on text. We\u2019re going to have to get systems to understand the real world. That is really hard.\u201d<\/p>\n\n\n\n

LeCun’s argument underscores a critical bottleneck in current AI development. While LLMs have achieved impressive feats in natural language processing, their reliance on text data restricts their understanding of the world. A child’s sensory experience, particularly vision, provides a much richer and more nuanced understanding of reality, encompassing physical properties, spatial relationships, and causal connections that are difficult to capture in text alone. This disparity suggests that a paradigm shift is necessary, moving beyond text-based training towards models that can process and learn from multi-modal data, including visual, auditory, and perhaps even tactile information.<\/p>\n\n\n\n

There have been hints to this by other researchers as well. Ilya Sutskever has said <\/a>that data was the fossil fuel of pretraining AI models, and was getting exhausted. Elon Musk too has said that humanity was running out<\/a> of data to train AI models. As such, future advancements in AI likely hinge on developing models that can learn and reason about the world in a manner similar to humans, by integrating and synthesizing information from diverse sensory inputs. This shift represents a significant challenge, requiring not only more sophisticated algorithms but also new approaches to data acquisition, processing, and representation. The quest for human-level AI, it seems, lies not just in bigger models, but in models that can truly see the world.<\/p>\n","protected":false},"excerpt":{"rendered":"

There have been some concerns that pre-training on text is hitting a wall in AI model development, and there might be structural reasons…<\/p>\n","protected":false},"author":1,"featured_media":50322,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1029],"tags":[],"class_list":["post-51375","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"yoast_head":"\nYann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI\" \/>\n<meta property=\"og:description\" content=\"There have been some concerns that pre-training on text is hitting a wall in AI model development, and there might be structural reasons...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/\" \/>\n<meta property=\"og:site_name\" content=\"OfficeChai\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/OfficeChai\/\" \/>\n<meta property=\"article:published_time\" content=\"2025-03-13T06:56:23+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-03-13T06:56:24+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"630\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"OfficeChai Team\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:site\" content=\"@OfficeChai\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"OfficeChai Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/\",\"url\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/\",\"name\":\"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI\",\"isPartOf\":{\"@id\":\"https:\/\/officechai.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1\",\"datePublished\":\"2025-03-13T06:56:23+00:00\",\"dateModified\":\"2025-03-13T06:56:24+00:00\",\"author\":{\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\"},\"breadcrumb\":{\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1\",\"width\":1200,\"height\":630},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/officechai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/officechai.com\/#website\",\"url\":\"https:\/\/officechai.com\/\",\"name\":\"OfficeChai\",\"description\":\"Startups, Businesses And Careers\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/officechai.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2\",\"name\":\"OfficeChai Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/officechai.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g\",\"caption\":\"OfficeChai Team\"},\"description\":\"Dotting the i's, crossing the t's.\",\"url\":\"https:\/\/officechai.com\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/","og_locale":"en_US","og_type":"article","og_title":"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI","og_description":"There have been some concerns that pre-training on text is hitting a wall in AI model development, and there might be structural reasons...","og_url":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/","og_site_name":"OfficeChai","article_publisher":"https:\/\/www.facebook.com\/OfficeChai\/","article_published_time":"2025-03-13T06:56:23+00:00","article_modified_time":"2025-03-13T06:56:24+00:00","og_image":[{"width":1200,"height":630,"url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1","type":"image\/jpeg"}],"author":"OfficeChai Team","twitter_card":"summary_large_image","twitter_creator":"@OfficeChai","twitter_site":"@OfficeChai","twitter_misc":{"Written by":"OfficeChai Team","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/","url":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/","name":"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI","isPartOf":{"@id":"https:\/\/officechai.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage"},"image":{"@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1","datePublished":"2025-03-13T06:56:23+00:00","dateModified":"2025-03-13T06:56:24+00:00","author":{"@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2"},"breadcrumb":{"@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#primaryimage","url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1","contentUrl":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1","width":1200,"height":630},{"@type":"BreadcrumbList","@id":"https:\/\/officechai.com\/ai\/yann-lecun-explains-why-text-data-alone-will-never-create-human-level-ai\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/officechai.com\/"},{"@type":"ListItem","position":2,"name":"Yann LeCun Explains Why Text Data Alone Will Never Create Human-Level AI"}]},{"@type":"WebSite","@id":"https:\/\/officechai.com\/#website","url":"https:\/\/officechai.com\/","name":"OfficeChai","description":"Startups, Businesses And Careers","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/officechai.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/officechai.com\/#\/schema\/person\/5861f1134993293cc28905de7624d6b2","name":"OfficeChai Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/officechai.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/61d744733248dc647d505d0676bb425323413132ee5447e86aa8eecbbb7b27d5?s=96&d=mm&r=g","caption":"OfficeChai Team"},"description":"Dotting the i's, crossing the t's.","url":"https:\/\/officechai.com\/author\/admin\/"}]}},"jetpack_featured_media_url":"https:\/\/i0.wp.com\/officechai.com\/wp-content\/uploads\/2025\/01\/MixCollage-02-Jan-2025-01-28-PM-2773.jpg?fit=1200%2C630&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p685C6-dmD","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/51375","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/comments?post=51375"}],"version-history":[{"count":1,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/51375\/revisions"}],"predecessor-version":[{"id":51376,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/posts\/51375\/revisions\/51376"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media\/50322"}],"wp:attachment":[{"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/media?parent=51375"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/categories?post=51375"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/officechai.com\/wp-json\/wp\/v2\/tags?post=51375"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}