AI – Winston Wang

Evaluating the Mathematical Reasoning Capabilities of Large Language Models: Limitations and Challenges

Leave a Comment / Paper Skimming / By Winston

LLMs have made remarkable progress in various fields, including natural language processing, question answering, and creative tasks, even demonstrating the ability to solve mathematical problems. Recently, OpenAI’s o1 model, which uses CoT (Chain of Thought), has shown significant reasoning capabilities. However, for a long time, the commonly used GSM8K dataset has had a fixed set of questions

Evaluating the Mathematical Reasoning Capabilities of Large Language Models: Limitations and Challenges Read More »

Paper SkimmingAI, Apple Research, LLM, Math Reasoning, TechNews

Unveiling AlphaFold 3: The Next Leap in Predicting Biomolecular Structures Across the Chemical Space

Leave a Comment / Research Highlights / By Winston

On October 9, 2024, the Royal Swedish Academy of Sciences decided to award half of the 2024 Nobel Prize in Chemistry to Demis Hassabis and John Jumper for their development of AlphaFold2 in 2020, a model capable of predicting the structure of almost all 200 million proteins discovered by researchers. Here is the official scientific background: They have revealed proteins’ secrets through

Unveiling AlphaFold 3: The Next Leap in Predicting Biomolecular Structures Across the Chemical Space Read More »

Research HighlightsAI, AlphaFold, Computational Biology, Protein Structure, Structural Biology

The technology itself is neither right nor wrong. Cal Gov. vetos SB 1047

Grade-School Math and the Hidden Reasoning Process

Leave a Comment / Research Highlights / By Winston

Currently, models like OpenAI’s GPT, Anthropic’s Claude, and Meta AI’s LLaMA have achieved over 90% accuracy on the GSM8K dataset. But how do they accomplish this? Is it through memorization of data and problems, or do they truly understand the content of the questions? GSM8K, short for “Grade School Math 8K,” comprises 8,000 math problems

Grade-School Math and the Hidden Reasoning Process Read More »

Research HighlightsAI, Deep Learning, GSM8K, Math Reasoning, Reasoning

How RAG Technology Powers AI-Driven Search Engines: A Deep Dive into Tech Behind Perplexity AI

Leave a Comment / Research Highlights / By Winston

Have you ever wondered how AI tools like ChatGPT, powered by large language models (LLMs), manage to answer nearly any question posed by users, especially in open-domain queries that require extensive knowledge or up-to-date facts? Relying solely on traditional LLMs to generate answers can be incredibly challenging. Here’s why: 1. Knowledge Limitations: LLMs are trained

How RAG Technology Powers AI-Driven Search Engines: A Deep Dive into Tech Behind Perplexity AI Read More »

Research HighlightsAI, Information Retrieval, LLM, Perplexity, RAG

Introducing Graph RAG: A New Approach to Addressing Global Query Challenges in Large Language Models

Leave a Comment / Paper Skimming / By Winston

Traditional Retrieval-Augmented Generation (RAG) systems often struggle when it comes to handling global queries that require summarizing entire datasets. To address this limitation, a team from Microsoft Research and associated departments has developed a novel method called Graph RAG. This approach combines the strengths of graph-based indexing and query-focused summarization to enhance the ability of

Introducing Graph RAG: A New Approach to Addressing Global Query Challenges in Large Language Models Read More »

Paper SkimmingAI, GraphRAG, LLM, Microsoft Research, RAG

Scaling LLMs for Long Texts with FocusLLM

Leave a Comment / Paper Skimming / By Winston

In traditional Transformer architectures, the computational complexity grows quadratically (O(L²)) with the length of the sequence, making it resource-intensive to process long sequences. This high demand for resources makes it impractical to extend context length directly. Even when fine-tuned on longer sequences, LLMs often struggle with extrapolation, failing to perform well on sequences longer than

Scaling LLMs for Long Texts with FocusLLM Read More »

Paper SkimmingAI, FocusLLM, NLP, Transformer

Averting Model Collapse: The Importance of Human Data in AI Training

Leave a Comment / Research Highlights / By Winston

Over the past year, the explosion of generative AI has flooded the world with AI-generated content. As we move forward, training future models on this machine-generated data seems inevitable. A recent study explores the potential issues when large language models are trained using data generated by other models. The researchers found that as training progresses

Averting Model Collapse: The Importance of Human Data in AI Training Read More »

Research HighlightsAI, Generative AI, Model Training, Perplexity

Maximizing AI Potential: From Adoption to Organizational Transformation

Leave a Comment / Thoughts and Reflections / By Winston

As generative AI took the spotlight last year, 2024 is emerging as the year of large-scale adoption. While some fear job displacement, others are already using AI tools to enhance productivity. Today’s AI excels as an assistant, but it’s still far from replacing human innovation, complex reasoning, and interdisciplinary integration. According to a recent McKinsey

Maximizing AI Potential: From Adoption to Organizational Transformation Read More »

Thoughts and ReflectionsAI, AI Adoption, Bussiness Transformation, Generative AI, McKinsey Insights

Enhancing AI Output: Understanding Prover-Verifier Games

Leave a Comment / Research Highlights / By Winston

As Large Language Models (LLMs) continue to evolve, their increasing complexity has brought a new set of challenges. One significant issue is the generation of outputs that are often vague, ambiguous, or logically inconsistent. These issues make it difficult for users to interpret and trust the AI’s reasoning. In response, OpenAI has introduced a novel

Enhancing AI Output: Understanding Prover-Verifier Games Read More »

Research HighlightsAI, LLM, OpenAI, Reinforcement Learning