How I use LLMs to learn complex topics
Discussion 509 comments
And then I begin to think to myself that I should just read a book on the topic written by a trusted source who put a lot of effort into teaching the topic properly and presenting the information in a thoughtful way. So, I am back to books and mostly try to use LLMs to clarify certain questions or ideas I have.
For example having an LLM summarize a dense topic and to find books so that you can filter faster and spend time reading those books works way better than having the LLM summarize the books or the topic (or even relying on second hand information). Another one is having the LLM quiz you on your topics of interest. With questions tailored to attack specific areas that you struggle with. Its wonderful at this, nothing I've used comes close to what an LLM can do here.
You define for yourself what your goals are, slowly refining them as you learn more, and use LLM as a tool. This ,I find works best for learning.
It's long been the case that the best way to learn something is to teach something.
Which is pretty unfortunate for those that want to learn. I used to enjoy writing documentation at work, it was my favorite part of the job. And it did feel like it benefited me more than it benefited all the people that were (or weren't) reading my documentation. Now I can't really justify spending much time on docmentation when LLM's can do it in a fraction of the time and it's "good enough"
This is a great idea. I'm going to try it.
And you notice when it's a topic you know well or something like software where you can immediately tell the options it's giving you don't exist on the page. Leading to the amusing statement "LLMs are bad at what I do but great at everything else".
The question: what's the net positive gain of turning people who know nothing in a given field into sub-novices, while weighing actual experts down with work slop and marginal returns?
And I wonder what the true cost is of arming so many novices with that level of dangerous knowledge.
Tangentially, but related: I'm old enough to remember when the spirit of your comment was pervasive on HN.
Just hoping folks don’t get hurt due to people not understanding what they’re doing with these things but believing they’re competent.
This statement would out you as someone who didn't attend an elite school.
The body of literature on learning theory, and beyond that on specific types of learning and specific mediums such as learning from text is so rich there are way more useful models to draw from. Believe it or not, prellm, researchers in the textual learning field had already demonstrated you can achieve performance equal or better than novice tutors using pretty basic computer aids that follow specific hint/pump interaction structures. Guiding an LLM to use these findings has evidence backing it and is way better than telling it "i guess be like socrates". The problem is, to realize there might be richer more effective and highly researched ways of tackling the problem beyond the first fart of a thought you had one afternoon requires the deep respect for expertise and specialization that precisely basically everyone in the AI space right now fundamentally lacks.
You need much more time and guidance.
Trying to diagrams/animations didn't yield good results even with frontier models. But pure text, any model does a decent job.
So it may be very slow or become unavailable, back end can't handle that, no caching whatsoever.
I'd imagine an application that uses LLMs will be created that better manages learning. It's just not clear what that UX is yet- it's obviously not just a chatbot
i run into context window limits, or practical limitations of digitizing the book
human conceptual thinking is very much a multi-dimensional graph, which relies on light "approximate" concepts that are "good enough". LLM AR token generation is extremely one dimensional and doesnt care about the "weight" of the concept behind a token.
LLMs hold billions of parameters in "mind" at once. humans hold like four "concepts".
This is the essential mismatch and the primary reason LLM conversation can be so painful and exhausting.
Explaining this and limiting "concepts" to four at a time tops is one of the very few AGENTS.md / system prompts I always use, and it has proven invaluable time and again.
Thinking traces show how effective this is at forcing the LLM to simplify its thinking.
[edit] Also, myself and nearly all of my peers are struggling to choke down the flaws of LLM tooling along with the benefits. the speed at which LLM adoption is being forced, without truly crafting them into quality tools first, is not ok, and not normal.
LLMs have stirred an inhumane hunger and fear. the tech is fine, but the way tech companies (creators and consumers) are behaving should be deeply questioned.
it's NOT normal. it's not ok.
The full PDF is worth a read (Figure 1 may be of interest to many here): https://www.cambridge.org/core/services/aop-cambridge-core/c...
If "attention is all you need" then it's something we do indeed lack, in comparison to LLMs! But it's an interesting question: might machine cognition benefit from similar bottlenecks in an attention algorithm? Advancements like Kimi Linear seem to indicate that we're far from the finish line: https://arxiv.org/abs/2510.26692
I just want to understand more.
Also, would you be willing to share the actual text of it that you put in AGENTS.md?
Personally, I find that its generated prose tends to have an undue weight to it, almost as if every topic I ask about somehow bears a heavy burden, or is otherwise load-bearing, to use its parlance.
Quite puzzling, really.
I think this is one reason why LLM text is pretty exhausting to read for long stretches.
It's possible that this quality you describe stems from the extensive training corpora utilized by the major AI labs. These almost certainly include work from the esteemed economist Jacob Silj:
I have a personal theory: LLMs are *fundamentally* handicapped at perceiving what's going on in the mind of the human (this can't be "innovated away") and that's at the root of what makes them suck at conversation.
Next time you're chatting with someone, notice how much understanding is shared without anything being said. E.g. the other person might share something deeply disappointing, and they can tell without you even saying anything whether you get what they're going through. This unspoken-yet-communicated information guides the conversation. Or as another example: humans can read the room -- you walk into a room and immediately adjust your demeanor based on what you see and sense.
LLMs are totally blind to things like this, and this adds an inescapable awkwardness to interacting with them. I don't believe they'll ever grow out of this. Which thankfully implies more long term demand for humans instead of robots. :)
Does anyone have a read on if this is primarily a Claude issue, or if all LLMs do this?
> I get exhausted reading LLM prose
So much this! If I see one more sentence with the words "genuinely" juxtaposed with "load bearing" my head is going to explode!
btw, I am building the tutorial here for anybody interested in this topic: https://github.com/avilay/learn-probml
it also researched vision correcting displays for me and i can finally put that idea to bed - i was never really going to pick up an optometry textbook tbh. plus it was able to pull together a bunch of geometric and physical context about light and the eye plugging exactly my personal knowledge gaps.
in general i suspect these materials might not be that interesting to others because they are so custom to my learning style and personal needs and preferences.
these are usually not one shot documents but rather many prompts deep before i get something I’m willing to sit down and read or study. but dramatically quicker than assembling it myself from primary sources. i wouldn’t say it matches master expositors but then they’re not available to write on any topic i happen to need right now.
plus I’ll just have a live voice discussion with the system when i go for a walk and there are still things bothering me on a topic. it takes a little patience but if i’m in the mood it’s amazing.
i generally find that it can help track down specific references if i suspect hallucinations. but especially on factual topics my experience so far has been extremely encouraging.
I view LLMs in education similarly to office hours. Some people abuse it to get homework answers without grappling with the material, but the optimal amount is not zero.
LLM certainly not a replacement for a book, where you get someone’s extended personal approach to a topic, thoughtfully organized, reviewed and edited, often times actual courses taught based on it, with answers checked and errata available online.
Perhaps the best example has been a native macOS app that is a completely custom text editor with built-in debugger, lsp support, fuzzy finder, etc stuff you'd expect. Inside the same app is a library of books i can read within the app completely formatted and for every chapter/section of each book that is a quiz to take (LLM generated of course), a "recitation" tab where i am asked a question and say outloud my response to the AI to evaluate me on and then finally practice problems to do within the custom text editor (these are usually programming books). The reader also has ai re-write built in.
As neat as this is, and i worked through K&R like this, i have ultimately fallen back on "just read the damn book and go to the AI when you've got questions."
So I have the LLM offer a very short explanation of something, and from there's it's just me asking questions. Anything that feels fuzzy or not fully internalized is something I poke at until I'm satisfied.
It really has helped me develop a sensitivity to what I understand vs what I don't, and the ability to drill into any part of it is amazing.
And yes, it is not that it is just presenting the facts. By me taking control of the direction the questions and answers go, I can flesh out my mental model. I won't retain every little thing it tells me. But I am much farther ahead than before.
But even with Claude, it's it's really the prose getting in the way you can install the caveman plugin or tell it to use that "standard technical English" thing.
Any more detail you can share? Do the others feel more "human"? Are there any that are particularly digestible/human-friendly?
I've been wondering for a while if this is just Claude because I mostly use Claude, so this is very telling.
I tried using a new agent service recently and could tell immediately that it's powered by Claude due to the way it writes.
You can either install that skill or put the Rules section directly in your Global CLAUDE.md for Claude or Personalization setting for Codex and it should cut down the output verbosity by quite a fair bit.
I'm not sure I follow how this is actually guaranteed? The fact-checking process mentioned just seems to involve asking AI to review its own work.
Personally I think this is a bad characterization of using LLMs to fix up LLMs because while you can never guarantee results this way (as the quoted line claims here, which is worthy of criticism), it is, in practice, useful to use LLMs on top of LLMs. And there's no infinite regress. Auto-mode in Claude Code, for example, seems to me like it's been successful at making the system more safe than --dangerously-bypass-permissions without prompting the user for permissions constantly.
What triggered my response was the “just review the output with another LLM and it’s perfectly correct”
“"The turtle moves," said Didactylos. "The turtle is a giant reptile that swims through space. It doesn't have to stand on anything. Swimming is what turtles do. The idea that it has to stand on another turtle, and that turtle has to stand on another turtle, is just silly. It's turtles all the way down, and that's a logical absurdity."
Small Gods, 1992
Full story in the book
People don’t even have to be lying to be wrong about this stuff. Someone can learn enough about a topic to be halfway up Mt. Stupid in no time flat, and in doing so, think they not only truly understand the topic at hand, but might be particularly adept because they were such quick studies. People that know less are impressed, because why wouldn’t they be? Anybody that knows more than them sounds like an expert. And people that know what they’re talking about cringe at the overconfidence, and probably try not to engage: who wants to have to prove that someone’s boundless confidence is entirely baseless? Most of the time, they think the actual expert is full of shit because they think they’re the expert. It’s incredible how many times I’ve had people in tech confidently, even smugly “explain” design concepts and strategies to me that they did not actually understand, knowing I was an experienced, degree-holding designer… and they didn’t even have a chatbot’s lips on their ass telling them how smart and insightful they were.
Opus 5 first built me a detailed plan, but a couple important details were either obviously wrong or felt unnecessary. I went back and forth asking for sources and more information probably like 4 times and every time it did the "in looking at things in more detail it appears my previous advice was incorrect" spiel. It just became exhausting at some point because it feels like it really lays bare how LLMs are just minimizing that loss function but don't actually "understand" anything. It was really useful as a search engine (it correlated some highly relevant source docs), but I just couldn't trust it to believe it was actually done at any step.
I know it sounds silly but 1 layer ends uo being way worse than 2.
i can't even get agents to remember core instructions like "use jq instead of writing a python script to parse some json"..
In particular:
- I limit it/encourage it to give me single sentence questions
- I sometimes will ask it to tell me a motivating, human-grounded story, when we're starting a new concept: claude responds "Maya is a bond portfolio manager, and her boss has asked her to quickly price in what happened if yields go down. She knows her bond's average duration, a measure in time, but she doesn't have a percentage, which is what her manager wants. How can she give him a percentage number with just a duration figure and the proposed new yield?"
- I'll often ask claude to let me work through it, to derive the thing myself, often resulting in a string of thoughts with "yes/no" trailers, to get the LLM to reply yes or no only, and avoid derailing my train of thought. If yes, my train of thought keeps going. If no, I've got something wrong.
- I'll sometimes stop and have it craft an artifact. I typically say "build me a Brilliant.org-style interactive demo of the topic", especially when we get into the realm of looking at the actual maths of a thing (for which prose and dialog is not optimal by itself AFAICT)
- I'll do this while I'm traveling, while I'm walking, while I'm doing chores.
It's so much fun.
""" In this project, I require a socratically delivered line of conversation. Here's the typical structure to the conversation. I ask some question. You need to factor and reason about how to conduct and deliver a conversation. Best practices would be to limit terminology, or assess with the user whether they have a firm grasp on terminology before you use it. You must be very strict about this, it's unacceptable to just introduce a new concept, actor, phrase or other complication into the conversation without first labeling who what or why it exists for the conversation.
Conversation structure needs to be front-loaded with a brief interview for the user, "you understand X?", "whats your understanding of Y?".
Conversation structure then needs to proceed with single-sentence questions from the agent. User replies with an answer. Sometimes the agent needs to correct the user, but only ever do so with yet another question. """
^^ these are the instructions I have installed at the root of a "project".
Keep in mind, this is claude opus 5 low effort we're talking about, in the "projects" area of the mobile app. Here's the process I use to set up its knowledge:
1. I take screenshots of the textbook on my iPhone, and upload a chapter at a time.
2. I have it summarize the chapter into markdown by analyzing screenshots. You could probably achieve this simpler, if you just had the textbook in PDF.
3. I walk with my boy Clau-crates.
I've done this for a couple of weeks and haven't seen it revert back into its typical context-dumping behavior.
On second read, there's probably some clean up I could do. Thanks for making me pull it out and look at it. Things that could probably be improved:
- tell it to cross-check resources online to further ground itself
- use simple, short sentences (long sentences make the brain blur a bit)
- not be sycophantic (it seems like project mode has discarded with my root-level anti-sycophancy prompt)
I'd be extremely cautious to ask it to have it explain any specialized concept even from a document.
I still learn new stuff, but I’m afraid it won’t have any value in a year or so.
For example, I’m pretty good at optimizing low level stuff, but right now you can just ask LLMs to do so and they are pretty good at it. They will profile the code and suggest reasonable options like 90% of the time.
Trust me when I say that in the hands of someone who doesn't have your experience, the LLMs would not be getting the results you get.
You might think what you're doing is trivial, it may be sessions that flow roughly, "Instrument this, okay this part is slow, profile this part, OK read the profile output and suggest a better approach".
But your experience will be steering it in the right direction, and you're probably unaware of just how much your experience is doing that guiding, as the LLM shoots off at 100mph, you feel like it's taking you with it, but you will be guiding it a lot more than you realise, and that's where learning and experience comes in, even if you're no longer operating at the lowest depth, your knowledge of that layer will be helping.
If nothing else, the experience to know when something is actually slow is a skill in itself. If a function takes 200ms, sometimes that's as quick as it can realistically go, and sometimes that's literally a million times slower than it could be, and there's actual skill and experience wrapped up in knowing what "slow" looks like.
https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/23049
So there's this 11 year old issue in a forgotten ffmpeg plugin and I fixed it with deepseek by putting it into a self-testing loop. Probably would have taken me a few weeks to even understand the initial code to begin with. I haven't done C work in a long ass time and have zero knowledge of even what sub pixel sampling means.
With DS4 took me a few days and a couple of dollars. And by few days I mean I checked on it a few minutes every half hour or so a few times.
I don't understand the code it wrote but it's been in production for a while now and no issues so it's good. Ended up speeding up our video processing pipeline by 20-30%.
Sending in the patches but refusing to take responsibility for them is a surefire way to contribute to maintainer burnout. Please don’t do this. Either commit to fixing something and driving the PR to merge, or abstain from it entirely.
The bottleneck isn’t the speed of coding, and what you’re doing here is actively worsening the situation.
But you can see how the parent's comment doesn't really hold, I was able to achieve this while not knowing anything other than what I need fixed and making the LLM test itself towards that goal.
And the craft is loose term, it can mean anything you like to get better at.
“asking the right questions” is also a moving target with each model release
People simply underestimate the value of doing the work and think that the end result is all that matters
https://en.wiktionary.org/wiki/eat_one%27s_seed_corn#English
What is fascinating is how you can witness it at so many levels of organization. One example: Employer executive get enamored with moving from labor to capital. They believe that by using LLMs, they can replace a lot of workers. At my place of employment, we have people that are surprised they can't file a Jira ticket describing a product ask, and have it kick off an implementation. You can build the skill to attempt that, but invariably you'll get back questions like "what do you mean by <x>" and "what do you want to do in this case, a, b, or c?"; questions that a product person or an exec are not well suited to answer.
In the past, programmers did that kind of interpretation and judgment call. So then you're in a quandary; who should do that work? Work that previously, you never imagined was an inherent part of what the replaceable code monkeys do at your beck and call?
And then, how do you hire for that? How do you find the training for the people that are experienced enough with... something... to know what a cohesive error response is, or what kind of telemetry strategy is best for that particular product and organization, what collection of product asks are incredibly complicated for what they're asking and can deliver 95% of the benefits at 5% of the work if we just do this instead, and whether you want to aim more towards thick or thin clients?
Who are those people? Wait, those are programmers? Wait, there's this whole collection of inherently human skills that we devalued, by not appreciating they were always quietly doing that for us in the past?
That's just one example. There's a repeating pattern of discovering where the work truly is, work that was embedded in manual patterns we might not have to involve ourselves with anymore, but is yet still essential. So the nature of our jobs changes massively, but the overall level of employment does not.
At least, not in the medium to long term. There is a lot of painful churn we have to suffer through first.
I wasn’t even really concerned with optimizing low level code before LLMs and that wasn’t why I was hired either.
However following that low level thread: We can look at the reasonable options and immediately know if they’re reasonable or nonsense. Why? We know the code. Now zoom a level out, where I think our expertise really lies.
Building a complex system isn’t easy. There are customers with requirements, there are budgets, SLAs etc. Sometimes one customer needs X and one needs Y. Our expertise is taking all of this in, and producing something that balances all the different variables. It’s knowing that we’ll expect X events a second so we’ll need Y to ensure we can tolerate failure.
Is it possible LLMs will be able to do all of that too? Maybe. But then why would our customers need the enterprises they pay for?
I have stuff to do now, the value of the knowledge in a year or two isn't important if it solves the issues I have today.
I'm not 'wasting time' but I'm also not really learning.
The more things you understand, the higher the chance you'll spot a situation to use them in the future.
I think the best innovations come from times when someone is uniquely able to combine two of their previous experiences together. The more experiences you have in your back pocket the more combinations you have access to and the more likely you'll have a unique combination when the right problem comes along.
They tell you have "hit the nail on the head" when you really haven't.
They tell you have had a "great insight" when you are really haven't.
They give you the illusion of learning and progress but essentially give you faulty preconceptions will trip you up further down the road.
You can ask the LLM to be more critical and less sycophantic but that only gets you so far:
They want you to continue using, being dependent on and feeding data into the LLM--your independence isn't a priority.
In my experience it's infinitely easier and faster to learn deep, "boring" things when you understand how they relate to your shallow and wide understanding of all of the related components.
The LLM is merely a tool. And you can use it for domain discovery that enables efficient deep learning at an unprecedented rate or you can develop a cursory understanding of a topic and think yourself an expert.
It’s true of most things. Running, dieting, weightlifting being uncomfortable is a sign of progress.
Learning is uncomfortable. Reading a difficult (for you) text in a language you don't understand is exhausting and confusing. But that is where improvement happens.
Its even worse since Duolingo added a life system (not sure if they still use it), where you were only allowed to make 3 mistakes before having to recharge your energy. If you get everything right, you are not learning, you SHOULD be making mistakes constantly. That shows you're actually being challenged.
One of the teachers on the college I went to had a note on his door with "If you understand everything you're doing - you're not learning anything.
This is the only path to mastery, or understanding if one prefers. There are no shortcuts to a person achieving deep understanding (a.k.a. "Aha!" moments).
Can a tool such as GenAI be beneficial to someone who already has done the work to understand? Absolutely. But it cannot infuse mastery into a person simply by its use.
Only the time and effort a person devotes can do that.
Another useful approach has been asking Codex to implement complex things, like a Kademlia DHT or BitTorrent client in a literate style with the explicit purpose to increase understanding by reviewing the source code.
Examples: https://rickcarlino.com/notes/note-dump-and-ai-summaries/ind...
Makes sense.
> I ask it to review the accuracy of the knowledge base it built in the previous step.
Ooookay that sounds good.
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation.
wat.
The main idea is you can do any style you want or like to learn.
I can appreciate using all of the tools at your disposal to learn a new topic, and in no way want to discourage learning. I've used LLMs myself to question my own understandings and it can be helpful.
However, "... 100% accurate and free of hallucinations." isn't a statement someone who just learned the topic is capable of honestly stating.
Leaving aside the whole can-we-trust-LLMs aspect, the ChipTycoon page is not really a simulation, and the animation doesn't actually add anything. I like the author's intent, but there's a lot of work to do still before he makes this useful.
Surprised no one's dropped a link to Bret Victor's https://worrydream.com/LadderOfAbstraction/ ("A Systematic Approach to Interactive Visualization") yet.
That's actually a fun way to learn processes!
Totally agree, unfortunately careful simulation games are very rare
Otherwise you probably get more confused as you have mentioned.
On the other side, Peter Diamandis describes a situation where a bunch of kids were given a internet-connected computer and they had no teacher. Instead of it there was a “grandma” that checked kids from time to time.
After that there was a knowledge test that revealed “no teacher” approach was more efficient.
But it was a group, not an individual activity…
For example, “explain how the code in this file works,” I am familiar with the overall codebase, I know the purpose of the file, and I can read it or write tests to verify if I suspect what it’s telling me isn’t correct. Or, if it’s really important, I can overcome my introvertedness and ask the team member who wrote it…but that’s a last resort nowadays, which I am very thankful for. In 99% of cases since at least Claude 4.2 days, Claude and Codex have been very accurate. Gemini on the other hand messes up more frequently and sometimes does weird things like try to delete files it’s not familiar with, at least the 3.6 flash model I’ve been using lately does this. But, code explanations are still good for the most part.
I don't see nearly enough discussion on the distinction between LLM and human reasoning, where the boundary lies, that type of thing. It's like we've all collectively buried our heads in the sand and accepted that they'll eventually be able to think exactly like us.
I don't really see the point of asking LLMs to summarize something for me when I can just read about it myself.
I will do chats back and forth about specific topics, but then always ask for follow up resources I could dive into.
This isn't fool proof - I'll sometimes get resources that aren't really what I'm looking for - but it feels better than searching the web.
Especially when I don't have a clear idea of what search terms to look for.
I haven't tried things like the Socratic method though or having LLMs teach me something! I've really just been focused on reading lately.
You can ask the LLM how to do this. Start with a topic you know well to get the mechanism working and trust it well.
I assume this will become less of an issue in the future as there is more trust between the AI tools and me.
How does he know?
> Time and time again, when talking to people who rely on ChatGPT, Claude, Perplexity, and other general AI tools, I hear them say, “AI is incredible. It handles nearly everything I throw at them.”
> “What does it fumble with?” I’ll ask.
> “Well, it still gets things wrong when it comes to my line of work.”
https://www.dbreunig.com/2025/04/08/on-ai-observational-comi...
> I proceed asking it to build a simulation of that topic in a low-poly, Rollercoaster Tycoon-like animation
Sounds like the Rollercoaster Tycoon part is just referring to the aesthetics/graphics, and the author is just suggesting building a top-down isometric 2D animated simulation (which I agree is a bit limiting, there are definitely some things where you'd want to be able to fly around a 3D space in first person, move time back and forth, manipulate parts of it).
Learning from a book is still the best... Although I have been told that learning a complete subject from a book is now an inproductive use of my time... Perhaps they are right, but I still do it.
The approach that works for me is using Justin's skycak methods he mentions in his books:
https://www.justinmath.com/books/
Check the shorter "Advice on upskilling" or "The Math Academy way" for well researched approach.
So what works for me
- open a project in ChatGPT/Notebook LM - dump all the relevant and highly cited materials (textbooks, papers) - dump the advice on upskilling text or a short summary I've written for the LLM
- create "Learning Goals", that contain what I want to learn, and how to estimate is my level good enough
1) Ask it to create a learning path from the materials, following the approach. Give that to an adversarial LLM for cross check. (just for sanity check)
2) Ask it to create an "entry test" to check what I do know and what I don't
3) Iterate step by step on each module/submodule from the learning path that intermingles the approach of: small theory step + small practical task + small test. Log what's missing/wrong in my dept log. Give the dept log at the end of the session to the LLM to incorporate/create another test/task.
What I have found useful in this approach is that it will generate a lot of practical tests/tasks for me and it will explain a concept in many ways until I understand it. Also it finds some prerequisites I might miss, but based on my tests and debt log unexpected things I thought I understood surface.
So with the limits of LLM and while building a mental map of the relevant parts it's usually enough to spot the hallucinations, but if you apply structured approaches these are minimal. And it's super good, because the number of practice tests and explanations is endless.
The interfaces are a bit clunky, but current multimodal LLMs are ok with images or even hand writing.
I will recommend that structured approach.
Even if you know everything about bicycles their mechanics, components, and how they work you still have to learn how to actually build, repair, or ride one through practice. Knowing, understanding, learning, and practicing are completely different things.
There are also niche areas of expertise that can take years to develop,not just to the point where you know the terminology and jargon, but where you understand the nuances of the field, can recognize the "unknown unknowns", and eventually have the ability to push the boundaries of existing knowledge. Maybe that is what we should really call learning: not simply acquiring information, but developing enough understanding and practical experience to contribute something new to the field.
Ironically, on the same front page of HN, there is a post about Andrew Wiles and this. I don’t think I would be able to comprehend Fermat’s Last Theorem, the Poincaré Conjecture, or Gödel’s Incompleteness Theorems, even with the availability of LLMs.
I can keep asking LLMs to explain a complex topic until I get it. Ask to explain it 10 different ways, explain it using physical analogies, explain it using visualization. If I don't get it, just say that out loud so that they can keep explaining it to me in different ways. We can keep going that until I really get it. That is the value I get the most using LLMs to learn things, especially complex topics.
Or you'll be confused by the slop it generates, which is very conversational but may be just false. Or - you could "get" something wrong and believe it to be the truth because the AI said so.
Colleagues often suggest podcasts and videos - I very, very rarely listen to them or see them.
The bandwidth is too low. It's not efficient and ultimately I'm bored.
This is a nice project, it looks cute. I watched some of the pages But I want more than that, more information, and faster - still a Wiki fan.
Also, step number 2 in the flow: have the LLM check itself... Naah, I don't believe that.
But you're not the only using gen ai like that. Take care.
Indeed it's often a waste of time to just focus on talking people fully if you want to learn fast, reading and especially deliberate practice are better for that. But if you don't have the time, energy or focus, then listening to interviews in the background can be useful supplementally
The skill then riffs with me, judging my ideas and suggesting alternatives. We go back and forth until something useful comes out of it. This process isn’t unlike how I do normal development.
However, once agreed it breaks the work into “steps”. It then creates a tutorial for me, for those steps, explaining each line, why each change happens etc. I can then ask questions, muse about an alternative idea etc. Then I do the steps, and I’ve learned and gotten what I wanted to get done.
This has been how I’ve been learning Godot and making a game for the past month or so. I didn’t go in blind, I started with a course from GDQuest so I could feel confident guiding the tutorials. I will say though, having a tutor to bounce ideas off of has been really useful.
I still try to figure it out myself, consult the docs, discord etc. But if I’m stumped I’ll run my tutor skill and have some fun.
I have done this for all my work this week and it works quite well.
For one it lets you actually query the LLM as to why, their plans give a high level not every single change and it allows you to correct it as you go and the plan will change.
As for how useful it is to understand thins, I believe it's still useful and hope it will continue to be.
Of course, you have to be careful with the answers. Especially when the discussions get longer. But usually I see that it is time to stop or to start a new session when the formulae do not make so much sense anymore or when the LLM repeats itself.
But with enough caution, LLMs are really a not-so-bad intellectual sparring partner for discussing ideas and insights.
To be clear, I say "inaccurate" rather than "wrong" in this case because even if the information it returns is factually correct to the question being asked, students don't have an understanding of the complexity of the interdependent tectonic, regulatory, and spatial / experiential factors of a building sophisticated enough to ask their questions of the specificity and nuance necessary to get a good output that addresses the entire problem.
Anyway - with the students still learning to ask questions the right way, and the conditionally-incorrect facts making their learning more complicated rather than less, I hit on a strategy for them to use LLM's that seemed to help much better.
I suggested that instead of ask the LLM for the factual answer, or even better for the facts and an explanation, that they ask it to direct them to the proper place in the source material to find the answer themselves. Then, to treat it like a lab partner. IE:
Hey Claude I'm looking for "x."
Claude: "look at foo, bar."
Thank you - chapter (foo) part (bar) table (goo) says "car." However I notice that footnote (hoo) says there's an exception if "dar." Which is what I have. Walk me through this exception...
It seemed to have good results as a guide to understanding the disparate bodies of knowledge that they will eventually have to keep together in their heads and work synthetically and non-linearly through, rather than just as an external source of blindly trusted authority.
Does anyone else use Claude like this?
It's sped up my learning by 10x. I struggled with 'just reading a book.' Take kubernetes. I hemmed and hawed and spent years periodically reading some dry book or blog or official doc, falling asleep, and forgetting while I got busy. Now I'm aggressively working with it, almost like I'm addicted to a gamification, of getting through our learning timeline, and I'm excited to move forward as quickly as possible and pass its tests.
It's like a fake teacher, because I can also ask it to drill into a topic or re-explain itself if it made no sense.
The only thing that worries me is, sometimes I'll say something like, "Um, are you sure about that?", and it'll apologize and correct itself. I barely challenged it!
I'm using LLMs right now to build a terminal browser, a GUI browser, and a PyTorch/LibTorch replacement. It's really fun to be able to learn and make progress this way. It's like reading multiple interactive books, where every concept can be explained again and again until I understand it.
Hallucinations bring to question what you think you've learned. That's going to cost long-term if you labor under mis-apprehensions until you maybe figure out you learned something wrong.
Yes, sometimes its wrong, most times its right, cross checking is fairly easy, not using it because of the possibility its wrong seems a baby with the bath water thing.
I learnt a lot of functional programming from it, stuff I've always wanted to learn, but just didn't have the time and really the sources can be difficult, it really explained things well, and as someone else said in this thread, you can ask questions over and over until you understand, asking a person that (if you can get an expert) would drive them nuts. Maybe my experience isn't typical, its hard to tell, everyone reports something different.
In my experience, LLMs are really good for taking up your time and making you feel like you're learning, in the same way that many of the popular educational videos on YouTube are fun to watch and don't really teach you anything.
If you ask an LLM to give you a 500-word summary of quantum physics, it'll give you an oversimplification that probably leans on a hodgepodge of pop-sci metaphors. And if you start drilling down, you risk drilling down on these ELI5 metaphors, which can get you farther away from truth.
I've spent a lot of time with LLMs for the last two years. Something I've tried, almost for decades, is to learn enough CUDA programming to be productive with it when needed. About 6 months ago, after again banging my head against it for weeks, something finally clicked and I feel like I've overcome the initial step of at least grokking the needed ideas so I know where to go next, and I can actually write + compile + use kernels made for my use cases. I won't claim to understand everything, but I couldn't do what I can today, before I learnt the things I now know.
~2 years ago, because of my very weak math foundation, I basically said "Well, CUDA looks really interesting and really fun, but it's too difficult, lets focus on other things", even after reading some starting resources and stuff. But, by asking countless of dumb questions to LLMs, forcing it to steer me in the right direction, when I'm otherwise just driving on the highway or what not, I finally feel like I have a grasp on something I earlier only dreamed about understanding, and I'm able to be productive with it now.
Stuff like Triton, nvidia warp (the language), numba, cupy jax/pallas and so many others really paved the way. You can start out really high-level, run a profiler and then dive deep into the bottlenecks.
TL,DR: Keep going, it's a great time to have fun with GPUs.
Well, yeah, but what I've being doing is learning proper CUDA, not "Python-compiled-to-CUDA" (otherwise it'd take like a just a week to understand enough :P ) and that's looking more or less the same today (although bunch of more complicated stuff piled on top of the fundamentals) as it used to, AFAIK.
With that said, the environment is a lot simpler to setup today at least :)
In [0], I ask: „When applying Hidden Markov Models to POS tagging in NLP, what do the latent states and observations usually represent?” I then follow up with some specific questions and requests for walkthrough. You can’t see it from this conversations, but I have Wikipedia and a bunch of other resources open in separate pages, cross-reading, and I follow up with a handwritten toy implementation of a Viterbi-based POS tagger once my mental model crystallizes. This is very different from a 500-word summary of quantum physics, and I still had to put in effort (this is unescapable!), but I found the experience rewarding. Also note that this is relearning of a topic that was part of my uni curriculum but long forgotten.
In [1] and [2], I’m learning Spanish by reading García Lorca’s poems. Here again I’m going through the texts with a dictionary, and augmenting my learning with what a dictionary won’t tell me: given the usage of a word or phrase in this specific poem, is it something that could occur in everyday speech, or is it poetical?
[0]: https://chatgpt.com/share/6a743bc7-d0dc-83eb-acc9-8f2faaffc4...
[1]: https://chatgpt.com/share/6a731be6-26bc-83eb-8d21-c965da5364...
[2]: https://chatgpt.com/share/6a731c01-5604-83eb-a5b8-cd1295d0ef...
- Getting through textbooks and lecture notes. LLMs have gotten very good at answering basic questions on quite advanced material (e.g. representation theory and quantum field theory). By asking a very specific question or even giving the LLM a screenshot, I can get unstuck a lot faster.
- Learning e.g. new python packages. Instead of hunting for examples on Stack Exchange, now I ask an LLM to write a minimal working example and then build off of that. By writing most of the remaining code myself and only using the LLM to answer questions, I've been able to learn new packages significantly faster.
In both cases, the LLM isn't providing the curriculum or guiding what I learn. The textbooks, papers and coding tasks are. But now I can pick these things up much more efficiently.
The other day I realised I had no idea how DNA and life works. I guess I studied it at high school (25 years ago), but maybe it didn't go into much detail or it just didn't click.
So I asked ChatGPT to explain it to me, I came up with my own mental model from it's explanation, told it that, then it corrected me where I misunderstood things. We went backwards and forwards for an hour, me asking questions, it correcting me, until I felt like I understood the whole picture.
Am I going to become a biologist and study the origins of life from that? Definatley not! But if my kids need help on their biology homework, I now understand the basics of it.
If you're only checking your understanding against the one source you used to obtain it, how can you tell whether your understanding coincides with reality (or rather, with general scientific understanding), and not just with the source you read? And I'm not asking just about ChatGPT; the same question could apply to any source. Books are not exempt from containing errors.
Turns out that the author had done a thought experiment but neglected to factor in the rotational inertia.
So, I think the point OP is making is that most people don't really check sources while learning things 'the conventional way'.
EDIT:
I think it's a kind thing you need to tune yourself into. OTOH, I've observed many (most?) people seemingly being completely oblivious to self-consistency issues of their beliefs and mental models, or even texts they're reading or instructions they're following, and yet... somehow they're generally more successful at life because of it ¯\_(ツ)_/¯.
Aka confirmation bias.
We like explanations that fit what we expect, even if they're completely wrong.
Its easy enough to prompt ChatGPT for primary sources when doing research to validate any claims its making.
> I want you to create a tutorial series about X for me. The prime objective is that I improve in topic X so never provide a solution but guide and teach. (for programming never write code). First create a question catalog to assess my current level.
Then I would ask it to structure the tutorial challenges in the following way: - Goal - Concept - Instructions
I figured that if I don't need to read any additional material on the topic the LLM is giving me too much information and I need to change the prompt. Works for me and I used this too learn topics I feel now comfortable with, like nushell, opencyper, elisp, boot loaders etc. But maybe you don't consider this "complex"
From the hilarious "There are 3 ways to learn. (Knowledge Fight Animated)"
https://www.youtube.com/watch?v=NsqZZiWDHAQ
If today's top LLMs are reliable enough (without grounding) to academically learn "complex topics" from, may be I need to adjust my priors. I must say, I do find myself chatting about other topics (without the need for grounding) that I'm trying to "absorb" (not really learn), like Behavioural Psychology & Philosophy.
[0] Products like NotebookLM are built specifically for such usecases.
LLMs give me structure based on my current skill level. And basically always I accompany this with books, I love reading. It's a nice combination for me.
You literally proved the OP's point ... thinking you're learning. More like scratching the surface, with lots of invalid data while not being able to recognize what's invalid.
It's like with latest vector of attacks being spamming Github with malware injected in proper looking code in hope of AI to index it.
Then you paste the code because you don't understand it, but you take it as working and only doing what you've asked for.
It's about guiding me in _doing_ exercises so I learn and I can evaluate if I learned something if I can apply the learning myself.
Dunno, I've been learning a lot of Rust in the past few days. Just dove right into a project and asked AI to teach me stuff on a need to know basis. I'm actually getting used to Rust by now.
It takes way more time to master and be very comfortable with a language due to its ecosystem, though. Some languages are more likely to click with a person, yet underneath they are all the same (minus the functional languages that form their own group), e.g. some performance issues may require looking at the generate assembly code.
However, saying you can't learn "anything" is just too strong. I'm definitely managing to distill the AI's weights into my own brain.
Additionally I asked it to also give me problems relevant in my business domain so that I learn how to directly apply the knowledge in a realistic scenario.
I am getting much more comfortable writing rust than I was barely two weeks ago. More than I was just reading tutorials.
The truth, as always, is in between. There’s loads of people using it for useful things, learning, automation and getting good results. But it’s also wrong enough that you need to deploy it carefully sometimes.
Don’t worry about either group. Keep objective and use AI where it helps and do it yourself where you are better. That’s all.
As a tangent, I think the concept of pop science has wasted so much time of what could be considered brilliant minds. I can't believe how much YouTube people I consider really smart consume under the guise of "learning stuff." And the videos are always designed to be addicting and to entice you to watch other of their stuff, which makes sense, because theyre a business, not a school.
I'm guilty of wasting time on YouTube as much as anyone else (I like watching stand-up routines and Red Bull extreme sports) but I am never under the guise that I'm doing anything productive with my time. Its okay to have fun learning, but I always felt that entertainment and education should be kept separate. You gotta learn something intentionally, not just get it served to you via algorithm.
Note im talking about the educational "shorts" not the 60+ minute deep dives that are basically a college level lecture.
It's the kind of high-brow entertainment that makes you feel like you learn something.
For me, most "push" things are edutainment, whether that'd be Youtube videos or public-broadcaster television programs. Things you seek out yourself are not.
In a similar vein, there's "newstertainment" (news that makes you feel like it's important to watch, but actually changes nothing tangible about your life).
On the other hand, I’m fine with not being an expert on topics outside of my domain, as long as I retain some basic knowledge and fun party facts. So there’s that.