[00:00:00] Andrew: We are
[00:00:00] Andrew: live.
[00:00:01] Nathan: Cool. Twitch and YouTube.
[00:00:03] Andrew: Allegedly
[00:00:04] Nathan: All right, I [00:00:05] guess we'll find out.
[00:00:06] Andrew: how you doing?
[00:00:07] Nathan: Good, good. Bit, a bit busy this month. It's gonna be a lot of [00:00:10] travel, a lot of fun, interesting things. But, it's more exciting than, than nothing [00:00:15] happening. How about yourself? Hmm.
[00:00:17] Andrew: I'm good. In retrospect, I'm really glad we moved to Thursdays [00:00:20] now
[00:00:20] Nathan: Hmm.
[00:00:21] Andrew: it seems like it actually works better 'cause I was gone for... I [00:00:25] l- we've talked about this. I love long weekends like Friday,
[00:00:28] Andrew: Mondays, [00:00:30] and, it-- when you record Friday, it's hard 'cause a lot of times I would either have to [00:00:35] record and then run away and usually be late for something or leaving or something [00:00:40] or in traffic.
[00:00:41] Andrew: But on Thursdays, other than I still haven't got in my brain we [00:00:45] record
[00:00:46] Nathan: Yeah, I was, I was thinking, I don't, I don't know, was there a reason we chose Friday? Just 'cause it was [00:00:50] like one of the days where
[00:00:50] Andrew: I think it, I think, yeah,
[00:00:52] Nathan: just, it was arbitrary, right? You just, we just picked something. Yeah.
[00:00:54] Andrew: [00:00:55] If anything, it's a bad day to stream,
[00:00:57] Nathan: Right.
[00:00:58] Nathan: Yeah. Yeah. [00:01:00] May- maybe I'm, I'm open to changing if we find a better day, if, if the
[00:01:03] Nathan: audience has a better time slot [00:01:05] too.
[00:01:05] Andrew: Yeah, no, same. I, I think, I don't know. Maybe Friday night works [00:01:10] for some people, but I, I feel like most of the people I know who I think would be the audience are, [00:01:15] like, burned out on technology and need a break and are doing literally anything [00:01:20] else other than staring at a screen or they're tr- watching something to just let their [00:01:25] brain subside.
[00:01:26] Andrew: They don't wanna listen to intellectual stuff. But yeah, let's jump into the
[00:01:29] Andrew: [00:01:30] topic. I don't
[00:01:31] Nathan: Is that what we do?
[00:01:32] Andrew: we do
[00:01:33] Nathan: We just chat.
[00:01:34] Andrew: We just chat, just [00:01:35] chatting.
[00:01:36] Andrew: I I think like the, the topic I put up or the title at [00:01:40] least is something I think I, I wrote earlier today and I had an [00:01:45] epiphany to break something really hard down into three sentences that [00:01:50] is, in a kind of a Zen-esque way, very simply put, [00:01:55] but also incredibly complex to try to unwind, and I think that would be a good topic for us to chat through [00:02:00] for AI
[00:02:01] Nathan: Yeah
[00:02:02] Nathan: So I I, think we've h- we've talked about [00:02:05] this topic before, right? It w-
[00:02:07] Nathan: In the same vein, right? It, it was along the lines [00:02:10] of there's there's a changing of the way that we work particularly when [00:02:15] it comes to artificial intelligence, and a lot of people hadn't sunk in yet.
[00:02:19] Nathan: And given the [00:02:20] time that it's taken to sink in, especially with a, a lot of the new developments around what's happening [00:02:25] geopolitically, but also these models that are getting better and better or seemingly getting better and [00:02:30] better it still quite hasn't sunk in apparently. That's, [00:02:35] that's the argument, and we were talking a bit about before this which it'll be an interesting discussion.
[00:02:39] Nathan: I [00:02:40] think we have differing viewpoints maybe, perhaps.
[00:02:43] Nathan: Um, I'm [00:02:45] trying to understand where your perspective is coming from because it, it seems [00:02:50] counterintuitive to me given what I do know of you. And it seems like that
[00:02:54] Nathan: you're [00:02:55] arguing opposite of what you normally do. L- let's go ahead and jump right into it 'cause I think it's...
[00:02:59] Andrew: Yeah.
[00:02:59] Andrew: So I [00:03:00] don't know what I, what I, titled the stream or what the episode title will be, but, if you're [00:03:05] using AI to code for people, or if you're trying to use AI and do [00:03:10] coding that's human readable, you're doing it wrong. And that's-- I will die on this hill.
[00:03:14] Andrew: [00:03:15] And what prompted that wa- I keep hearing...
[00:03:19] Andrew: One of the [00:03:20] biggest detractors I h- I hear from-- One of the arguments that I hear from people the [00:03:25] most who are the closest to the problems, like engineering friends or folks I work [00:03:30] with, is that, AI produces slop. So I [00:03:35] don't-- we've had a, we've had this on the cast before. We talked about it. I don't really [00:03:40] know what slop is.
[00:03:41] Andrew: I think it's just a rejection of a technology that people don't like, [00:03:45] and they're trying to find every single reason to hate on it. And a lot of the [00:03:50] AI-generated art and stuff does suck, I think. It's just not good, and it's not [00:03:55] meaningful. And it's really easy for someone to just put out something where if they have very little [00:04:00] knowledge or low understanding about something to be like, "Ah, this is a miraculous piece of [00:04:05] art," because they're not exposed to the craft.
[00:04:08] Andrew: I do think that any [00:04:10] time you industrialize something, you devalue, what we now call artisanal [00:04:15] work or human-crafted labor, and [00:04:20] that's industrialization. It has nothing to do with AI. It's just AI is now-- It, it's, it's the [00:04:25] pop art effect. We've went through-- There was Andy Warhol in the '60s
[00:04:29] Nathan: yeah[00:04:30]
[00:04:30] Nathan: Really quickly, uh Can we define slop? 'Cause maybe that I don't know, you tell me [00:04:35] if it's relevant to the discussion, 'cause I, I
[00:04:37] Andrew: I was just getting to the coding piece.
[00:04:39] Andrew: [00:04:40] And yeah, and so the... Yeah, so and then so the, the slop with the [00:04:45] engineering coding world becomes, I think, like the way [00:04:50] AI accomplishes stuff with code is not a [00:04:55] way that a lot of people or a lot of teams would choose to work in or choose to do. And [00:05:00] like you know as well as I do, there's a million different ways you can accomplish something with code.
[00:05:04] Andrew: [00:05:05] There's established standards, there's practices, there's things that are easier for humans to [00:05:10] grok versus machines and, or most humans. There's the, the [00:05:15] brilliant line of engineering problem, right? Where you accomplish a solution but no one can [00:05:20] understand it.
[00:05:20] Nathan: Yeah,
[00:05:21] Andrew: and AI does that kind of a lot where I'll even read some stuff and I'm like, "I [00:05:25] don't, I don't get it, but I can also figure out why it doesn't break and I can't figure [00:05:30] out how to break it, so I guess it works."
[00:05:32] Andrew: And that I think is, [00:05:35] is it's a rejection of people who are in the craft against a [00:05:40] tool that minimizes something about the craft that they like. In this case it would be [00:05:45] coding or writing code. ' Cause I don't think it changes engineering. If [00:05:50] anything, it makes engineering more valuable because you still have to build the right thing.
[00:05:53] Andrew: You have to get the right [00:05:55] context in. And if you, if you have a high aptitude for engineering, [00:06:00] you're much more efficient and effective with AI than sitting there and just kind of like YOLO [00:06:05] single prompting stuff to try to make it work. And like eventually given time and you'll probably get [00:06:10] a solution that works
[00:06:11] Nathan: Probably, yeah. Yeah,
[00:06:12] Andrew: Probably. Maybe not. I don't [00:06:15] know.
[00:06:15] Nathan: It's, it's inherent in the, in the, in the generative AI piece.
[00:06:18] Andrew: I very,
[00:06:19] Andrew: very [00:06:20] rarely anymore get stuck where I have to really, really, really [00:06:25] reason through something with AI. I... AI will get stuck and I'll have to [00:06:30] think through how to give it context or narrow the [00:06:35] focus to f- to fix something. Or sometimes I'll have to break the problem down for it, which I still [00:06:40] think is engineering task.
[00:06:41] Andrew: But it very, very rarely is just "I [00:06:45] don't get it. I can't figure it out."
[00:06:46] Nathan: And, and this is in the context of building a technical [00:06:50] solution and you're trying to get the AI to fix something.
[00:06:53] Nathan: Yeah.
[00:06:53] Andrew: something, yeah[00:06:55]
[00:06:55] Nathan: Yeah. Okay. Okay. Yeah, I so the fundamental piece [00:07:00] of having AI needing to write human-readable code, I actually agree with you [00:07:05] there. You, you don't necessarily have to. We were having a discussion before this where I think maybe it was a little [00:07:10] tangential or it was like a, a downstream conversation,
[00:07:13] Nathan: but back [00:07:15] in the early days when people wrote code, you did start with something like [00:07:20] Assembly and you had to write Assembly. I don't know if anyone's seen Assembly.
[00:07:24] Nathan: It's
[00:07:24] Nathan: not easy to [00:07:25] read. It's not human,
[00:07:25] Nathan: it's not human-readable. Exactly. Exactly my point, right? So we [00:07:30] graduated, quote-unquote, "away from it" by writing more or creating more
[00:07:33] Nathan: programming languages that look more [00:07:35] human-like, exactly, but that did eventually get compiled [00:07:40] back down into Assembly language. The, the whole construct of coding and [00:07:45] human readability is something that humans created so that we can more easily work with it. So [00:07:50] I, I think I'm in agreement with you to that point is if you have a tool like AI that [00:07:55] doesn't necessarily need the same human level readability to code, you don't have to [00:08:00] design your AI to write code like that.
[00:08:02] Andrew: Exactly. I
[00:08:03] Nathan: downstream, I [00:08:05] think, you've won on some, on some level. I, I think downstream to that, and, and this is where my [00:08:10] engineering experience is, is bled into this, when you are building products [00:08:15] or when you're working with teams or working with other people, that human readability is [00:08:20] important for other people to understand what you're doing or what someone else on the team is [00:08:25] doing.
[00:08:25] Nathan: The argument here is that you don't need that because you just use the AI to, even if
[00:08:29] Nathan: it came [00:08:30] out in some other weird language, grok it down, right?
[00:08:33] Andrew: right?
[00:08:33] Andrew: Exactly.
[00:08:34] Nathan: [00:08:35] yeah.
[00:08:35] Nathan: And, and I, I don't dis- I don't disagree with that. I, I do think that makes a lot of [00:08:40] sense. What we were talking about before was more around having human in the loop, I [00:08:45] think. At least that's where I was going at it was
[00:08:47] Nathan: if you're building products with code, [00:08:50] it doesn't have to be human-readable, but it needs to be human-led. So
[00:08:54] Nathan: AI [00:08:55] needs some instruction on, on what that, what that looks like in order to build it. At least, at least now. Maybe I'm [00:09:00] not seeing enough into the future where it has its own... I think if you get to the point where AI [00:09:05] has its own initiative to go do things,
[00:09:08] Nathan: you've hit AGI, you've hit [00:09:10] ASI, you've hit
[00:09:10] Andrew: Yeah
[00:09:11] Nathan: intelligence, 'cause it's starting to have its own intention, right?
[00:09:13] Andrew: So that's [00:09:15] interesting. I gotta say we were like debating this before and I was like, "We need to record it." This is... [00:09:20] 'Cause
[00:09:20] Andrew: it's like '
[00:09:21] Nathan: Cause
[00:09:21] Andrew: I was v- yeah.
[00:09:23] Nathan: loop back into what we were, we were
[00:09:24] Andrew: [00:09:25] Yeah. Yeah, I I think there's a little bit of misunderstanding 'cause I, I, I don't think code needs to [00:09:30] be human readable at all at this point.
[00:09:31] Andrew: And like a prime example is [00:09:35] if, without code, so if there's a bug, what do you do, right? In, in most [00:09:40] organizations the engineering team has to support it and figure it out, right? If something doesn't work or [00:09:45] doesn't work the way that it's intended or someone can't figure it out, you, you try to make it better, make...[00:09:50]
[00:09:50] Andrew: I think everybody hopefully is familiar with this. Either you've had a bug or you've had to fix one. [00:09:55] And I think where we were breaking down is, is that I don't [00:10:00] think ownership over a feature or a slice of [00:10:05] a stack needs to be human readable still. I don't... I [00:10:10] don't think that's a required...
[00:10:10] Andrew: It helps, but I don't think it, I don't think it's required. [00:10:15] And that's simply because you can just ask AI to tell you what it does [00:10:20] or you give it the problem and it'll figure out a solution. That doesn't mean it's gonna be right. It [00:10:25] doesn't mean it's gonna actually fix what you need it to. But it is good enough [00:10:30] in, I don't know, I'll say in my like direct experience probably 95% of stuff where [00:10:35] the first pass is at least 80% of the way there.
[00:10:38] Andrew: And it usually takes a [00:10:40] few minutes versus me sitting to have to pull a repo. Like the act of me even pulling [00:10:45] up the code is usually slower than AI just solving the [00:10:50] problem or like
[00:10:51] Andrew: having a rough idea of like it seems like this is the problem and this is the [00:10:55] solution or this is a description of the problem.
[00:10:57] Andrew: I'll read it and be like, "Oh, maybe it's this other [00:11:00] thing or or not," right? And so I'm using it as an interpreter, like a [00:11:05] code interpreter almost. And then when you start to fix stuff it's, it's [00:11:10] you... it... That's the pattern of engineering I think that really matters is it's does [00:11:15] this thing fit in our infrastructure?
[00:11:16] Andrew: Does it fit with the rest of our stack? Are we sending and [00:11:20] receiving information in, in ways that we're expecting to do that? Is it gonna consume all the [00:11:25] memory in our server stack and crash everything? Does it scale to a million users or [00:11:30] whatever, right? It's like it'll do an okay job with all of those random things.
[00:11:34] Andrew: If you give it [00:11:35] the context of your organization it'll do okay. But that, those like core engineering skills are still [00:11:40] required I think. But I don't think you, you... In order to execute on those [00:11:45] skills I don't think you need to be reading the code and I don't think you need to [00:11:50] be consuming code in human readable form.
[00:11:53] Nathan: Yeah, I
[00:11:53] Nathan: mean, I, I think the [00:11:55] way
[00:11:55] Andrew: with that. It's more of an abstraction layer than a, a direct layer I [00:12:00] guess.
[00:12:00] Nathan: I, I think in, in the ideal scenario, like if, if AI did get to that point [00:12:05] where it were consistently or more consistently delivering on [00:12:10] solving technical challenges, then yeah, I, I'd, I'd agree with that. You, you don't necessarily need [00:12:15] to know what the code looks like or even how it's structured in order to [00:12:20] be able to build things, and you may not ne- never, you may never even need to, to look at [00:12:25] it.
[00:12:25] Nathan: My eyes, it, it's like an insurance policy now because I, I've... And I'd be curious to know if there [00:12:30] are technical people in the audience who have encountered something like this. But even with something [00:12:35] like, I haven't seen it with Table five yet, but it's only because it's been out too, too shortly.
[00:12:39] Nathan: But even with [00:12:40] something like Opus, I've seen whack-a-mole situations where I would ask it to do [00:12:45] something, it
[00:12:45] Nathan: would fix it, but then it would break something else, and it would have to go and fix that. And then you realize [00:12:50] maybe three iterations, four iterations in that it's actually just breaking something else, right?
[00:12:54] Nathan: So [00:12:55] it helps as an engineer to be able to look at what the heck is going on, and I need to be
[00:12:59] Andrew: Yeah
[00:12:59] Nathan: to be [00:13:00] able to tell what that looks like and then
[00:13:01] Andrew: You don't. But see, I
[00:13:03] Nathan: it, [00:13:05] how..."
[00:13:05] Andrew: You don't though. You
[00:13:06] Andrew: can, you can use-- So you wanna go to the [00:13:10] source literally, but you don't have to. You can use the abstraction layer of the [00:13:15] AI to accomplish the same goal. So it's so you're, you're [00:13:20] saying that the AI isn't working the way you want it to or the way you think it should, so then you go [00:13:25] to the source to try to correct the error.
[00:13:27] Andrew: What I'm saying is, in my experiences with [00:13:30] AI, you can get it there without having to do that. Is that ideal and is it the most efficient [00:13:35] way? Maybe not. But it does work very well. I very, very, very [00:13:40] seldomly look at code anymore ' cause I find it's actually easier for me to give [00:13:45] it different direction or spin up a different agent and give it a different [00:13:50] perspective than to sit there, read the code, reason it out, think about [00:13:55] whatever the problem is, think about whatever is written, and try to solve it myself or like [00:14:00] even grok what the hell is going on.
[00:14:01] Andrew: 'Cause like in any kind of complex stack, like the line [00:14:05] of or the code or the function or whatever probably is the problem, but the, the, the, [00:14:10] the actual like core issue is probably something totally unrelated in a [00:14:15] sense or, or not in the same, section of code or whatever [00:14:20] you wanna call it.
[00:14:20] Andrew: I'm trying to be a little abstract for folks who've never looked at source code before.
[00:14:23] Nathan: Yeah
[00:14:24] Andrew: [00:14:25] But, but I think that's like the big with Opus [00:14:30] and, the higher reasoning models and like UltraCode with Claude, [00:14:35] I've really found that, having an excellent plan, even if it's a [00:14:40] bug, is like pretty damn good.
[00:14:42] Andrew: And I will just disclaim, like there are some things you're doing in [00:14:45] code that still like AI could shit at, like anything low level [00:14:50] firmware exotic languages, it, it's not. But if you're in a modern stack or a [00:14:55] semi-modern stack and you're doing like databases and event streams [00:15:00] and like what I would call standard engineering stuff, like it crushes as far as I'm concerned [00:15:05] at this point.
[00:15:06] Andrew: So there are caveats of course, and I, I think people get polluted on edge [00:15:10] cases all the time in tech and it's yeah, okay, it's not-- I'm not saying it's perfect, but I'm saying that we've [00:15:15] rounded a corner in the utility of it where yes, you can go to the [00:15:20] source and it still helps, but you don't have to.
[00:15:22] Nathan: Yeah, yeah
[00:15:23] Andrew: been working on, I haven't looked at the code in six [00:15:25] months. It works.
[00:15:27] Nathan: Yeah. I'm curious to know [00:15:30] if there are people out there who work... Like I, I'm, I've hear- I've heard things about [00:15:35] when you talk about enterprise level code bases, just the sheer amount of
[00:15:38] Nathan: context. A- and maybe that's [00:15:40] attributed to the human readability of it, right? But
[00:15:43] Nathan: There is a point where you just [00:15:45] cannot contain all the context for a code base in a single inference of [00:15:50] an LLM.
[00:15:50] Nathan: And you, you could try to cleverly break that out and split up the context, [00:15:55] but there's something lost in the distillation of certain sections of code [00:16:00] into context or tokens that it can actually follow
[00:16:03] Andrew: don't know if that's true with [00:16:05] Fable
[00:16:05] Nathan: Not true in the sense of, i-in the sense that it has enough context or that it [00:16:10] doesn't
[00:16:10] Nathan: miss anything out in, in the tokenization process?
[00:16:13] Andrew: f... It gets interest. What is [00:16:15] context?
[00:16:15] Nathan: In the, the actual context window of the, the model
[00:16:19] Andrew: Yeah. [00:16:20] So I don't-- I, I think that there are enough [00:16:25] techniques and approaches that you don't... I, I'll just say I've run [00:16:30] through several million lines of code with different repos across different [00:16:35] services, across all sorts of shit, and I was a little shocked [00:16:40] that it actually did. Like the thing we were talking about before we went on that, that's not publicly released [00:16:45] yet, which will be hopefully this weekend, knock on wood.
[00:16:47] Andrew: Like it took a while and I had to kinda [00:16:50] push it through some obstacles of AI-ing, but we'll [00:16:55] see. It's building right now. Like I s- I'm, I'm sure it'll be 80% of the way there. [00:17:00] But, it, it's-- The, the context window problem is a huge problem, [00:17:05] but it's the models and the, the tools that you [00:17:10] can use with the models are getting so good now that, I'm trying to figure how to [00:17:15] describe this. So it's like When you're trying to solve the problem, it, it's really [00:17:20] how you break the problem down. So if it's like a bug fix [00:17:25] and I'm trying to think of something that's like kind of complicated but universal that people [00:17:30] could understand.
[00:17:31] Nathan: Like, life?
[00:17:32] Andrew: Okay, like logging into your email.
[00:17:33] Andrew: Everybody has an email [00:17:35] probably, or knows what it is, hopefully. And so you log into your email and you get into your [00:17:40] inbox and there's no messages. What happened? And so that in a, in [00:17:45] a scaled system is you have authentication layers, you have the [00:17:50] m- the data store, your, your mailbox data sits somewhere.
[00:17:54] Andrew: You probably [00:17:55] have some sort of a middleware stack that's taking that data and rendering it in the web. There's a [00:18:00] web stack. There's m- or maybe all sorts of other features that are maybe [00:18:05] blocking the data stream or the... There's all sorts of problems you could have, and that could be many [00:18:10] millions of lines of code across 50 repos across microservices, blah, blah, blah, [00:18:15] blah, blah.
[00:18:16] Andrew: But a lot of the, the, the tool [00:18:20] calling now allows you to distill that context down so [00:18:25] that the AI knows enough to know when it doesn't know, and then can find [00:18:30] out. And then the, the, the tricks with so-called memory,
[00:18:34] Andrew: right? So as it's [00:18:35] going, it'll create a memory and be like, "I need more information." So it'll go out, [00:18:40] it'll either ingest it if it has the context, or it'll dump the context that it [00:18:45] has into a quote-unquote memory, go do the thing, recall the memory, [00:18:50] and then have enough information to keep going.
[00:18:52] Andrew: Usually. It doesn't always... That's how [00:18:55] it's supposed to work, but it like, eh it, it doesn't always work that way. [00:19:00] Additionally, the way that at least Anthropic is moving is [00:19:05] that you have a control agent, which is like an Opus or a Fable, [00:19:10] and this is where Fable's really, really good at,
[00:19:13] Andrew: It's like a self-orchestrator, [00:19:15] right?
[00:19:16] Andrew: So a- all Opus needs to know is I'm trying [00:19:20] to figure out why the user doesn't have access to their email, and I [00:19:25] know enough about the project 'cause I've given it this is my email server and here's the [00:19:30] services that it touches, and I did a good job engineering land explaining [00:19:35] it.
[00:19:35] Andrew: And there's MCP servers and shit, and all sorts of other stuff that it can pull on. It can go [00:19:40] look at the repos. If the repos are well-designed and readable, so it can just pull what it needs. [00:19:45] And so then it'll spin up dozens of sub-agents if it needs to, [00:19:50] to go do that work. "Okay, you go look at the database repo.
[00:19:52] Andrew: You go look at the authentication repo. You look at [00:19:55] the logs." and then they'll all report back, and then it has to reason [00:20:00] the synthesis of those contexts. So it's really just extending its context across... If it's a million [00:20:05] context window, now you have 13 million context you're working with effectively, right?
[00:20:09] Andrew: 'Cause you [00:20:10] have 12 sub-agents and a- an orchestrator
[00:20:12] Nathan: number of, of agents that they're...
[00:20:14] Andrew: Right, And so [00:20:15] it'll report back something like... I, I'll try to illustrate the problem for folks. So it'll say, "Oh, database [00:20:20] servers are running, everything's good, no errors, dedication services are running," [00:20:25] whatever. So then it's okay, then it, it'll have its plan of like attack of how it's reasoned it to, to do the [00:20:30] troubleshooting, and then it'll refactor and, and go through and, and eventually probably [00:20:35] fix it.
[00:20:35] Andrew: Again, it might not. But those... I [00:20:40] think the method of using a single agent and, [00:20:45] trying to do the work for it or trying to guess what it needs to know is [00:20:50] over. And I'll say the big caveat with this approach, and what we were talking about earlier, [00:20:55] I don't think you need to r- have AI write human-readable code anymore, as long as you're cool [00:21:00] using AI.
[00:21:01] Andrew: ' Cause if you're having AI come up with crazy shit, then you're [00:21:05] probably not gonna be able to maintain that yourself or it's gonna be really, really, really hard. And so now [00:21:10] you're building in a d- AI dependency, and that is potentially a problem depending on what you're [00:21:15] wanting to do. But I think... And, and it's also a cost thing, right?
[00:21:18] Andrew: 'Cause spinning up agents isn't [00:21:20] free either. And but that's a calculus. So do you wanna pay an engineer who makes [00:21:25] $200,000 or $300,000 a year at most companies to do that work? Or do you [00:21:30] wanna pay a person maybe less or maybe more to be able [00:21:35] to extend that work without having much more people involved?
[00:21:39] Andrew: And more, I [00:21:40] think importantly for the business, is that that will get done much faster in the [00:21:45] hands of a capable person because they'll be able to read all of the repos within 20 [00:21:50] minutes and probably have a solution or at least have enough, AI-informed [00:21:55] context to then make an engineering determination to say, "Oh, it's this thing.
[00:21:58] Andrew: Let me go look," or point [00:22:00] the AI in a more focused path versus the broad, "User can't log in. What's [00:22:05] going on?" and so then as you're, as you're reasoning with it or like you were saying human in the [00:22:10] loop, now you have somebody who's a 10X right? 10X more [00:22:15] capable. And I'm
[00:22:16] Nathan: Thanks, mid-way through here.
[00:22:17] Andrew: to remove a bunch of people from your company, and that's [00:22:20] where some people will go and "That's stupid."
[00:22:22] Andrew: If you have 10 really good engineers today, give [00:22:25] them AI, and now you have 50 equivalent engineers
[00:22:29] Nathan: Yeah. [00:22:30] The, the ones that, that know how to work with AI in, in this context, right? And, and to tie the-- [00:22:35] or to circle back to, we, we have talked about before on the [00:22:40] podcast, why people weren't working with AI the right way, right? And this kind
[00:22:43] Nathan: of loops back into [00:22:45] it. Yeah, I, I'm gonna argue your side here is, is the argument about being AI [00:22:50] dependent.
[00:22:50] Nathan: You can make the same argument for a team of engineers or even like a contractor. If you end up [00:22:55] contracting out a dev team, you become dependent on the dev team because they're the only ones that know how to build your product or manage your product, [00:23:00] right?
[00:23:00] Andrew: 100%. And,
[00:23:01] Nathan: exact, same exact,
[00:23:02] Andrew: and every team works different, and then you [00:23:05] lose a person on the team, they quit or they get fired or whatever, and now you've lost a way of [00:23:10] working. You have to onboard somebody, you bring them in. So again, I'm, I'm not saying we replace humans with AI, [00:23:15] but a capable human who knows what they're doing with AI is every [00:23:20] time a new model releases, every time a new tool releases.
[00:23:24] Andrew: And like what I'm, I'm [00:23:25] talking about for folks too, like this is something you do today with Opus [00:23:30] today. You don't need Fable. Fable's really good at it, but you don't need it. And I've-- I was doing [00:23:35] something this morning where Opus pulled up, I don't know what was however many agents, it was like eight [00:23:40] agents to do something that I was surprised about, but whatever, it got done and it worked.
[00:23:44] Andrew: I don't care. [00:23:45] And it, it was... The cost was, I don't remember, it was like [00:23:50] 20 bucks
[00:23:51] Nathan: Yeah, yeah
[00:23:52] Andrew: To implement a feature you [00:23:55] can argue money, but in tech three... I'm, I'm working with 200, [00:24:00] $300,000 in salary plus 100 or 200K in [00:24:05] benefits. That's a lot of AI,
[00:24:08] Andrew: right? And again, I'm [00:24:10] not saying replace people with AI.
[00:24:11] Andrew: I wanna be really clear, and I think it's really, really stupid [00:24:15] for a company to, to sack a bunch of people and then try to replace them with AI, because [00:24:20] you still need that human context. I think, though, we are seeing, and this might be [00:24:25] controversial, there's a lot of shitty engineers out there who are, like, getting exposed
[00:24:29] Nathan: No [00:24:30] that's of course is true.
[00:24:32] Andrew: It's the team who's "Oh, this is gonna take six months," and [00:24:35] then you spin up Claude and it's "Oh, I got it done in 10 minutes." And like we've talked about this [00:24:40] before too, it's like probably a little bit more h- complicated than that. But like it also probably [00:24:45] not either, right? It's there's some kind of balance con- but there's a lot of people who pad the estimate.
[00:24:49] Andrew: There's [00:24:50] a lot of people who don't want accountability, like we were talking about earlier. I don't wanna build a feature. My team doesn't wanna [00:24:55] own it. There's internal politics. If you're a manager in a tech company or a company with tech [00:25:00] people, you gotta get the politics bullshit, the fiefdom building, the king of the hill shit, that has to die [00:25:05] immediately.
[00:25:06] Andrew: That, that doesn't work anymore.
[00:25:08] Nathan: Yeah, yeah
[00:25:09] Andrew: [00:25:10] Like people are starting to figure it out, but they're really not, and it's like that, that just shut the fuck up [00:25:15] and work is like the mantra of some tech companies at this point, and I think [00:25:20] it's taken a little bit to the extreme where it's becoming like only work culture, [00:25:25] which is bad.
[00:25:26] Andrew: But the balance is like stop sitting in meetings and [00:25:30] like politicking and not producing anything of value or experimenting when you could [00:25:35] just have a AI write a bunch of experiments and try it, and then you have the data and then [00:25:40] make the determination and move on. And I've been in companies, I know you've been there too, where [00:25:45] should we do it?
[00:25:46] Andrew: While that used to be a f- 50, 100, [00:25:50] 200, $300,000 decision, maybe a couple million 'cause you're deploying a bunch of really [00:25:55] expensive people. An en- an engineering team working on something for two weeks is [00:26:00] probably 40 to $200,000 in most companies.
[00:26:04] Nathan: Yeah, yeah, yeah[00:26:05]
[00:26:05] Andrew: So when I say something takes 20 bucks and a few minutes, like that's [00:26:10] the scale that we're now working on.
[00:26:12] Andrew: And so I, I think again, it's how you're approaching [00:26:15] stuff, but the human readable thing, I think it's a symptom of a [00:26:20] problem, and you also then have to change how you work, and you also have to accept for other f- putting [00:26:25] on my PM hat, you have to accept that like you might get issues with AI [00:26:30] code that are not straightforward and do not solve immediately.
[00:26:33] Andrew: And so if you're [00:26:35] like hedging this on your company's, success, you better really [00:26:40] know what the fuck you're doing and what trade-offs you're making because that sev zero [00:26:45] might not be solved in a few hours by a capable team of people if you have a bunch of AI working. [00:26:50] It might take days.
[00:26:52] Nathan: Hmm
[00:26:52] Andrew: Conversely, the thing that would take a few [00:26:55] hours to solve now might take minutes
[00:26:57] Nathan: Yeah. Yeah. One of the things I'm reminded of [00:27:00] as, as you, you said, you were going through something with Opus and [00:27:05] it spun up eight agents and it did something and you didn't care it worked, right? That reminded me of, I think the first time we [00:27:10] did talk about this topic, we were mentioning this, how far [00:27:15] abstract are you from the, the problem being solved,
[00:27:17] Nathan: right? I think that was a, a [00:27:20] textbook example of where the abstraction is starting to grow. used to be like you [00:27:25] necessarily didn't have to know exactly how the thing worked, but you had to know a little bit to get in there and do stuff. [00:27:30] But now we're at a point where you're using AI to pull yourself out farther away from the actual [00:27:35] thing being done.
[00:27:35] Nathan: As long as the outcome is there,
[00:27:37] Andrew: Yeah, it's outcome-driven development, and, and [00:27:40] that means like engineers don't get to sit behind a Kanban board anymore. Like you need to [00:27:45] know how the product works. You need to know how people are using the product. Product managers, you need to know [00:27:50] very deeply what the market is. You need to know very deeply what your product does.
[00:27:53] Andrew: You need to know very [00:27:55] deeply, what not to build and why not to build it, right? 'Cause just 'cause you can doesn't [00:28:00] mean you should. And where a lot of PMs get stuck too is like we're now in this [00:28:05] world where, yeah, you had a few months to figure it out or you could defer some of the things you needed to do for user [00:28:10] research and like I've been there.
[00:28:12] Andrew: All the PMs listening are like, "Oh, he's talking about the [00:28:15] secrets." And I'm like, "Everybody knows. This isn't a secret." And sometimes that's what you have to do, right? But [00:28:20] n- now you don't have to. The, the thing you were saying earlier too is like you need a human in the [00:28:25] loop. I think we've talked about this before.
[00:28:26] Andrew: I don't remember if we've talked about it publicly or not, but, like c- [00:28:30] customer support is something that everybody claims AI can do. Having worked very deeply in [00:28:35] that also putting AI into customer support and working in regulated [00:28:40] systems and non-regulated systems with it
[00:28:41] Andrew: Not easy. And, sure it can [00:28:45] help, but yeah, no, it's not gonna take anyone...
[00:28:49] Andrew: It's not gonna take [00:28:50] anything over. It's gonna give you a really shitty customer experience. Your customers are gonna go elsewhere, and if the only [00:28:55] thing you're, you're working on is customer lock-in, then you're probably not gonna be a company very long 'cause [00:29:00] someone's gonna come along and do it better.
[00:29:01] Andrew: Anyway, a lot of the customer feedback I get, [00:29:05] I will just turf it to The AI, like verbatim, like the [00:29:10] email or a, a chat. I, I guess for folks all my company [00:29:15] products have h- support or whatever, and if people have issues, they write in. [00:29:20] And a lot of times I get a lot of actually really positive stuff.
[00:29:23] Andrew: They're like, "Hey, this is really great." I'll get like a [00:29:25] feature request or like some idea or "I don't really... " I, I would... It [00:29:30] would be great if it did this." I love that shit, so please, if you're listening and you buy my [00:29:35] stuff, please just send them in.
[00:29:37] Nathan: Let him
[00:29:38] Andrew: Yeah I'm not gonna promise we're [00:29:40] gonna build everything, but like I was working on something this morning that someone wrote in and, it was, it [00:29:45] was actually a u- education issue where they were, they didn't, weren't aware that it did do something [00:29:50] that they were asking for, but it didn't necessarily do it in the way that they wanted either.
[00:29:53] Andrew: So I thought about it and
[00:29:54] Andrew: I was [00:29:55] like, I was like, "Actually, this is a really great idea." So I turfed it to AI and I said... [00:30:00] Literally just copy and pasted it, and I think I gave it a couple lines of like additional context around [00:30:05] where I was thinking I wanted it to go. And I was like, "Build a plan."
[00:30:08] Andrew: Built a plan, [00:30:10] flipped through it. I was like, "Yeah, close enough, and then let's go, and I'll try it." And then 20 minutes later [00:30:15] I tried it and I was like, "Oh, that's pretty good." so we have a new feature coming out. It spun up like 40 [00:30:20] sub-agents or whatever too, so I make it sound easy and I'm like, it is easy, but it's [00:30:25] also like it's a little bit more involved than that.
[00:30:26] Andrew: You gotta remember
[00:30:27] Andrew: too, I-
[00:30:27] Nathan: conversation. I, I still don't know how your agent spins [00:30:30] up
[00:30:30] Nathan: 40-some
[00:30:30] Andrew: I've been building soft- yeah I've been building software for [00:30:35] 20-plus years, right? So it's if you're-- we're doing this at home, like maybe it's really hard. [00:30:40] I'm not trying to make it sound easy or diminish it. It, it's like, it's just the way I do. [00:30:45] But that's not that far from automating support, right?
[00:30:49] Andrew: I'm not gonna use [00:30:50] AI in support. I don't believe in that. It, it's... There are instances where if [00:30:55] it's easy to solve questions like I, I need to get my password reset or how do I find my bill, that [00:31:00] kind of stuff it's helpful for. But as a general rule, I think if the [00:31:05] AI can't figure it out in one turn, you need to turf it to somebody who can.
[00:31:08] Andrew: And then, and, and for folks who [00:31:10] don't understand, if you're chatting in or emailing in, it's that never-ending loop you get in with AI that [00:31:15] literally every human being I know hates
[00:31:17] Andrew: " No, billing." "Oh, so you wanna open an account?" [00:31:20] "No, I want my bill. I don't understand why you have this charge." "Oh, you want your bill?
[00:31:23] Andrew: Your bill is blah, blah, blah." [00:31:25] "No, why did you charge me this thing?" "Oh yeah, we charged you blah, blah, blah, blah, blah." "Thanks," or, "I [00:31:30] got it." And it's no, if you, if it, you, if you ask a question and maybe two turns, like it just, [00:31:35] it depends on your industry and what you're doing, but that's just terrible, terrible, terrible customer [00:31:40] experience.
[00:31:40] Andrew: And you're not solving those problems, and I think more importantly, you're not getting [00:31:45] the level deeper where if you're actually running really great customer service, one, you solve the, [00:31:50] the problem and then you put those fixes upstream so the problem doesn't happen for other people. Or if [00:31:55] they can't find their bill, you figure out a better way with your product team to make it easier or more [00:32:00] transparent.
[00:32:01] Andrew: And then two, you're losing that deep side of knowledge that you have [00:32:05] from how to make improvements or how... Oh, we have 50 billing questions. Maybe we need to [00:32:10] look into this more deeply and figure out why we're having whatever subset of pro- [00:32:15] aI can do that, but not really. It can help kind of like sift it, but [00:32:20] w- but you still have to use your judgment, and that's what people
[00:32:22] Andrew: keep missing on that too.
[00:32:23] Nathan: initiative to mention that, right?
[00:32:24] Andrew: [00:32:25] Yeah. But, but we're,
[00:32:26] Andrew: but I'm getting to the point, and we've talked about this I've been [00:32:30] thinking about spinning up like a shadow AI support network [00:32:35] to like kind of test this. Again, I'm not gonna push people to AI. There's a lot of [00:32:40] people who hate AI too, so I get it. I, and I, I don't want that. But I [00:32:45] do think there's really interesting embedded learnings that you can make, [00:32:50] and I can see the very clear pathway to like feature [00:32:55] recommendations and it's pretty cheap in quotes to implement a [00:33:00] feature in a test branch somewhere, test it out, see if it's even worth continuing on.
[00:33:04] Andrew: It's [00:33:05] still expensive at scale, like you, you wouldn't wanna just spin up every customer, [00:33:10] but I, I think queuing that and, and getting that in a, in [00:33:15] front of somebody like a PM who could make some decisions, run some tests, [00:33:20] engineering team doesn't even need to get involved or last mile kind of stuff is a really interesting [00:33:25] idea.
[00:33:25] Andrew: No, I will not work for your company to do that. But you can h- you have that [00:33:30] one for free. So yeah, so I, I think it's, it's interesting, right? So I, I think we're, we're in a... [00:33:35] And maybe we should do a cast next week on the sub-agent things. Like I was using [00:33:40] Fable for we were ragging on it 'cause they neutered it.
[00:33:43] Andrew: You did neuter it We [00:33:45] know. You didn't fool us, Anthropic. We know. But when it isn't [00:33:50] neutered, and the-- my biggest complaint, and we talked a little bit about this too, it's so inconsistent for me. [00:33:55] Sometimes it just like I had a, a problem I was working on that I could talk about next [00:34:00] week once the product is released, but it spun up over 100 sub-agents, and it went [00:34:05] for three hours.
[00:34:07] Nathan: You are ridiculous.
[00:34:07] Andrew: tell it to, right? And, and I, I [00:34:10] don't think the problem was that... It was a really complicated problem, [00:34:15] and maybe trying to reason that solution as an AI required [00:34:20] that. It required a village. But at the same time, if I were to have to write [00:34:25] that solution, it probably would've taken me a couple days just to actually write the actual [00:34:30] code, test it, make it work, find all the bugs, that kind of stuff.
[00:34:33] Andrew: But-- And sometimes it won't do [00:34:35] that, and Opus will do it sometimes. So it's I like that, but it's also at the same time, [00:34:40] sometimes I don't want that 'cause I know the solution is really not that challenging, and I don't wanna pay for a bunch of AI [00:34:45] credits that I don't need to pay for,
[00:34:47] Nathan: yeah. You didn't need to spin up all those agents, right?
[00:34:49] Nathan: [00:34:50] Yeah
[00:34:50] Andrew: So I would like that feature, Anthropic, to be able to limit the number of [00:34:55] sub-agents or have some sort of approval
[00:34:57] Nathan: I think you, you, you might be able to just throw that [00:35:00] into the prompt and tell it, say, "Don't spin up more than 20 agents for this thing," and, and that might work [00:35:05] 'cause anytime I need to do anything with sub-agents, if I instruct it to spin up [00:35:10] sub-agents, I'll tell it how much to do, and it always seems to
[00:35:12] Andrew: Yeah.
[00:35:12] Nathan: instruction.
[00:35:13] Andrew: Huh.
[00:35:14] Nathan: [00:35:15] So
[00:35:15] Andrew: But I don't wanna limit it. I don't wanna limit its potential. I want it,
[00:35:18] Andrew: I want it to [00:35:20] blossom
[00:35:20] Nathan: You, you don't wanna, you don't wanna further neuter it.
[00:35:22] Andrew: I also don't wanna pay the bill, so [00:35:25] I
[00:35:25] Nathan: Yeah,
[00:35:25] Andrew: and that's the flip side with this stuff, right? Is it sounds easy, but the [00:35:30] big problem is what are you using all the spend on? And I, I'm confident in [00:35:35] most of the stuff I do that it's creating value. Folks I [00:35:40] work with I think are the same way. Small team helps, [00:35:45] too.
[00:35:45] Andrew: Major companies are having problems with this. Have a thing I kinda wanna plug, but I [00:35:50] won't. I just refuse to plug stuff
[00:35:52] Nathan: oh,
[00:35:52] Nathan: oh, oh, okay, okay. Yeah, yeah. [00:35:55] I yeah, do-- we didn't talk about offshoring yet, no. We, we, yeah, we didn't. I
[00:35:59] Nathan: don't [00:36:00] think we did, but okay. Yeah there's one thing that you, you threw out some, some [00:36:05] free advice as well. But the, the, the thing I liked about your idea with the shadow support agent and [00:36:10] being just monitoring
[00:36:11] Nathan: things is, is for people who are more engineering driven or [00:36:15] engineering focused, a more concrete one is having a like a cloud agent watch if you [00:36:20] have any error monitoring, like in Sentry or something. Anytime you're, you have deployed software, a bug [00:36:25] happens rather than someone report it, it's better if you have gates that re- [00:36:30] log it somewhere, and then you have an agent watching that log, and then they pick it up. And if it's a very clear [00:36:35] error, which most error tracking tools generally have enough context there to, to
[00:36:39] Andrew: Yeah.
[00:36:39] Nathan: solve the [00:36:40] error,
[00:36:40] Andrew: Yep.
[00:36:41] Nathan: go and fix it, right?
[00:36:42] Andrew: Yep
[00:36:42] Nathan: A- and that's a situation where I can imagine [00:36:45] a team having that just running and not even knowing that it happened unless they look at the log
[00:36:49] Nathan: itself. [00:36:50] But it's, it's self-healing. It's building
[00:36:53] Nathan: itself
[00:36:53] Andrew: no, it's it's all filling [00:36:55] with intelligence. Absolutely. And I will say that turfing a bunch of AI or [00:37:00] customer service tickets to AI is not how you do it.
[00:37:02] Nathan: Yeah
[00:37:03] Andrew: I'll s- I'll save you that [00:37:05] pain, pain point right up front. And, and no, most times customers don't [00:37:10] actually know what they want, which is the hard part of customer service
[00:37:12] Nathan: true.
[00:37:13] Andrew: AI
[00:37:14] Andrew: doesn't [00:37:15] really do a great job.
[00:37:16] Andrew: It can and it can't, and it, it's... I've said this many [00:37:20] times, if you're okay giving your customer the wrong answer, then AI [00:37:25] is a solution for you
[00:37:26] Nathan: Yeah. It's, it's a potential solution for you.
[00:37:29] Andrew: [00:37:30] I'm being a little facetious, but mean that
[00:37:33] Nathan: Yeah
[00:37:33] Andrew: for word. If giving [00:37:35] your customer the wrong answer is a solution, then AI is for you
[00:37:39] Nathan: [00:37:40] Hmm, hmm.
[00:37:41] Andrew: I'm not saying AI isn't useful in customer service. I'm not saying AI [00:37:45] can't help, and I'm not saying there aren't instances where you can't use it. It's just most companies aren't [00:37:50] doing it.
[00:37:50] Andrew: It's, it's a little bit harder than just plugging it in
[00:37:53] Nathan: Yeah. Trying to [00:37:55] figure out how to loop this back to, what the main topic was.
[00:37:58] Andrew: We're talking about applied, [00:38:00] applied AI now, but,
[00:38:01] Andrew: um, yeah. You don't need AI to write code [00:38:05] for human readability. Done. Stream over
[00:38:07] Nathan: Yeah, yeah, yeah. My, my point or my corollary [00:38:10] to that is unle- unless as an engineer you want the insurance, right? I think there is [00:38:15] insurance, in the sense
[00:38:15] Nathan: of
[00:38:16] Andrew: no, I agree. And, and if you have business owners or people in the [00:38:20] company who are gonna try to hang you because something doesn't work and you can't give them the exact [00:38:25] reason why in a few seconds, then yeah, it's not good, but your company sucks, don't work [00:38:30] there. Go somewhere else, frankly.
[00:38:32] Andrew: Like this, this, this [00:38:35] punitive hierarchical corporate management is the problem. It's not [00:38:40] AI, it's not teams, it's not solving the problem. And the thing [00:38:45] we've said a little- we- I've said on the podcast prior too, it's like all these companies who think they have moats [00:38:50] because they have a product or they have traction, you don't have it.
[00:38:54] Andrew: A [00:38:55] small group of motivated people can take out pretty much any SaaS platform overnight. Like you still have to do [00:39:00] customer acquisition, you still have to do the business stuff, which is hard. You gotta do marketing, all this kind of [00:39:05] stuff. But emulating your core product is not hard at this [00:39:10] point with technology for most things.
[00:39:11] Andrew: There's some regulated industries or like niche industries or [00:39:15] specialized, there's... Yeah, sure. But like most companies are not that. Most companies think they're [00:39:20] that, but they're not that, right? "Oh, we have a bunch of customer data we can use." Maybe. [00:39:25] Can you execute on it? I'm gonna guess no, because you haven't done it yet.
[00:39:28] Andrew: The companies who have done it [00:39:30] are now gonna apply AI to that layer and fucking crush it, because I think we've talked about before [00:39:35] too, it, it pretty... I think everyone knows now that you have to have good data to AI, and if you
[00:39:39] Nathan: Garbage [00:39:40] in, garbage out kind of thing. yeah,
[00:39:41] Nathan: yeah. Yeah
[00:39:42] Andrew: But the companies who are already doing that are fine, and they're not the [00:39:45] ones that are treating employees crappy and making them work 80 hours a week and firing people 'cause [00:39:50] stuff breaks.
[00:39:50] Andrew: They're like, "Oh, it broke. How do we fix it? How do we make sure it doesn't happen again?" And "Let's learn from it," [00:39:55] versus, "Nathan, what do you mean the server went down and, and we lost a sale? You're [00:40:00] fired
[00:40:00] Nathan: Yeah. I as you're, you're talking about this whole punitive, you [00:40:05] don't-- why don't you know the reason kind of thing it obviously, at least for [00:40:10] engineers, like when you're presented with that, "Oh, you don't know why it broke?" Is, It, it hurts [00:40:15] a bit more as an engineer because code is deterministic, right?
[00:40:17] Nathan: You, you should be able to know why code doesn't [00:40:20] work. So when you're asked a question, even though in gen- in general saying, "I don't know," is, is, is, [00:40:25] it's acceptable in, in the
[00:40:26] Andrew: Yeah, but
[00:40:26] Nathan: of engineering, it's
[00:40:27] Andrew: c-code is deterministic, but systems not, [00:40:30] are not necessarily deterministic
[00:40:31] Nathan: Yeah, sure, sure. Yeah. You, you could, yeah, so you could get away with [00:40:35] if you're talking about engineering a solution or a system that you c- you don't know exactly why it broke because there are [00:40:40] so many variables involved. And, and maybe that's the thing that we have to let go of is feeling like we [00:40:45] should need to know exactly how things are working, which goes back to your point of not necessarily [00:40:50] needing to write human-readable code, is you just gotta let go of, of not
[00:40:54] Andrew: And it's [00:40:55] how you orchestrate it. I, I'm, I suspect a lot of the success I have is [00:41:00] how we've built things. So interesting case, [00:41:05] and then we, we should, we
[00:41:05] Nathan: gonna say you need to, be a little more, a little more specific how we build
[00:41:08] Andrew: Yeah.
[00:41:09] Andrew: It [00:41:10] was funny, I was, I was sharing with a friend today, we were talking about stuff and we were talking about [00:41:15] how engineers love to change shit all the time, and snake case versus camel case is always a, [00:41:20] a mortal debate, and tabs versus spaces and yeah.
[00:41:24] Andrew: And, like, all this shit's [00:41:25] meaningless now 'cause it's... You don't have to write code, but in most cases. Again, people are gonna hear this, they're like, [00:41:30] "Ah, it's... Coding's dead." I'm like, "It is dead, but it's not." It's just like writing is dead, but [00:41:35] we still use it.
[00:41:36] Nathan: How interesting that you went there. [00:41:40] Yeah
[00:41:41] Andrew: I wrote a letter, but I did write something down moments ago, [00:41:45] right? It's not, but it, it's not, it isn't needed. You could prefer it, that's totally [00:41:50] cool, right? And, but anyway. And, and so it was funny because AI has this [00:41:55] problem too.
[00:41:55] Andrew: We've, we've talked about this. I don't know if we've ever talked about it publicly, but when the early days of [00:42:00] writing a product that is now dead, AI was hallucinating snake case and camel case for the [00:42:05] API. Yeah.
[00:42:06] Nathan: yeah. I remember you, you talked about this.
[00:42:08] Nathan: Yes, yes. Great
[00:42:08] Andrew: so one thing I've [00:42:10] learned is that AI does that probably because engineers debate that incessantly, and that's how it's [00:42:15] trained.
[00:42:15] Andrew: And so now what I do is all my APIs are compliant for snake and camel [00:42:20] case, right? And that sounds insane. Any engineer's probably "That's crazy. You have to maintain that." [00:42:25] I don't have to maintain it. AI has to maintain it
[00:42:27] Nathan: Yeah, it's, it's not crazy. Back in the day, [00:42:30] we did have to maintain that. You
[00:42:31] Nathan: had to write going from one code base to [00:42:35] another. If the convention in one code base was snake case Python, another one was [00:42:40] camel case or Pascal case JavaScript, you would
[00:42:42] Andrew: Yeah
[00:42:42] Nathan: write a translation layer to make sure it [00:42:45] was compatible both
[00:42:45] Nathan: ways.
[00:42:45] Andrew: and anyone who's any- ever worked in B2B software, it's the same thing. It's [00:42:50] "Oh, our internal coding standard is snake case and you're a camel case. We're gonna send you snake [00:42:55] case and it's gonna error, and we're gonna be mad, and we're gonna be like, 'You didn't uphold your SLA,' and then you're gonna [00:43:00] investigate, and you're gonna tell us we didn't send you the right thing."
[00:43:02] Nathan: Yeah.
[00:43:03] Andrew: So anyway, but, but that's a, [00:43:05] that's, an adaptation of AI coding, right? Where it's like I would never wanna have to [00:43:10] maintain that even as a product person if I-- my-- if this was my API, I'd be like, "Fuck no, we're picking [00:43:15] one." now I don't really care, and AI... It, and it, it-- I would say 95% of the [00:43:20] time it gets it right now, and it, it, if I do that it still has pro- [00:43:25] AI's not perfect.
[00:43:26] Andrew: I'm not trying to pretend like it is, and I'm not trying to whitewash it. [00:43:30] But the things that are tripping people up that I hear in the zeitgeist, I'm like, "This is basic level shit [00:43:35] that y-you should not be getting tripped up on," but it's also really hard to kinda, like-- 'cause it's so abstract, it's [00:43:40] hard to abstract through it.
[00:43:41] Andrew: So this is an example of how I've done that. Yeah so now it, it doesn't [00:43:45] matter, frankly, and it works. It actually-- I find that it reduces API errors a [00:43:50] lot. It creates other problems sometimes where if your code base isn't [00:43:55] completely snake-cameled, camel-snaked
[00:43:56] Nathan: Yeah, yeah. Camel
[00:43:58] Andrew: weird data [00:44:00] problems. But
[00:44:01] Andrew: But yeah, and then you have AI write up a API spec [00:44:05] that other AI can read and refer to, and 99% of the time it [00:44:10] works. And then when it doesn't, you catch it in your
[00:44:12] Andrew: test
[00:44:12] Nathan: these numbers are suspicious.
[00:44:14] Andrew: I know, [00:44:15] right? No, I, the API stuff is
[00:44:17] Andrew: mostly solved for me at
[00:44:19] Andrew: least. Like [00:44:20] it... Yeah
[00:44:20] Nathan: I think that's, I think that's fairly now. Y- yeah, you don't really have to worry too much about it [00:44:25] anymore.
[00:44:25] Andrew: But I'm sure you've been through
[00:44:27] Andrew: this. I've been through hours-long discussions about whether we should do one or the other or something [00:44:30] different, or maybe we should invent our own where we capitalize every other letter for some reason, right? [00:44:35] 'Cause then, then engineers really like to brainstorm and get creative, right? And now you're in this weird,
[00:44:39] Andrew: [00:44:40] like, reinvent it.
[00:44:42] Andrew: What if we just used non [00:44:45] Cyrillic, we use Cyrillic characters instead of Arabic characters? Wah. Anyway. [00:44:50] We'll leave it there. You don't need to code for p- people anymore. It's done. It's dead. You heard it here [00:44:55] first. Prediction 2026.
[00:44:58] Nathan: Ah, yes. At the end of 2020, [00:45:00] okay.
[00:45:00] Nathan: We'll do that for your prediction. I'm expecting, I'm expecting that for your prediction of [00:45:05] 2027
[00:45:05] Andrew: Read a- readable code is dead. No, it's not. It helps. And one thing I guess [00:45:10] we didn't make either, human-readable code helps AI
[00:45:14] Nathan: Yeah. [00:45:15] Yes. Yes. And, and that's, yeah. It, it's a bit circular, right? Because,
[00:45:18] Nathan: because it was trained on [00:45:20] human
[00:45:20] Nathan: readable code
[00:45:21] Andrew: No, but,
[00:45:21] Nathan: So
[00:45:22] Andrew: the, the new Claude models or the newer Claude [00:45:25] models are much better at code commenting.
[00:45:27] Nathan: A little bit, a little bit too... They're, they're too, in
[00:45:29] Nathan: my [00:45:30] opinion, they're a little too
[00:45:30] Nathan: aggressive with commenting code.
[00:45:32] Andrew: readable now? See, you can't ever please an [00:45:35] engineer. It's not readable enough, it's too readable.
[00:45:38] Nathan: No, I, I, I never said it's, I [00:45:40] never said it's not readable enough. But yeah, to your, to your point it do- [00:45:45] it does help with that. One w- maybe it's a side thing there, I don't wanna say it too, too much, but [00:45:50] we're talking a little bit about code context and re- human-readable code. I- [00:45:55] if we wrote languages specifically for machines to communicate with the machines, it would look [00:46:00] very, very different. It would be much more data optimized, right? You
[00:46:03] Andrew: That'll be
[00:46:03] Andrew: next
[00:46:04] Nathan: yeah, [00:46:05] you can imagine exactly i- in the future where it's not writing code for humans, it's writing code for [00:46:10] machines, and in that sense it's, it's straight up machine code.
[00:46:13] Andrew: Wouldn't it be so much more [00:46:15] efficient if it just spoke assembly?
[00:46:16] Nathan: Yeah. Y- exact- exactly.
[00:46:18] Nathan: Your, your token... You wouldn't have to [00:46:20] go through this whole tokenization process and,
[00:46:22] Andrew: And I, and it,
[00:46:23] Nathan: yeah
[00:46:24] Andrew: It's a, that's [00:46:25] a, that'd be a fascinating conversation to have actually, 'cause ' cause I'm thinking like, binary, but then I was like, [00:46:30] actually binary sucks 'cause you basically have to run the full string in order [00:46:35] to know if it works. Whereas something like Assembly you can still take, I forget what they're called, but it's like [00:46:40] basically phrases, right?
[00:46:41] Andrew: It's like assembly strings or something where it's like i- it's [00:46:45] an understood execution layer on, on something on the, [00:46:50] the hardware you're working on and the register or something, and you can kind of like register program [00:46:55] through Assembly and like the people who'd know it, know it. Do you know enough
[00:46:58] Andrew: about
[00:46:58] Andrew: Assembly? [00:47:00] I don't.
[00:47:00] Nathan: a--
[00:47:01] Andrew: Yeah.
[00:47:02] Nathan: like I, I've heard about a company who is, who is training a model [00:47:05] specifically for that purpose, like on that level, that, that low enough level where you
[00:47:09] Nathan: can [00:47:10] hyper, uh
[00:47:11] Andrew: for folks, I don't know how you explain this in like a [00:47:15] easy... It, it's
[00:47:15] Andrew: like...
[00:47:16] Nathan: It just, it's low level. It's, it's on
[00:47:18] Andrew: Yeah.
[00:47:19] Nathan: never need to read or [00:47:20] understand
[00:47:20] Andrew: I guess
[00:47:21] Andrew: like for,
[00:47:22] Andrew: for Legos if, if a coding [00:47:25] language is like the bricks, assembly is like the plastic, [00:47:30] and instead of just taking a bunch of pre-built bricks and making something, you just custom make [00:47:35] the plastic to do whatever you want
[00:47:37] Nathan: Oh, an interesting analogy. There are [00:47:40] different, yeah, d- different architectures have different assembly language instructions, so it does look a [00:47:45] little different. So it's different, maybe
[00:47:46] Andrew: And it's a lot
[00:47:46] Andrew: f- yeah, and it's faster because any sort of [00:47:50] interpretive language has to run through... So all the words that you use get interpreted to [00:47:55] logic, and then that logic gets interpreted to effectively assembly most of the time, and then it [00:48:00] runs. Where as if you just write, it's summarizing a, a paragraph in a couple [00:48:05] sentences.
[00:48:06] Andrew: Like if your house is on fire, you can be like your house is on fire and you should get [00:48:10] out," or you can say, "Get out," right?
[00:48:13] Andrew: You don't have to explain the fire. You don't [00:48:15] have to explain why fire is bad or, or, or, or where it is. You're just like, "Get out," [00:48:20] right? It's very effective ways for machines to run.
[00:48:24] Andrew: But [00:48:25] we'll see. That'll be the next big innovation that AI will make, right? Oh, we've invented a language that [00:48:30] only AI can understand, and it's 20% better. Weird. Doesn't have all the [00:48:35] crappy human shit that doesn't make any sense
[00:48:37] Nathan: That's the prediction for
[00:48:39] Andrew: 2027.
[00:48:39] Andrew: You heard it here [00:48:40] first. All right
[00:48:41] Nathan: All right, take
[00:48:42] Nathan: care.
[00:48:42] Andrew: I'm done now.
[00:48:43] Nathan: Okay.
[00:48:44] Andrew: [00:48:45] Andrew's weekly rant is over.
[00:48:47] Nathan: That's...
[00:48:48] Andrew: Take care
[00:48:49] Nathan: All right, [00:48:50] bye