From Swift to Mojo and high-performance AI Engineering with Chris Lattner

And so I just started again nights and weekends. Didn't ask permission. Just started fiddling around seeing what could be done that would be better. The first year and a half it was literally just me working in nights and weekends and I had a day job and I was managing a big team and a lot of stuff going on. >> Hold on. So at this point, yeah, you were already second level manager or something and had a team of 40 people running a lot of very interesting and very fun stuff. >> You're good at juggling stuff. >> This is a passion project. I'm also a fairly good programmer, so that helps. >> How do you go from this blank canvas to like, okay, here's what I think the language will be. >> I start from a lot of blank canvases across my career, right? So, LVM blank canvas, ML later blank canvas, Mojo, Swift, many of the systems I build are blank canvas projects. >> Do you think there is any logic in having a new language that is designed to make it easy for LMS to code with? So, I often get asked given the AI is writing all the code, why are you building a programming language? Which is another way of asking the same question, maybe a little bit more aggro. And so to me, I don't think that optimizing for the LLM is the right thing to do at all because the important thing Chris Lner created some of the most influential programming language and compile technologies of the past 20 years. He's a creator of LLVM used by languages like Swift, Rust, and C++. Created the Swift programming language, worked on TensorFlow, and now works on the Mojo programming language. In this conversation, we cover the original story of LLVM and how Chris managed to convince Apple to move all major Apple dev tools over to support this new technology. how Chris created Swift at Apple, including how we worked on this new language in secrecy for a year and a half, and why Mojo is a language Chris expects to help build efficient AI programs easier and faster, how Chris uses AI tools, and what productivity improvements he sees as a very experienced programmer, and many more. If you'd like to understand how a truly standout software engineer like Chris thinks and gets things done, and how he's designing a language that could be a very important part of AI engineering, then this episode is for you. This podcast episode is presented by Statsig, the unified platform for flags, analytics, experiments, and more. Check out the show notes to learn more about them and our other season sponsor. So, Chris, welcome to the podcast. >> Well, thank you for having me. It is so nice to meet you in person because I feel like the work that you did has had an impact personally on me at Adooper. We we migrate to Swift and of course a lot of the software we use runs on things that you've built. So can we rewind time back all the way to the early 2000s and for some of us who have not really been around there can you describe like what was the industry like in terms of compilers, languages and what was the status quo back then? Yeah. So I mean I was involved in the kind of early days of Linux and so I started maybe not the really early days but I started working with Linux in the mid '9s and this is when Java was coming on the scene and a lot of things were changing in the industry as that played out. Uh GCC was the compiler that that standardized a lot of software and a lot of u people were using it to build Linux based software and this is how I got to know it. Um GCC is a great thing. Um, before GCC, a lot of people don't know this, every hardware vendor was making their own compiler and it was a gigantic mess for even just CC code because none of these compilers were compatible with each other. >> And and the reason they did that, just so I understand, is like you have C code, it needed to translate to assembly for a specific hardware and then the hardware vendor, you know, they did the mapping and figuring out like what custom instructions or something like that. Was that a reason? >> Yeah. So basically back in particularly like the late 80s and then the early 90s there was a lot more diversity in terms of like instruction sets and chip makers were innovating and you had HP building things and you had uh Intel was making a lot different systems back then and there there's all kinds of different stuff going on and so you had risk computers that were being invented and so the challenge at the time was that everybody had to build their own compiler and so because everybody had to build their own compiler they all wanted C and some C++ was emerging um and C was a standard and so there was a spec that said this is what the code is supposed to look like but as we know for many standards and many specifications they're never complete and so what ended up happening is each of those compilers was a gigantic mess because they had different bugs they had different misfeatures they lacked certain capabilities and so software of the day was built with systems like autocomp which was this really weird macro processing thingy that tried to work around some of the limitations and it was a gigantic nightmare so GCC came out scene kind of in the '9s really and uh clean up clean up that mess. It became the standardized thing that people could plug into and a lot of chipmakers embraced it and it was free software and so that led to the rise of Linux. Linux adopted it and that really kind of brought the world forward. Now GCC was a good thing. It really did help standardize the world and and I think a lot of free software and open source pays a debt of gratitude towards GCC but also it was a really old thing and so I made fun of it at the time. It was over 20 years old and and its architecture was, you know, very very old school in many different ways. It was also not built to be a modular design. It had, you know, was designed to do one thing, take C code in and then put out assembly code for a given chip. And so there's a lot of things you couldn't do like JIT compilation and you couldn't at the time even optimize across files within one >> and and JIT is just in time compilation, right? >> Just in time compilation. Exactly. And so and so there's a lot of things that people wanted to do that you couldn't do with GCC. And so um in around 2000 that's when I was in university and I was studying compilers and I said oh okay well wouldn't it be interesting to build something new in the space and so I started working on LVM LVM is a it started out as a code generation system so you could target multiple different architectures but really for me it was a learning process it was about saying okay compilers are cool don't let anybody tell you otherwise compilers are cool uh >> even today right >> even today and so um and so but I didn't really understand anything about it and So I wanted to learn by building and so across my university project I built this thing up more and more and more. We then open sourced it. It got a little bit of a community. Uh later I went to Apple which really helped foster and fund a lot of the development and that early starting point was coming from you know GCC and open source technologies are really good but there's a lot of things we can't do. And so that's where I kind of fell down this rabbit hole. >> And like when you started and and open source what did you open source? What could it do? >> Yeah. And then you know what was the reaction to to this early version of LLVM? >> Oh, so LVM back in the day and this is super funny because many people look back on uh successful systems and they assume that everything was obvious when when actually every step along the way is challenging and you have to earn any success you get. Very very rarely do you like luck into it. And so when I first started working at LVM again it was a research project at a university. There's a lot of research projects at a university that don't go anywhere. And so >> I'd argue most of them probably won't go anywhere. >> Exactly. And so when and the default assumption is that LVM also would not go anywhere. That's a safe assumption for any vers. And so we used it internally. And so my adviser Vicram Madve encouraged us to continue building it and then we taught it to the to a couple of classes. And so we had a few people using it internally and we got some like use case with it. But at that point it was really just optimization and code generation. And so it plugged into GCC used the parser and the to actually parse C C code for example and so it was just about code generation and it was useful for compiler people trying to learn things but it wasn't very useful for general application developers really when we open sourced it then we got I don't know two or three people that were interested in it and it was mostly other compiler nerds that you know are delightful um but there was no major community there was no major reason for people to contribute and what I did was I said okay well actually just treat the world like the rest of the research group and just have open development, encourage people if they they come by and they want to help and do something awesome. If they have questions, I'll answer them and just treat people with respect. And what happened over time is slowly the community grew. We got in individual people that were interested in different things. Some people were interested in esoteric programming languages and and they were interested in compiler technology for that reason. other people were interested in uh performance and different people had different interests and so very very very slowly it kind of grew and what was the big difference between GCC and LLVM right like of course modularity was one thing but but also you mentioned capabilities that you wanted to do that GCC was just either couldn't do or was really difficult to do what were those capabilities >> yeah so so originally it was just in time compilation so runtime code generation >> that was a thing with GC >> couldn't do they could do only compyon >> they could only do a kind of batch upfront traditional Unix style code generation. At the time, uh, GCC could not optimize across files in your project. And so if you had a C, if you had a function in one C file, it could not inline it into another C file. So things like that. LVM was also written in C++, which was highly controversial at the time because >> GCC was all C and uh, Richard Stallman was very opposed to C++. And so um actually there's a parallel universe because in 2005 I had joined Apple. We decided okay well let's see if maybe LVM and GCC should merge and actually proposed to the GCC community hey we've got this interesting technology this seems very complimentary it could lift the architecture GC maybe we should do this and uh it didn't go anywhere because primarily was written in C++ but it was also not invented here there's a bunch of other you know very serious very experienced people and they're like okay kid what do you why would we listen to you and so it did not go anywhere and thankfully so um it made meant that LVM had to grow up on its Yeah. >> And then did LLVM really start to take off when you got to move to Apple to now work on it full-time? I assume there must have been a at least a small team like investing in this. Is that how it went or or something? >> Yeah. So, so the the story of getting to Apple. So, I when I was graduating, I was looking for uh jobs in compilers and jobs that ideally would let me work work on LVM and continue the work because it was still an advanced research project. It was 5 years in, the community grown quite a bit. Um, I was doing regular releases every six months and so was really putting a lot of energy into advocating for the work and showing momentum and velocity. I caught the eye of a VP at Apple and there was a couple of people that were working on adding a Power PC back end to LVM because that's what Apple was doing. And so I got to know them and they said, "Hey, come like we are frustrated with GCC. That is the foundation of all of our all of our uh compiler technology >> because GCC uh compiled Objective C for example." Right. >> That's right. Exactly. And so and it was again an older architecture. It was very difficult to work with. They weren't getting the performance they wanted. It was very difficult to add features. The community was also kind of annoyed with Apple for a variety of reasons. And so basically management said, "Come like work on LVM." And I said, "Yes, sign me up." And so when I started um they did give me one other engineer to work with, but really they didn't give me a whole lot of guidance. And so I was like, "Okay, >> you pretty much >> cool. I will I will I will uh go implement PowerP PC support and integrate with Mac the Mac OS back in the day and things like this. And then there became a time when they said, "Okay, cool. You're working on this." And my manager said, "It's great that you're working on this cool technology, but at some point we need to make sure that it's actually relevant to Apple. >> Was it not relevant?" >> Well, it wasn't used in any products yet. >> Oh, I see you added support, but it wasn't used just yet. >> Just yet. Yeah, exactly. And it wasn't as good as GCC in a number of different ways. And so I basically got the vibe that it's like, okay, well after a year if Apple's not using some product, then we'll ask you to start doing something else cuz we're not just going to pay you to work on an open source project actually have impact on our company. And so when I got that memo suddenly and my manager helped me a lot, he was amazing. Went around shopping for okay, well, what are the the near-term impact that we could have on the business? And so from there, the first use case was actually for graphics. And so OpenGL was doing just in time compilation uh to do some graphics stuff. And so we were able to do something very small that actually had value. And suddenly aha this is not just a science project. This is actually interesting. Now it's still missing tons of features. But that enabled the development of the next set of features, the next set of features, the next set of features >> and it was now in a product that Apple actually used and shipped and it was you know generating business value if you want to put it like that. >> A small amount but at least non zero. >> Non zero. >> Exactly. And so and so what I did over the course of many years across Apple is then say okay cool we'll keep investing the technology keep building an open technology and an open team so community but make sure to deliver more and more and more and more value to Apple and as that happened suddenly what ended up happening is I end up replacing all of the developer tools compiler code generation debugger technologies within Apple and that >> one one step at a time. >> Exactly. And so over the it took about 5 years to the point where we built a new C++ parser clang and Objective C front end and all this kind of stuff. We got to about 2010 that's when Apple was releasing the first 64-bit iPhone. And suddenly this was made possible by LVM by all the technology we had built and all this kind of stuff. And it was a it was a epic moment in the industry because everybody is convinced that 64-bit phones were stupid. That was not a thing. and 64 bits doesn't make sense. Why are you going to have more than 4 GB of RAM in a phone? Right. Hilarious. >> Wow. Back in the day. But that's how people thought, right? Like >> that's how they thought. And they're like, "It's inefficient. It can never work." And so it actually the first 64-bit iPhone, it was the iPhone 5S shipped in production before ARM got their chips back to test. And so we had built the entire software stack. We enabled the entire operating system, the entire tool chain, all this kind of stuff internally. And we were collaborating with ARM. But they had no idea how far ahead we were. And then suddenly we're shipping it in production and the whole world's heads explode because how good the performance was and how all the capabilities we enabled and it was it was quite fun. So I'm having just trouble putting two things together and you can help me out with with what I'm missing on on one end you know you made it seem like okay like you know initially when you got to Apple for a year it was a experimental project then you just went slowly with one team and after the other but now next thing we know fast forward to four or five years later it's actually powering like like the core of Apple. How h how did it go from like you know the the the small projects to actually getting into like the the core of of the business? Did it help that you started with developer tools and more developers were convinced like how did you get through to you know like probably the highest level kind of architects and decision makers who in the end probably I assume they had to take a risk on technology that was like somewhat new compared to like at least GCC that's been around for like 25 years at this point. >> Well, so it's a combination of having top down support. So people that wanted me to be successful and that's because they're frustrated with GCC and so they wanted a new technology. So that was very helpful. A combination of bottom-up success and it wasn't one thing. It was like every 6 months we'd have a new thing that worked well and and go. Um I became known as somebody who'd get things done and so I got more scope and responsibility and so my you know I went from being an engineer to a a manager to a second line manager to eventually a senior director over the course of number of years. Um, and so it wasn't one thing. It was just a lot of hard work and it was a lot of fun because we were able to, you know, a a great thing about Apple back back in that time was that you could have a lot of impact because the team was growing, but also if you could get stuff done, you had an amazing uh platform in which to work because the iPhone came into existence and a whole bunch of other technologies were made possible. And Objective C was very Apple specific, but we could move it. That was huge. We added a whole bunch of features to Objective C. Um and so as we built into this like there's a lot of opportunity but it was hard one right it wasn't in in any given moment it wasn't uh guaranteed but as uh my team at Apple was making progress we were doing all this in the open then suddenly other people started seeing value and like Cray built a supercomputer and used LVM for it and Google eventually adopted it 2010 and started using it within Google for some small things and then kind of a parallel project within Google of another hero within Google like built a team and add a lot of impact to Google. Therefore, justifying more people to work on LVM within Google and different tooling and different projects and different things that all happened eventually, you know, uh, Intel or ARM or these companies ended up canceling their internal compilers and switching to LVM. >> And so then suddenly the vast majority of their engineering teams all working on a thing and then what you get is you get value that compounds and you get a today LVM has a community of thousands of people that come together and so it's amazing to see that. >> It's incredible. It it it feels a little bit like when you're starting to roll snow falls and you're starting to roll this small thing and then you know you do it for long enough and most people don't do it but then you you could have a snowman and you're like oh where did you get all that snow from? Just incredible. >> Well, so I mean it's funny because people always imagine success, right? And so you you start from a starting point and you say I want to be the president of the United States or something, right? But but and some people get very motivated by that and it's very powerful. Uh what I really love is doing all the work, taking each of the steps, doing the thinkless thing, figuring out that really obscure thing that you know if you get the architecture right, it makes the whole thing compound and scale and compose the right way and it allows you to solve problems that nobody else can solve. But what that means is most of the time I'm working on things people don't understand. And so I'm fine with that. I've normalized to this. This is what I expect at this point. I don't expect people to understand me. But what happens is that these projects over time they grow and they get to the point where suddenly it clicks and suddenly people start to understand and suddenly you can explain it because it maps into their value system in a way that you know they can measure. Basically Chris just mentioned how things click a lot more for people if it maps to their value system in a way that they can measure it. In my experience measuring things is helpful all around. Not just when trying to convince people but also when figuring out if a feature you built works as you expect. Here's a challenge most teams face. They ship features but cannot easily measure if they're working. Did that new checkout flow improve conversion? Is the feature helping retention or hurting it? Without measurement, you're just hoping things will work. That's where Static comes in. Static is our presenting partner for the season. And they've challenge the status quo of how product teams work. Much like how Chris challenged how compilers and programming languages are supposed to work. Most teams accept fragmented tools. Feature flags in one system, analytics in another, experiment somewhere else. You're stitching together point solutions, running ETL jobs to sync user segments, hoping time stamps align across different systems. That's a status quo, but it doesn't have to be. Static built a unified platform that gives you everything in one place. Feature flags, experimentation, analytics, and session replay all using the same user assignments and event tracking. So you ship feature to 10% of users and the other 90% automatically become your control group. You can immediately see if conversion changed. Drill down to where users drop off in your funnel. Then watch session recordings to understand what went wrong. All with the same data. This is how you measure impact at the pace you ship. Companies like Graphite, Notion, and Brex rely on this approach to make datadriven decisions without waiting for data teams or manually correlating information across tools. Static has a generous free tier to get started and propricing for teams starts at $150 per month. To learn more and get a 30-day enterprise trial, go to stats.com/pragmatic. with this. Let's get back to the conversation with Chris. One one one thing I feel that is probably pretty like applicable for anyone is is the fact that you know you were inside a company that you know like was pretty big and successful at that time Apple and you still managed to get so much things done just as as you said by by solving hard technical problems continuously figuring out how to help the business and there was this interesting part where where Clang got open sourced and Apple back then LLVM was open source so that kind of I understand grandfathered in. Yeah. >> But I want to ask you on how you managed to convince a company that back then was pretty close and even today is pretty close outside of a few things to be okay open sourcing a new project. And I assume your reputation or hard work or getting things done might have been there. But but what else did you kind of use for for for leadership to like green light this? Because I you know I'm sure top down they needed to say like yes, right? >> Yeah. Well, so so Clang was actually the easy one because we we didn't really ask too many people for permission. We just kind of did it and so that that was the easy one. The the hard one was Swift. >> Let's let's get to >> So I think we'll get there. But but that that was the actual hard one. >> And and with them, let's get to Swift. The story of Swift as you as you shared uh many times is how you started to work on it in secret. Can you tell me like what led you to start experimenting with a new language? what you saw the problems being with Objective C especially because now we solved the compiler problem like you know the comp compilation was probably as as good as it it could have gone and how did you pull off this kind of what is secret mean right >> yeah well so so the backstory on Swift is that uh so I joined Apple in 2005 and I built and basically replplumb their entire C++ and C and Objective C tool chain and by 2010 at WWDC which is their summer conference um we had launched a full stack replacement and it is working and building a full stack fully integrated C++ compiler is no small feat. C++ was even then was a very complicated language and it was a very big technical challenge but it was also very demotivating because C++ at least to me because C++ is just a beast of a language and so you couldn't come out of that and not wonder could we do something better right and as you said we built a lot of this fundamental technology could build pretty much anything we wanted at that point and so I just started again night nights and weekends didn't ask permission just started fiddling around seeing what what what could be done that would be better and to me what I did was I actually said okay let's go look at a lot of existing languages so let's go look at both the new ones um let's look at Java or let's look at C++ things but let's also look at functional languages like okammo let's and Haskell and things like this let's also look at esoteric languages like the the really weird uh other things let's look at TypeScript and uh Dart and other things that were kind of on the scene and happening and Go and languages like that and so it was really about saying like well what do I like and how do I see success and what I defined success as with Swift was making it easy to use and scalable. And so I saw a lot of languages JavaScript is probably the best example of this where you know you start with something really simple and you know a weekend hack with JavaScript and then and then it gets adoption and then you get developers and then developers bring their tools to other domains and so you know JavaScript was designed for onclick handlers but now people are running web servers in it. >> Yep. Well, and so >> we see it at all all too much. >> Exactly. And so what my my observation was is that success meant that you would be naturally brought into other domains and so think beyond just the the first win but think towards more generality and scale. And what I realized is if you take something like Objective C which I was deeply familiar with for example you know this dichotomy between objects and uh C. So Objective C had this small talk inspired object model and then it had C for performance. It was actually a really beautiful combination because you can get all the way down to the metal with C. Uh, but you had high level APIs. And what I realized is that they didn't have to be two languages. >> So, so you could you could take the good parts >> and pull it together into one system that could actually scale and would be way easier to teach, way more memory safe, way more modern, good type inference, fix some of the syntax problems because Objective C was a little bit of a a mashup between two different worlds. And the funny thing about that is that you fast forward all the way to today. That's exactly what Python is. Python is a beautiful language for objects built on top of C and C++ and Rust and it's got this two world problem going on within it. And so it's very interesting to me just again how history rhymes with itself and there's so many different things going on. But the initial the initial ideas of swift was really about saying how do we build a scalable language and how do we pull together the best ideas shamelessly you know borrowing ideas from different places >> and by scalable as as as you meant that that means that it can be used for like your specific task that may that be let's say for iOS but then it naturally lends itself to like other domains >> yeah and and so specifically scaling all the way down to embedded systems development all the way up to something that kind of approach scripting right so super easy use for very high level programming >> can you How me how did you design a language in the sense of like as as developers we we use a lot of languages right like again like I think all of us like have have tried and and built with with many different ones especially with AI tools we it's so easy to try out you know for myself like TypeScript rust go you know the the Haskell try out this and that prologue like and I'm used to using them right and I I kind of discover it figure out what is there but when you have a black canvas or when you were thinking like how do you go about it do you kind of like I don't a meta program like kind and write down what you love to see. Uh you also of course built compilers you start to immediately think of of how that would work like can you give us a little bit of a a view of like what what how do you go from this blank canvas to like okay here's what I think the language will be >> well so that's a good question for me in general because I start from a lot of blank canvases across my career right so LVM blank canvas y right mir later blank canvas mojo swift like many of the systems I build are blank canvas projects and so for me it's it's a couple of things first of all doing that kind of analysis the reflection the deciding what is success ultimately look like? Then do the survey. Go look at what's out there. What's good? What's bad? One of the things I really strongly believe is that you can take a a system that you don't like, but buried within a system, even if you don't like it, there are often good ideas. And so you can take the ideas that are good and ignore all the rest. So, for example, Java, I didn't like uh like garbage collection and finalizers and like there's all this stuff that came with Java, but being able to be JIT compiled was very powerful, right? And so there's a lot of things you can look at and you can say and Java is a huge step forward from the technology space for a variety of reasons. But you can look at that and you can say okay well I like that one aspect of it. All the rest of these things are separable and maybe I don't like how they work and do that survey. And then when I start building a new system I expect to design and redesign and iterate and learn and grow. And what I try to do is I try to go through proof points and validate specific assumptions and say okay I'm not going to solve all the world's problems. I'll keep this part and this part and this part super simple. It's like you know the two circles that make your owl just leave them as circles and then uh and then invest in proving this piece and if I can get this piece done and I can understand that it works and I can gain valid validation and confidence in that. I can know the interfaces going in and coming out of it well now I can go out into the next system and build that out and build this thing out so that we can um understand it and learn as we go. but every step along the way being unafraid to go massively change and flip over the table and try try again. And so you started to experiment with Swift uh like the kind of weekends and on the side in 2010ish. How did it take until 2014 for a first release like what really goes into building out a language? Because again I I think we can all I can kind of emphasize like this is cool like you have these ideas of course you have a lot of experience you try out things but then like as a as a software engineer I shouldn't ask this but I'm going to ask what took so long. >> Oh well so it turns out building programming languages is not easy uh takes about four years I guess. >> Can you give us a bit of like emp emphasy of like you know like what like makes it so hard? Yeah. Well, so so let me let me just frame up how it went just so you know because uh the first year and a half it was literally just me working on it. >> Oh, >> okay. So in nights and weekends and I had a day job and I was managing a big team and a lot of stuff going on. >> Hold on. So at this point, yeah, you were already like pretty >> second level manager or something and had a team of 40 people and running Yeah. a lot of a lot of very interesting and very fun stuff. >> Well, okay. You're good at juggling stuff. >> This is passion project. Yeah. Yeah. I'm also a fairly good programmer so that helps. But after a year and a half got to the point where I told management about it. I said, "Hey, I'm working on this thing. What do you think?" They're like, "Uh, yeah. Uh, no. Why would we want a new language? Objective C is what made the iPhone successful." >> Right. >> Right. And so I'm like, "No, what?" I'm like, "Well, how about I just have like one or two of the people in my team like work on this part-time?" They're like, "Uh, I guess you can tell them about it, but we can't commit resources really, you know?" And so, and so, but you know, again, I was respected and doing good things. And so I was given a little bit more rope than some other people would. And I got to the point of saying like, okay, let's build up some demos. And then it was about 3 years in and most of the feedback I was given was, okay, well, if you don't like Objective C, go make Object C better. You own Objective C. Also, >> it's a natural reaction. I I mean, as as much as engineer, I don't want to hear it. If I put a little bit of a business hat on, it's what you would >> makes total sense. That's what the entire community is using. It's very low risk. It makes your existing investment even bigger. And so what I did was I said, "Okay, well, uh, across that four-year journey was not just building the thing full out with like a team of people that knew what they're doing. That's the discovery, the research, and also the socialization process with executive leadership." And so, um, what I realized, I said, "Okay, well, I want to have memory safety. Objective C does not have memory safety." >> That was one of the biggest things, >> retain and release, and it was completely manual, and it was just a kind of a nightmare. And so we invented ARC, so a way of doing reference counting automatically. >> That made Objecti way more teachable and safer. It also pulled it much closer to Swift. We then did modules. We then did literals. And Objecti, we added all these features that what what it was doing is was pulling the lived experience of doing Objective C programming closer to what Swift would be. So that ultimately when Swift came out, it would be less of an abrupt leap and it was still a pretty big leap. But um but we're we're doing that. Um but meanwhile, you know, management's like, "Okay, cool. Why are you doing this?" and they're having their own issues. And we got to about 3 years in and said, "Okay, well, we think we should do this in a serious way." At that point, I had a team of 250 peopleish, something like that. I was running Xcode and the whole developer tools team >> and and still on the weekend, you were still doing this. >> Well, yeah. You can look at my GitHub history like you can go back to 2014 and see what I was doing. Yes. But at that time, right, they they said there was this big executive review and a whole bunch of internal discussion. And I said basically, look, I want this to be the focus of the department to bring this into market. >> Y >> and so, um, at that point, we didn't really have it talking to iOS, right? It was still missing a lot of the object features and things like this. We had no apps that were built with it. And so that last year was a massive amount of work in terms of finishing the language, but also integrating with the debugger into Xcode into code formatting and to tons of different features. um building more uh integration with the iOS SDK that was a big deal and we had not done that and so a lot of that made it possible to finally launch it four years later and I would say when we launched it so mistake we made is we called it 1.0 >> I was about to come to this because I that's where I I have like firsthand experience with with 1.2 but yes 1.0 you know came out and you know the feedback I can just talk from you know like what what I remember from there there's two things people were super excited that Apple is releasing a new language and iPhone was was huge 2014 was was the time where like you knew that you could make a build a big business on iPhone Uber I think Snap all of these companies were just going up it was a status symbol already and not everyone had iPhone just yet however uh like when you tried it out it it was just not good Xcode didn't have refactoring support. Debugging was a nightmare. We should launch. >> So that's that's so h how did it go go for why why did you decide at the time like you know just I'm interested in your thought process at the time on how we went about it and you know like what were some learnings now that you look back? >> Yeah. So um so it had to be 1.0 to launch it because Apple doesn't launch 0.5s basically. So it if we wanted to launch it had to be called 1.0. So that was one aspect of it. But the other aspect is we wanted to get it out there. And at the point that we launched it, only about 250 people at Apple knew about it at all. Most of those people were in my team and then some executives in marketing and other people like this. And so we knew we couldn't build a good programming language in in a vacuum with an NDA that you had to sign to get access to it, right? You have to actually get it out there. You have to get usage experience. And so the learning for me is don't call something or 1.0 until it's actually ready and it's validated and you have a lot of usage experience and things like this. We bring a lot of these lessons to Mojo by the way. Chris Lner and the team launched Swift back in 2014 and it's since become the de facto way of building native iOS apps which brings us nicely to our season sponsor linear. Linear's iOS app is also built on Swift. It's a fully native app. It's not built with React Native or using a web rapper. And there's good reason for this. Linear's entire philosophy is about speed and performance. The desktop app is famously fast, but that same obsession about speed carries through to mobile. Linear have just redesigned their mobile app with something pretty interesting. Their own take on Apple's liquid glass design language. But instead of just using Apple's API under the hood, they rebuilt the UI from scratch. The linear team wanted more control over the navigation and customization than Apple's system would allow. Classic engineering mindset. If the existing solution doesn't give you what you need, build your own. Here's my favorite detail about the app, though. Linear has this feature called polls. It's basically a feed that gives you an update on all your development work, project updates, what's on track, what's blocked, etc. On mobile, you can actually listen to it as an audio briefing. The linear team told me how they get raving feedback from engineing leaders to listen to their polls update on their commute. It's like a personalized podcast about the team's work. One VP shared how he gets to his entire standup prep while driving to the office using polls. This is why I love partnering with engineering teams like Lineers. They're not just porting desktop features to mobile. They're actually thinking about how people use their phones differently. If you want to see what they're building, check out linear.app/pragmatic. You can use the mobile app to manage issue tracking on the go and that liquid glass implementation is genuinely beautiful engineering work. And now let's get back to Swift 1.0 and how the language is pretty painful to work with on launch back in 2014. But also it's about making sure that we communicate clearly with the community. And so one of the things I think we did well is we said look uh internally to Apple we said look we we need to launch this we know it won't be perfect and so we're going to tell the community that you can adopt it and you can build apps and submit them to the store but we will break your source code. >> Yep. >> And this is because we know that we have to change the language when we get usage experience. And so we we told people like we will help you but but but expect code breakage. And I'm glad we did that. >> I I remember that that was really clear. People were upset about it, but they knew like when we had the discussion of should we move over to Swift or not and uh at at Uber uh the rewrite happened in 2016 where Swift was at 1.2 and the the platform team made the decision after evaluation that they will take a risk and it was a big risk and there's now stories about it. I have a blog post there some other people with their experiences but they did it because they knew that Apple is committed. I think the open source part also helped build a lot of trust and I almost feel that that was one of the last times that I saw like the Apple community really unite against uh like iOS developers specifically >> just go like you know what we'll be in the pain together with we know it's going to be painful but like all mobile engineers I know knew that this was pain they told their you know their management and all that they said this is the right thing to do and of course I think it really helped that Apple had like this trajectory and it was like okay Well, they didn't want to be left behind. Uh they they wanted to risk not having access maybe in the future one day to APIs. Not at that point. One thing I was surprised when when Swift was launched, I don't know if this was right on launch day, but Xcode playgrounds and the ripple, the read about print loop was launched day >> was was also part of launch day. And you know, for those that don't know Xcode playgrounds, it was like you could just like try inside Xcode, you could just try Swift and and with ripple, you could also just really easily play. Why did you decide that that needed to be there on launch? >> Well, so part of that came from that year year-long process of like getting it to launch and so part of it was okay, if we're going to do a new language, what are we going to get out of it? What is the developer experience? Everybody wanted to get something, right? And so um for example, the documentation team who I love uh said this is our chance to write a book which was awesome because Swift had a book on launch and it was fantastic. Um the Xcode team said okay well and you know management and marketing people said okay well can we get interactive programming this is something we've never had and so we we want that and so that's where playgrounds came from and so um the ripple came from me saying okay well modern languages should have a ripple this is a thing that builds into the debugger and there's a bunch of different capabilities we could use and it would make a lot of sense to make it much more modern feeling and if we're going to do this let's do it right and so there's a lot of that okay if you're going to do it let's get the advantages from it now the flip side of it is we were massively overextended and so a lot of the vision was right but particularly at 1.0 and even 1.2 many of these things were still unbaked and it took a long time and I think that we underestimated how hard some of these things were and waiting for 2.0 or 3.0 though, you know, maybe it would have been better for some of this. But anyways, it's easy to be uh negative and judgmental in hindsight, right? At the time, we were all working really hard and trying to make something amazing. And you know, we're learning as we went. And I was really thankful for the community. But one one of the other things I'll say is that while you may have been on the oh wow, awesome thing and let's adopt and let's get get ahead of the curve. There are just as many people that were the what are you talking about? Objective C is great. If it ain't broke, don't fix it. and they're like shaking their fists and saying like I will never use a new thing and blah blah blah blah blah and there's all these bad things which they write about with Swift and so it took quite a long time for the community center of gravity to move and >> well well this was a fun fact in 2016 I remember middle of 2016 or maybe the a little bit earlier Uber was one of the big mobile first companies built on iOS grew up on iOS lived and breathe iOS and of course Android and the mobile platform team ran a survey in sometime early 2016, 2 years after Swift was out. This is Swift 1.2 saying we are planning to rewrite the app. The rewrite was a thing >> just because they needed needed to for architecture. >> We we needed to redo every single screen, every single interaction which kind of meant a rewrite all the business logic. So it was it was going to be rewrite no matter what. And they ran a survey would you prefer, you know, as an iOS engineer, Objective C or Swift? And the results were pretty much 50/50. So and these were again very experienced engineers. they they knew and the two camps were violently disagreeing with another and and I don't I still don't know why Uber went with uh with swift someone was a tiebreaker in the end but back then this is how foot on foot it was and of course you know there was grumbling there was problems so the objective people said like told you so we could have just done >> there was at some point a thinking of like maybe we should have go but as you say it was it was not trivial but but I guess this is what you don't like see in hindsight >> well so but I've learned a lot and again you reflect forward because this impacts exactly what I'm working on now. same thing which is that >> with Mojo >> experts don't like change >> right what I what I got >> this is counterintuitive >> well I mean if you think about it like if you're if you're an expert objective C programmer I don't know if you were but if you're an expert Objective C programmer you've been doing it for five or 10 years you've learned all the baggage tricks all the weird cases you know all the arcane thing you are the person that people turn to for help comes out now your expertise is invalidated you're on day one just like everybody else you are not the expert You don't really want a new thing. You want to be the king of the hill of the old thing. And you don't want the new thing to be successful. And so now some people are intellectually flexible and will switch over and there are early adopters and things like this or people that like new things. And so it's not true of everybody, but it is actually sensible human behavior to protect your your prior ex investment. And for a lot of people, you have to have a really good reason to switch over. And so in the case of Swift, you know, Swift UI was like a reason that moved some of the longtail people, >> which was years later, >> years later, right? So there's different inflection points. And so what you look at is it's almost technology diffusion, >> right? And so they're early adopters and they'll get on get on board quickly just because it's a shiny new thing. >> Yeah. >> Um and then there are some people you have to drag objectives out. They're called dead hands, right? And so like everybody else gets mapped somewhere along this curve and it kind of looks like an S-curve. But this is interesting because this will you know we're we're going to get to like some of the like AI and LMS but I feel this playing out. >> That's right. That's exactly the same thing and it's it's part of human behavior. It's not about objective C. Now let me tell you the thing that I find most that I'm most proud of in this journey. So when I started 2010 and it was just a toy project like playing around with in spare time not expecting to go anywhere. I was kind of implicitly coming from the assumption that we could do something that was better than C++ and Objective C and that was more easy to use and things like this, but didn't really know how it would go. After we launched, I would have people that stopped me in the streets and said, "Thank you, Chris." Like, because of Swift and because of not just me, but the whole team uh working on this, um I was able to become an app developer. Before Swift, I tried building apps with Objective C, but it I could never figure out the pointers and the square brackets and it would crash and I couldn't like I'm not smart enough to build an app. model. >> But now with Swift, it became easy enough that even I could do it. And now I am an app developer instead of a web developer. And I've become the expert in my company. And now I have learned and grown in my career. And so now thanks to this new technology like I progressed. And if not for this enabling technology, it never would have happened. And that's the exact same thing happening with GPUs. Right now we have the exact same thing with CUDA and C++ and all these tools that are gatekeeping so many developers away from this emerging kind of compute and kind of technology. And it's the same exact thing. And so this is what really makes me passionate about working on this kind of technology because I believe in the power of programmers. I believe in the human potential of people that want to create things. And that's fundamentally what why I love software is that you can create anything that you can imagine. And and to me, that's what's so powerful about working in this space. >> I really feel it and and and see it and just looking back on Swift because you created it. It's it's your baby. It started as a as as a project. You you were a guardian of it or or shepherd or however you want to call it for a while and of course now you you stepped away. It has its own life. Looking back at it, uh how do you feel on how the language has evolved? What what are parts that you're really kind of happy on how it lived up? What are parts where you're you're you're thinking that maybe if you would have stayed longer, maybe you would have like tried to have it in different ways, if there's any. >> So, it's it's very hard for me to say if I'd stayed at Apple and if I had kept my my fists on and like controlled it, if it would have been better or worse, but I I can talk to the mistakes that I made, right? Because and they're learnings that I could bring forward and that way it's my fault and it's not me saying other people weren't able to do it. Right. >> Fair. I like that. But like early Swift, it was, you know, I'll just say it bluntly, I didn't know what I was doing, right? So I'd never done it before. I'd built a C++ compiler which had a spec. >> Well, it was the first language you ever built. >> It was the first language I had ever built, right? And so a lot of the the ideas going into it were new. A lot of the like the type-checking algorithms were exciting, but I didn't really know what I was doing. I'm also not a math guy, and so if I actually understood math, then maybe it would have been obvious. But things in Swift like expression too complex to type check in reasonable amount of time. That's my fault, right? And there there's specific features that cause that to happen and it's very difficult to fix. And um and so that's one mistake. Another mistake is that we were very ambitious in terms of adding features that we wanted to see in the language and we added them faster than the architecture could keep up. And so particularly if you talk about Swift one, but even today what ended up happening is the internals of the compiler and the design didn't really line up right the right way. And so you get a lot of things that work kind of 80% or 90% and they didn't quite fit together and then as you start adding more and more and more stuff it doesn't really line up right and it just gets more and more complicated. C++ has the same thing going by the way. Swift's not unique this way but but as a consequence of that like you get an emergent complexity that then developers can feel and swift today people make fun of too many keywords and things like this. Well, it's because the origin story was really not, in my opinion, didn't put enough energy into like really refactoring, really redesigning, really making sure the core was great. Instead, it was okay, let's get to app development as quickly as possible. And so, as a consequence of that, what happened is then, you know, I and many other good people put more stuff onto it and more stuff onto it and more stuff onto it and the language got more complicated. And so, you know, partially that's maybe in any individual decision, but also it's because some of those original things didn't line up the right way and so the complexity kind of compounded. >> Wow. I'm I'm just like hearing we have a lot of tech depth inside a language and it builds up. >> Yeah. Well, I mean, if you >> Well, I mean, this is just one way of putting it. Of course, you put it differently, but it just feels this all repeats itself. Like it's it's whatever project you do. >> Absolutely. Well, let me let me let me talk about like a simple example in C++. In C++, int is hardcoded into the language. >> Yep. >> Float is hardcoded into the language. Complex numbers are not. They're templates. Why is that? Why don't you just get rid of these concepts, have one fewer thing, and just make int and float part of the library, and then everything's more self- similar, and everything will work the same way, etc. And so, Swift fixed that problem, right? So int and float are now part of the library and all the stuff is much simpler in that way, right? But C++ you can't go back and fix it because it comes from C and so they added templates after C and so all the C stuff had to be hardcoded in. They couldn't fix that. And that's the origin story of why C++ works the way it does. When I was talking with Steve McConnell uh on on the podcast, he told me that he when he worked at Microsoft for a while, the best developer he met there did this thing where he wrote everything three times. First, he wrote a first version in like a week and a half, then a second version in 3 days, and a third version in a day. And when I think back of the history of Python that I just heard there's a doc there's documentary where where we can listen to it, it was also the second or third iteration of a language. And and what I'm also hearing now is is is also, you know, like in Swift, you saw some mistakes in C++, but you couldn't for some of the things, but but now we're we're going to get to Mojo as well. >> Well, and and that's always my goal, right? The goal is so you're always going to make mistakes in life just so there's things you can do. You can decide to make new mistakes. So learn from the past and make new mistakes, not old mistakes. And then you can make the decision to fix mistakes wherever possible. And then you try not to paint yourself into a corner. And so again, a thing that I think Swift did well is it said, "Okay, well, the initial release was 1.0. That was predestined, but we're going to break the language." And so Swift 1, Swift 2, ultimately to Swift 3 broke the language to try and make it better. And then only at Swift 3 did we say, "Okay, now it's stable, right?" And I think that was really good because can you imagine if it still swift one, >> but we be left with all these mistakes, >> all the mistakes, right? And so it was painful, but I think that was a good set of decisions that we made and it led to a much better better result. Yeah, there's there's a lot of trade-offs. So, as we get to what you're doing today, you're after Apple, you worked at a few interesting places, Tesla, Google Brain, High Five. Can you talk about what you've learned at each of these places and kind of how it's, you know, because I think it'll be interesting to look back on how it just all is leading to what you're doing now. >> Yeah, totally. So, so the way way I frame it up is that, you know, 1.0 of my career was at Apple and building developer tools and understanding CPUs and I built Open CL, which is a GPU thing also. and compilers for the GPU and like all this kind of stuff. And so learning a lot about that hardware software boundary and developers and developer tools and Xcode and like that whole thing, right? 2016 I fell in love with AI. And the reason was is that about 2016 the photos app had uh the ability to tell you is this a cat or is this a dog? And it was a very like it was inception v1 or some really ancient AI model was in there. I'm like I own developer tools. I know a lot about programming. I have no idea how to write an algorithm to detect a cat in a picture. How the heck does that work? So I fell down this rabbit hole in 2016 and this is what leads me to 2.0 of my journey which is a very AI focused like go figure all this stuff out and so Tesla is doing applied with self-driving cars. That's where I learned cafe and tensorflow and I realized tensorflow was a good thing but it could be a way better thing and so that led me to joining Google and helping scale TPUs there. I ended up owning the bottom part of TensorFlow, all the CPU, GPU, TPU, plus a bunch of other weird internal AS6 uh within TensorFlow, but notably I scaled the software platform for TPUs and across years uh learned both, hey, it's actually really hard to make AI software without CUDA. TPUs don't have CUDA, >> CUDA, which is Nvidia's uh way to program kernels, right? >> Yep. Yep. So, so basically the entire AI ecosystem back then but also today ends up being very heavily biased by Nvidia and by their API and software and language toolkit called CUDA. >> Mhm. Which >> like 20 years old or so? It's about 20 years old and so and it's awesome and it's very mature and there's a lot of good things about it but it's also 20 years old and so there's time and space for new things but when you're building an ASIC so TPUs are very large large scale data center training and inference accelerator very fancy very very uh frontier particularly back in 2017 you don't have software and so you have to create everything from scratch and then you have to get TensorFlow and PyTorch to talk to it and and nobody really understood how that worked and so across the years uh I learned so much and I'm so thankful for my experience at Google because I learned about the algorithms, I learned about AI, the frontier uh applications um like you know the attentions all you need paper was invented in that time frame on TPUs and so we wow >> yes and so tremendous amount of things going on large scale embedding systems for ads and recommenders and stuff like this was going on it was it was a brilliant time and so across that kind of an early version of what I'm doing now said okay well we need to reinvest in the software technology and the platform technologies and so I built built components of what is now I think pretty widespread AI software so there's this ML framework uh new runtime technologies a whole bunch of different things so each of these things ended up being kind of 2.0 of the first leg of my journey and so ML is a compiler framework it's kind of like LVM 2.0 And so it's it's a rough analogy and we probably won't go into ML full details but >> but it's for ML, right? >> Well, it's for it's for domain specific chips and specific chips. Yep. >> Yep. >> And um and so but I could not have built that without the experience of building L LVM, right? And so having gone through that experience and being willing to do a 2.0 a clean slate, you know, looking into the abyss, you know, you're standing on a cliff and every it's just blackness around you and you could do anything, but you have to decide where to start, right? That really helped having gone through that and having made a bunch of mistakes with LVM and being able to then fix them. And so we're able to build something that has now been widely adopted by roughly a every AI company or every AI chip company that exists which is very exciting. >> Wow. >> And through that journey sci-fi for example then said and then and Sci-Fi built specific like built CPUs right on on risk people one of the best in the world as I understand. >> Yes. So uh Sci-Fi builds risk five hardware. >> Risk five hardware. >> Yeah. And so I decided, okay, I've always been on just the software side of the hardware software boundary. How about we go play in the other side? That sounds fun. And so I learned a tremendous amount about physical design and chip design and the business model of hardware and IPs and like all all the different technology for verification and things like this and uh and tremendous amount of fun. And uh one of the things we decided to do is let's go build an AI chip or an AI IP. And so you get to that and then suddenly you get to where's the software? And so across like each of these legs of my journey, I'm like, okay, where is the software? Where's the software? Where's the software? And in the case of AI, there's basically nothing there. There's nothing that you could take off the shelf that provides an endto-end AI solution that you can then just go implement your chip. And so I mean, the origin story of modular really came down to my co-founder and I who were buddies. as we got to know each other previously at Google, but really saying like, you know, we need something like LVM, but for AI, >> but for AI, >> like we need the thing that people can implement support for their chip into it and then get an AI software stack because, you know, if you're building CPUs 20 years ago, you didn't want to have to build a whole C++ compiler and C compiler and code generator and kernel and web browser. You just want to do a little bit of work and then get software. And that doesn't exist for AI right now. And then for for for those of us who are software engineers but we don't build AI software we we just you know we use the models we we might invoke the API or we might invoke the trained one. Can you explain to us if you know we moved into the the world of actually building AI applications running them directly on on GPUs may that be Nvidia or now there's AMD or or Google TPUs. What software would we use today? like what is the status quo or what was at least the status quo when you started modular and then what is your vision which I I sense there's a bunch of similarities with like GCC and LLVM here. >> Yep. Absolutely. So um so before GCC every chip maker had to build their own software say that's literally the shape of AI software today because there is no thing to plug into besides modular. Uh everybody has to build their own vertical software say and so when I was at Google we built a thing called XLA that's the thingy that we built kind of like CUDA but for Google TPUs. >> Google TPUs. >> If you look at Apple they have you know metal and MLX and uh their their whole stack. If you go look at AMD, they have Rockm, their whole stack. Everybody has to have their own verticalized stack and they share very low code. >> Oh, so this is why I I I was just looking at uh I talked with some folks at Entropic and look at their job sites and I saw that they're hiring both CUDA engineers but also Google TPU kernel engineers >> and Trrenium which is the AWS >> and and Trrenium >> and so they're writing the same models three times over and over again >> depending on the hardware they do and then I guess when it's a new version. Okay, I I get it. So this this is where we are today or where we were at least. >> Well, that and so like you mentioned Anthropic, they have an amazing engineering blog post from a few weeks ago. I can remind you and I can dig up the link. >> Yeah, we'll put it in the show notes. >> It talks about the problem of reimploying same models three times over. >> Turns out they don't work right. You have bugs. You have quality problems that then users can feel in your product. You have outages because things three times in parallel with three different teams, of course, they don't line up. So where modular came from was saying okay well I've lived this experience with Google with TPUs and a number of other chips by the way but TPUs are the public one um have seen this at sci-fi building into the hardware ecosystem at that point this was uh about four years ago at this point um the AI hardware ecosystem wasn't mature but suddenly I had built up this wealth of knowledge of like how to do this stuff and when I looked around at all these projects I looked at PyTorch I looked at I there's onyx runtime time and like there's a million of these things out there. I didn't think anybody was on the right path. Nobody was willing to do the work because what I saw is I saw the problems I faced at Google. I love my experience at Google. I learned a tremendous amount that people are amazingly brilliant. But the problem that I had and that we had as a team was that every year there'd be a new chip and every year you're scrambling to get the next chip to work. At the same time, all the product teams are using the current chip. And so all your best people are getting assigned to put out the burning fire for search or ads or whatever the work YouTube whatever the workload is. And so you could never actually get the mindset or the bandwidth to be able to do something fundamentally new particularly if it took more than a 6 or 12 month performance review cycle. M >> oh yeah Google is infamous for that and yeah >> and so and so what I saw is I saw a lot of these very short-term projects and a lot of improvements and a lot of micro optimizations and a lot lot of great work a lot of massive innovation and models and the architectures but the progress the research progress was happening because Google had infinitely smart engineers that had built every level of the stack and could hack the daylights out of it just to make simple things work. It wasn't scalable. It wasn't beautiful. wasn't like built to compose and it certainly wasn't built to bring up new hardware quickly. And so, as you know, I've been working in compilers and dev tools and languages and stuff for quite a long time. And so, I said, "Okay, well, I think it's finally time to solve this problem." But to solve it, you can't solve it in a weekend with like three people. You need years of time and a deep, very high-tech team to be able to invest in this technology because CUDA, as you mentioned, is 20 years old. It's had thousands of people work on it. huge investment. Um, any one of these chips and these algorithms is extremely complicated. The AI research landscape moves really fast. You need a huge amount of effort and the only way to tackle this was with a V VC funded startup because that's the only way we could pull people out of Apple and Nvidia and Meta and Google and all these companies to be able to go build this. >> And you said something interesting in a previous uh podcast or interview. You said between kernel engineers, compiler engineers, and AI engineers, they just kind of think differently and there's not much overlap. Can you tell me a little bit about like cuz you have been all three. You of course know so many of them, but like what is the the difference in the end? You know, they all need to work together at a place like this, but you said that somehow they don't like >> understand each other. >> Yeah. So, I'll tell you the um the aha moment that came to our technology approach for modular. Again, if you go back four years ago, you're talking um 2021. Funny how that works. Okay, so late late pandemic, a lot of amazing stuff had been done and a lot of people had built uh first of all uh TensorFlow and PyTorch. I consider them to be some of the very early production frameworks. Uh Tensflow, PyTorch both come from 2015, by the way. So they're nearly 10 years old, which is ancient in AI time, right? And so they were built under this idea of we will take CUDA kernels and we'll take uh Intel MKL kernels and then we'll build Python APIs on top of these. And so one of the challenges with that generation system is that um it was so easy to write kernels that people made thousands and thousands of kernels but then they couldn't be ported and you had to rewrite thousands and thousands of kernels to be able to bring up a new chip. Uh when we worked on TPUs we said aha you know what's cool compilers. And so instead of writing thousands and thousands of kernels, what we can do is we can use compilers to generate the code and automatically synthesize kernels on the fly. And this was a huge thing. And we said, hey, look, we can get performance. We can get things like autofusion where you merge two kernels together. And compilers are a lot of fun by the way. I encourage people to learn them. And so that was that was great fun. Now the problem with this is that uh where you can't scale kernels by writing thousands of kernels every time you bring up a new chip. Turns out AI moves really fast. Turns out compiler engineers are lovely. You know, I love many of them. Uh but there's very few of them. There's very few of them. And so you now what you ended up with is you ended up with a bunch of these technologies where it was cool because you could support, you know, some compiler for some chip, but suddenly you couldn't do flash attention or some new algorithm comes on the scene. It's a massive breakthrough in research, but the compiler people couldn't be in the loop to implement the new sparsity or the new float for format or the new this or that or the and so having compiler engineers in the loop was a huge problem and it held back for example Google TPUs. A lot of what happened particularly when Gen AI gen AI came on the scene really invalidated that technology approach. So I believed and so going into this and also part of the learning from working on TPUs is that we needed programmability. We need full power over the hardware. We didn't need the watered down simple solution that made it so that uh you got most of the performance. What I believe in is getting all of the performance. And so >> otherwise engineers would just like write kernels as they >> Exactly. Exactly. And there's so many technologies you see this across software where it's like okay it works up until a point and then you have to switch to a different system or a different stack and I don't like those things. Go back to Swift. I wanted to be able to scale from embedded all the way up to application level. Like actually unify this. don't make it so that you have uh like fragmentation in different systems. And so what we bet on was we bet on a two-level stack. We bet on Mojo >> a programming language, >> a brand new programming language. And as we all know, making a new programming language is doomed and like never works and you cannot get it adopted like all all those things. Well, >> you have a good track record though. >> Yeah, it's it's actually just a hard problem. But but you have to be smart about it and we can talk about that. And then um on top of that saying like let's take this programming language and design it so it can solve these problems and scale across hardware etc. But then take these compiler algorithms and put them into the language so that you as a mojo programmer get the power of compilers without having to be a compiler engineer. And so a lot of what makes Mojo really special is it takes power out of the compiler and gives it to you as a developer. Because it turns out a lot of people can write code, they can't necessarily write compilers. And so what this does is break breaks open the talent problem. And so now we can have a lot more people building these very highly adaptive, highly scalable, highly composable systems in ways that they couldn't do before. So can you tell me about both mojo but and also how this language can actually break the compiler problem? That that part, you know, the new language I think we can all understand, but how you can get through to compilers and get the performance out of it. How does the language do that or how does the stuff around it are underneath it do it >> well also again this is this is my life's work my journey and so I've made a lot of mistakes along the way and so a lot of it as we talked about before when you don't know what you're doing go learn from other people and go decide what they've done right and wrong and what you like uh for me a lot of it was learn from what I had done wrong and make sure to do it better the second time and so um one of the things that uh classical compiler engineers and myself back in 2005 or something which which is now 20 years ago somehow. Yeah. Uh was is that like compiler engineers want to show how awesome their compiler can be. Let me show you. I can implement this optimization. I can detect this pattern. In this case, I can make this code go you know two times faster or five times faster, 10 times faster. The challenge with that approach and this is actually this is really important for benchmarks because if you think about if you're a compiler engineer, you generally can't change the source code like you know if you're optimizing for the spec benchmark suite or something. Yeah. the source code's fixed and so all you can do is implement crazy stuff within the compiler matching and transformations and maybe change the memory layout of something using some heristic but there's a problem for that with as a user and so if you're a programmer and you're using one of these compilers and say uh you're working with a loop vectorzer for example loop vectorzation can make your code go four times faster by using SIMD to to take advantage of the hardware right but it requires a lot of pattern matching on your code and it has analysis process of how you use pointers and all kinds of other stuff that all has to line up for the optimization to work. So now you can have this wonderful moment where you pick up one of these magic compilers and then you get amazing performance and you're like, "Wow, this is cool. It goes four times faster. I love this thing. This is great." But then you go and fix a bug. If you fix a bug and you break the the hack in the compiler, well, suddenly you get a four times slowdown. >> And and you're not sure why. And and it's you cannot really debug it unless you're a compiler engineer. >> Exactly. And so now you file a bug and again there's not enough compilers in the compiler engineers in the world and so they may or may not look at it or they may tell you you're holding it wrong and too bad like it has to be this way. Right. But what that experience is is that's caused by compilers that are trying to be sufficiently smart. And the sufficiently smart compiler never works. What it ends up being is it ends up being an abstraction that's leaky. Leaky abstractions exist in lots of software and lots of different domains. But when they exist in a language, it means that you have an unpredictable programming model. So fast forward to today in AI. Today in AI, basically people want peak performance because GPUs are really expensive. As you said, if if you build a system that doesn't deliver performance, they'll switch to a different system because they have to they have to get that performance, right? Particularly for the really large scale frontier labs and things like this. >> So they need to squeeze everything out of it. >> They they need to it's required. the cost is just astronomical otherwise. Well, you can't have a system that is like unpredictable or flaky or weird. You want something that has control. And so what Mojo does, unlike Swift and many LVM languages, Rust, all these all these guys build on top of the same kind of technologies, we go for a much simpler compiler. It's way more modern. It's way fancier than what like Swift or Rust or languages like that are built on for a variety of reasons. Um, but it's way more predictable and so it puts control in your hands. And so instead of, for example, vectorization, instead of like hoping to magically do it for you and maybe it works out, maybe it doesn't, it gives you library features where you can say, "Hey, vectorize this for me." >> And I I saw one at Mojo, of course, there's a tutorial and and and getting started. I saw there's these special character/keywords where if you want to, I mean, you can just like write code that looks like Python. It's a subset of it and you're like whatever, but then you can kind of take control, right? So like as a when you're coding when you're like you decide you want a certain compiler feature that it supports can can you give examples of like what I can do with it? >> Yeah. So for example I take SIMD SIMD so vectors even I I don't think this is even a really standardized feature today in most programming languages but it's essential to get performance out of a CPU right most of the operations and floatingoint operations in a CPU are in SIMD because they allow parallelism. So Mojo makes that very na native. Uh Mojo also makes things like Bflat 16 and compressed floatingoint formats and stuff like that very native. And so it exposes that to you as a programmer. But then you have this problem of you want productivity like you don't want to have to write assembly code effectively or very low-level code to be able to do basic things. And so you want very powerful abstractions that you can build in the library. And so what Mojo does is it has very powerful metarogramming capabilities. And so um and by the way I didn't invent these. It roughly stole them from Zigg and made them better. So it's thank you to the Zigg people. They have amazing ideas. But the idea is to say let's pull this uh compile time metaroming system up make it much more powerful embed in the language. And so now you can build compile time transformations of your code. And so you can say hey I want a vectorzed function. Well what does it do? It takes a higher order function which is you know all the operations you want to vectorize and then it stamps it out across your vector lanes and it handles that transformation for you giving you very uh productive way of expressing these very high performance algorithms that give you extreme predictability. And so this this design point is very different. It's very important for accelerators because with accelerators you can't have the memory allocation that magically happens like you you need control. But these these features are a very different design point than what most languages are coming from. The other advantage as you say is it makes the language way simpler. And so if you take a language like C++ C++ has metro programming has templates and constexer and like this whole weird system that is >> it's a bit bit not the simplest to use. I mean you you learn it. So like >> you learn it and we I've I've written way too much C++ and so like you know even I can handle it. But but what C++ does is it has the program and then the meta program and they're effectively different languages and so this is really complicated and writing something using templates it's almost impossible to debug. You can't even pass a string to a template really like there are hacks but like it's just really difficult to do things. You certainly can't like make a binary tree or something and pass it into a template. That's not a thing. And so in Mojo and Zigg what they do is they unify the program and the meta program. And so you can just write code and then run that at compile time. And so now if you want to debug it, cool. Just use the same code at runtime and step through it in a debugger. Makes it super simple. It means you have one thing to learn and it works in both places. It means you can have real types and general types. You can do uh in Mojo it's very fancy because you can use like a list or a dictionary kind of data structure that does heap allocations and you can build it all at compile time and then plot the output of that into your program. And so it will synthesize the malix for you for a bunch of things you compile time evaluated and it all composes the right way. And so what this does is it gives you tremendous amount of power when you care about performance when you care about building these higher level algorithms which end up being like mini compilers. People generally don't think about it that way but they give you the ability to specialize based on you know dimensions and numbers and magic constants in your algorithms and allows you to build these very high powerful very high power abstractions very naturally. And if if if I want to get started with Mojo outside of just the basics which are so like using some of these more advanced features, is is it the best way to actually like rent a may that be a GPU or or or or like some some these days is is pretty easy and and go and try to implement an algorithm a program that actually needs high performance and and start to play and understand these additional you kind of the ways that I can control a compiler or or what would your suggestion be for someone who It's like, okay, Leon believes that this is the future and this is like cool to to experiment with on their job. They don't currently have a problem like this. How would you go about it on on the weekends, right? >> Yeah. So, um so the I can talk about what makes Mojo cool. >> Yeah, let's let's do that. >> Well, but see that ends up in the rabbit hole of really weird technical things only Chris understands. Wait, let's not talk about linear types. Let's not talk let's but instead, let me tell you about why you might care. Okay, so Mojo is a member of the Python family. So it's super familiar, easy to learn. And by the way, with the AI coding, it's easiest time ever to learn a programming language. Just like you mentioned, one of the simplest easy things to do with Mojo is to extend an existing Python module. So if you ever been coding up Python, building and building building, it gets big and big and big and big and suddenly gets slow. Well, historically your option is to go rewrite a big chunk of it in Rust or C++ and then you have to use bindings and it's real kind of a pain in the butt to be able to do this kind of work. Mojo is the most beautiful way to extend Python you can imagine. No bindings just like Swift and Objective C. They they natively spoke to each other. >> Oh yeah. No bindings because because Rust is is a popular choice to extend Python but with a binding. >> Yeah. But you need bindings, right? And so it's very mechanical and you can get it wrong. And so Mojo does that without the bindings, but also integrated with uh pit packages and the build systems all integrated and it's just super beautiful and awesome and you can stay on CPU. You don't need anything. It's all free like go nuts and go do this. We have very simple >> because I guess one one thing with Mojo that I don't think we you mentioned but uh I know that uh it's it's also not just for GPUs or TPUs but also CPUs. CPUs and today's CPUs are extremely fancy and complicated machines and getting performance out of them is also really interesting >> and so um so what I've seen people do and come back to the empowering stories uh that I love we had a a person come to one of our community meetings and they rocked up and it's like oh okay tell us what you're working on and they said well I don't know anything about this fancy you know AI stuff right I am merely a geneticist and I'm like like merely, right? He's like, I work with DNA sequencing and the stuff and I'm a Python programmer and and so I I saw that Mojo makes Python go fast. Awesome. And so I said, okay, well, maybe I would try using it from some of my DNA sequencing stuff. And so I I pulled over some code and wow it was like a hundred times faster because I pulled it over to Mojo and I could learn how to use threads and then I could vectorize that I could you I it was very cool because >> and then you can start to use the CPU's resources if you're running on a CPU which as you said are are have a lot of capabilities and you can play around with it if you learn how to control the >> Exactly. And so Mojo makes it really easy to do that where you are on a CPU. And then the thing I loved about it was he said and then I read that Mojo is really cool for GPUs and I'd never done that before. I don't know anything about GPUs, but I decided to give it a shot and in an afternoon I got my thing running on a GPU and now it's a million times faster and now I'm a GPU programmer, right? And so that to me is super empowering. And this is where when you look at technologies like CUDA and things like this, again, they're amazing systems and there's a lot of good energy and good work put into it, but it's 20 years old. It's C++. It wasn't designed for modern architectures. GPUs today have tensor cores and CUDA can't even acknowledge that that exists really and so this is a huge huge opportunity for us to get more people into the space and for people to learn about this for the first time. I mean I mean you know just as an engineer who is like I I don't program GPUs but but of course like everything I I write runs on CPUs or cloud or or wherever. This reminds you a little bit of a story of the what you previously shared of of the swift person because like I can't first speak tweaking for myself I can't really imagine myself like just saying all right I'm gonna like write CUDA I don't really have a use case but I can absolutely imagine myself just like adding a bit of mojo optimizing it and then maybe just deploying it on a GPU to see how it is if I see stuff that are paralyzed play around with it is it faster and then just like with the genesis >> build build and learn as you go >> yeah and then start to understand these other hardware which of course will increasingly probably be useful if if you know we know that and and also they're now available. You can rent them on on the clouds. You can soon maybe not today but I think today you can rent them by the hour. >> Y if not it's going to happen for sure. >> Yeah, you can totally do that. Well, and so we we even go further. It turns out that lots of things have GPUs today with Mojo you can run on AMD, Nvidia, and Apple GPUs. And so if you do want to learn about GPUs, you can go to our website, search for GPU puzzles, and we'll teach you how to do this. And actually, it's an amazing time. I love it >> because what we're doing and we're we're still early. We're still building into this and we'd love help. Please come join our open source community and and help us build into this. But what what we really truly believe is that GPUs and AI accelerators and like all these chips whether they're they're for AI or for DNA sequencing or chemistry or bioinformatics or oil and gas exploration like there's all HPC there's like all these applications that can use these these accelerators but all the software around them is really weird, right? And if we can if we can break that down, if we can make it easy to learn, if we can teach people, if we can make them consistent across the hardware vendors, because these hardware people don't always get along, by the way. Um, but if we can make it more uniform, then we can get a bigger community in that community. Sure, some of the people, some of the old dogs from the existing community will convert over and we love that. And you know, Mojo's way nicer than C++. And so it's way better from a user experience perspective and compile times. And there's so many things that are better about it than older technologies like CUDA. But also what I believe is that there's an entire new generation of people that should be programming these chips. And if we can get more people into this ecosystem, they can upskill. They can get better jobs. Like these are highpaying jobs to do this kind of work. And so it's the best time, it's a better time than ever to learn how to do this. And so it's, you know, part of my life's work here is to enable this and enable people to get into this ecosystem and break down the gatekeeping. and and really it it really feels you know you have been just always been passionate about compilers technology and and going towards here. Can you tell me where modular is as a as a team? You started about 3 years ago. Can you tell me like how much progress you made, how how large the team is right now and like where where Mojo and and the ecosystem is and you know where you're focusing next. >> Yeah. Yeah. So um Modell will be four years old in January. It's hard to believe. Turns out things do take four years sometimes and so >> even with the full focus. >> Yeah. And so we we are an unusual startup and we we went into this knowing this but we are a very large team and very expensive because we believe and our investors believe that the this problem that we're tackling is very important. And so at this point we're about just over 140 people something like that and we've enabled seven different architectures from three different vendors. So that's Ampear Hopper Blackwell from AMD. They're from from Nvidia, the 300, 325, 355 from AMD, plus all their consumer stuff, and now Apple GPUs in beta and coming online, which is very exciting. We're probably not done yet. We'll keep going, but can't share stuff before it's time to share. But it's still very early days. And so what we're doing is we're building into this from a technology perspective. Uh Mojo, for example, uh I want 1.0 to be meaningful. And so I think that we'll have 1.0 early summer next year. So that'll be pretty cool and it will be way better than Swift. >> Yeah, but I was about to say like this this 1.0 will be >> so so the team's scoping that out. We'll come up with uh detailed dates soon and we'll share that when it's the right time. And so as we're doing this, what we're doing is we're being really thoughtful and learning from the previous my previous journey and from other people on the team's journey is to make sure that we do this well. And so we want to provide stability for the ecosystem which swift one didn't do by the way uh and build into this. And each time we do pass through one of these milestones that you know we can expect more growth and more more use cases and more more investment from different kinds of people. That's what I've seen. Um so commercially we we have customers and we're scaling into that. We have a cloud platform which is the way we monetize most of this stuff. Mojo is not a product that we're trying to make tons of money on. By the way, it's a thing we had to build to be able to scale across lots of hardware. And so we see a dual world where we can actually catalyze and grow and teach people how to build and use all these accelerators and then we can help enterprises um build and manage AI into their ecosystem. And a lot of what these folks are looking for is they want something that's easy to use that actually works that's reliable that they can scale. Yeah. A lot of them want optionality on hardware. They want to be able to buy the best chip for their workload. They want to be able to get uh multissource in different vendors. They don't want to have to rewrite it three times. >> Yeah. I I mean this all only makes sense when you're especially that there's this dependency on like one or two vendors, usually one. >> Yep. And so and what I what I see and what I predict in 2026 is there's going to be a lot of silicon from a lot of different vendors and we want to make sure that people can scale across that effectively. And so, you know, for people that are interested, we have lots of job postings on our website and so we're growing very rapidly and we have a very big mission. It will take, you know, it's not another six months and then we're done. We're just still at the beginning and every year of our journey is just another epoch of new things to build and learn and grow and develop into this. >> And of course you're you're building like your sounds like the infrastructure under AI or hopefully a lot of it could be. How is your team the engine team specifically using AI AI tools? May may that be, you know, ids, agents, you mentioned that Mojo is a premier language to uh you know, like right with AI, but I I'm interested underneath the hood like are are you seeing the impact of of of these tools? Is it helping engineers work faster or down in the current because there's always this question like if you're writing compilers, will these things help at all or no, you need to handcraft it and I'd love to ask for someone who's actually seen it. >> Yeah. Well, so I can tell you I can't tell you all the things I can share my experiences. So we we definitely encourage our team to use AI coding tools and so use cloud code and cursor and all 57 other things and so that that there's lots of different experiences. I s I've seen tons of benefit and so for me I still code and so I use cursor for example as my daily driver right now and so I feel like it's you know a good 10% productivity for particularly mechanical rewrites and stuff like this. It's >> but you're a really good programmer so that's pretty good pretty meaningful. >> Yes I I represent very sophisticated programmer correct. Yes. And so like I but it's really I enjoy it because it frees me from a lot of the mechanical stuff and so whether it makes me aggregate more more productive or not increases my enjoyment which is good >> now important. >> It it is actually um now for other folks that are building prototypes or PMS that are building uh like wireframe models of things like it's transformative because you can literally 10x somebody and you can make it so they could build something they otherwise wouldn't get around to doing. And so that that is transformative for production coding. I think that it's kind of hit and miss. And to me, honestly, I don't know if it's like actually a net win or not because I've seen many cases where people say, "Okay, just go let the agent like try a thing." And then it grinds and grinds and grinds and a gajillion tokens later like it doesn't work. If they would have just done it, then it would have been done sooner, right? And so to me, like again, there's lots of wins, but there's also some losses. And so I don't know how things will net out. I do know it's moving very rapidly and I do want us to be using it. Um, but also one of the things I'd say is I don't want people to turn their brains off. And one of the things I think is really important is that programmers in our team but also in general, they use it as a human assist, not a human replacement. And I think it's very important that we review the code. We understand the architecture. And it's not I kind of for production workloads and production applications at least. Vibe coding terrifies me. Not not just because of what does it mean for jobs, but what does it mean six months from now when you want to change the architecture of something? >> Yeah. >> How are you gonna even understand how anything works, right? And so I do think that it's really important to keep humans in the loop and architects thinking about something and and understanding things. Um not not just for security and performance and all the other reasons as well. >> Well, Eddie Osmani, I talked with Eddie Osmani who's been on the Chrome uh DevTools team for like 13 years now. I think a lot of the the tooling he he or his team built there and he said he he wrote this book called Beyond VIP coding and he said that what he does is he uh enables like a verbose kind of output for the agent and he actually reads through like what the thing was he makes sure that before he commits something or reviews that he just understands >> what there so that he always stays in touch so you know if this thing turned off like he could he could jump in it's just like he had now just chose not to do so >> well and I think that again the thing that I that I care about is that the architecture of production I mean prototypes are a different deal but for production you keep the architecture uh clean and right and it doesn't need to be perfect but it needs to be curated because I've seen AI coding tools go crazy with like duplicating things in different places well we know that's a problem because then you want to go change something now you have bugs when you update two out of the three places right and so there's a lot of these things where the tools are amazing but they still need adult supervision >> so when it comes to hiring uh like you you have like you're building something really uh interesting and cool and honestly super ambitious. Uh you already have like a pretty big team but but you're growing. What is your hiring bar? And I think you previously mentioned that you're building a company of elite nerds. I'm interested like >> what are things that you look in in in folks and you know for for software engineering experienced engineers especially like listening what tactics or strategy would suggest that they get to the level of that they could have a shot at a place like like modular even if not necessarily modular. >> Yeah. So uh we hired two different kinds of folks. one is the super specialized, you know, compiler nerd or something like this or the GPU programmer expert that knows how to do high performance matrix multiplications and they've got 10 years of experience and so you can go look at that track record. >> Um, we also hire people fresh out of school. >> Wow, this is refreshing. >> And so I really do appreciate people that are very early career because they haven't learned all the bad things yet. And so for me, I think maybe the more interesting thing we talk about is not the super expert that's already the senior engineer. It's about folks that are much earlier in their career. And so what I love to see is I love to see people that are really hungry. They have not given up and just said AI will do everything for me, but they have intellectual curiosity. >> They're willing to work hard of course, right? But then um but then they're fearless because everything particularly in the AI space is changing so rapidly. But a lot of people will just freeze up and they won't do anything versus if you say, "Okay, cool. Let's go figure it out. I want to learn how to do this. I mean, this has been my life's journey is just >> that sounds terrifying. People tell me it's impossible and it's doomed to failure, but how hard can it be? Let's just go figure it out and and work through things and do that. And um I personally really love when people have contributed to open source projects. I think that's the simplest way to prove that not only can you write code, but you could actually work with a team, which is a huge part of software engineering in reality. And so, um I love internships and things like this that people have done before. and kind of uh when talking with people you can tell whether they're excited about what they do or whether they're just you know kind of performatively kind of going through the motions. Um, now one of the really challenging parts is that in interviews people get very nervous and so one of the things I really believe in is giving people kind of the native tools they're used to using. And so like let people code in this the way that they would be normally coding and um and so today for example we want people to be using AI coding for the mechanical pieces, right? So saying okay well you have to write in a whiteboard and you can't use your native tools would be very strange. And so we kind of try to make sure to meet people in something that actually approaches a real world situation. >> One question I I've been meaning to ask uh because you built several languages, this comes up and and I asked folks like what is what are things they'd love to hear from you? And this this this was repeated quite a bit. Do you think there is any logic in having a new language that is designed to make it easy for LMS to code with and like attributes for example better pattern matching or other things that might be the strength of of LMS and if so like what would these patterns be? This is really weird like I I would I would have not thought we would talk about things like this but but now we should. >> Absolutely. I'm happy to talk about it. And so I often sometimes get asked uh so given the AI is writing all the code, why are you building a programming language, >> right? >> Which is another way of asking the same question maybe a little bit more aggro. And so to me, I don't think that optimizing for the LLM is the right thing to do at all >> because go back to what we were just talking about. The important thing is reading the code. Always has been, by the way, right? Code is read more often than it's written. AI has made it way easier to crank out the code than it ever has been. And I don't think we're going back to where we were. So writing the code is actually not the the key thing to me. It's about reading it. And so what I care about with Mojo and with these new emerging systems we're building is is really a combination of two things. One is expressivity. And so can you express the full power of the hardware? Yes or no? Like if the answer is no, then you will never be able to get it. And so JavaScript will never be a good way to write a CUDA kernel or GPU kernel. it just will not because it can't express the things you need to express, right? No, no slide against it. It's a good system, right? But but if if that's the goal, then you have to be able to do that now. Assembly code can express the full power of the hardware and that's not what I'm advocating for, right? The the other thing that goes with it is you need readability, right? And so you need this combination, this intersection between expressivity, the so can you express the important thing in our case performance um and then can you understand the code? Can you build abstractions? Can you build in scalable systems? And that intersection I think is the key thing. And so this is where you take Python syntax for example. Python's super easy to read. It's actually very standard. A lot of people know it. Let's embrace that. Let's go all in on that. That's a good thing. And so Mojo embraces Python and this the syntax of Python at least and says, "Okay, well now let's fix the problems with Python, which is basically the entire implementation. Let's replace the Python interpreter and all that kind of stuff. let's keep this good part and make it so we can express that full power of the hardware. And with that, I think the LLMs will continue to get better and better at dealing with any weirdness. And what I've seen is that the LMS are an amazing way to learn coding a new language. >> Yeah, this is interesting because I I feel we're starting to see like including in like software, how how to build good software often times like what is good and efficient for teams is also LLMs tend to do better with them. So >> yeah, I also get these these funny questions. It's like okay well a lot of people in agentic coding loops want to uh have error messages that feed back into the agents so that the agents know what to do and so should we make better error messages for the agents >> make it better for humans >> and my question is like what what is better for an agent that's not al also better for a human let's just have good error messages right the other key thing and I think the most important thing for LLM based coding and AI tools is have a massive amount of open source and so for us if you go to the modular repo we have I don't know 700,000 lines of Mojo code That's open source >> and when you open source it you open source the whole history as well by the way. >> Exactly. Exactly. And so that's super powerful because now you can say hey go index this. >> Yeah. >> And so now it knows a gigantic swath of everything you can possibly do. And so that that I think is also really important. >> Yeah. Now to close with you have mentioned that compilers are cool multiple times here. >> I I'm slightly biased I admit. >> Absolutely. But I I'm getting I'm getting convinced that compilers are cool. For software engineers who have not built or worked with compilers. What are some kind of like like pointers you would give them of like, hey, I'd like to get into like understanding compilers, maybe building one. Should I try to come up with a language? Um like what are some things that you point people to? >> Yeah. So, so I'll tell you why why I fell in love with compilers. So, if I go back to university, I was taking the standard computer science program of uh basic programming and then data structures and then like an operating system class and guey class and all this kind of stuff. The thing I love about compilers is that you built project one. And so in this case, it was what's called the lexer, the thing that tokenizes the source code. Um, and you had to use some data structures. So I'm like, oh, okay, cool. I'm using the things I learned, not just how to write code, but what a what a tree is and things like this. And then you get to the second part and you build on top of it. And so now you build a parser, the thing that decides the syntax of the language using the tokenizer. And then you build the type checker. And you build on the first two. And then you build the next thing and you build on those things. And so what I loved about compiler as a class was that I got to build and learn and iterate and then if I made a mistake I had to go back and fix it cuz you're building higher and higher and higher. And so I think that compilers particularly in the university setting reflects more of like real software development in a way that a lot of other classes I I experienced did not because a lot of the other classes ended up being build build a thing turn it in throw it away build another thing turn it in throw it away and so that's that's why I fell in love with it. >> Uh today if you want to learn compilers it's easier than ever you can go to LVM. There's a great tutorial called the kaleidoscope tutorial that I wrote wonderful >> 15 years ago or something. It's been a long time ago. The Rust community has a lot of compiler lovely compiler nerds in it and so a lot of compiler based technologies built in Rust these days which is also super cool. Um and so there's uh there's books and lessons and things like this. I don't literally think everybody needs to become a compiler engineer but I think that compilers don't get the credit they deserve and it turns out there's a lot of really good jobs in compilers and so if it happens to be that it's a interesting place and interesting set of technologies then I really do encourage people to do it because there we need more more folks in this field. Wonderful. Chris, this has been really, really interesting. >> Yeah, >> learned a lot. Thank you. >> Yeah. Well, thank you for having me. I hope it was useful. >> It was. It's rare to talk with a software engineer who is so passionate about programming as Chris is. And I hope you enjoyed this conversation as much as I did. I especially liked how Chris took us to the backstage of how things really happened at Apple. How was Chris's own stubbornness and track record of getting things done that allowed him to push things through like open sourcing the Swift programming language and creating the new language to start with? For more deep dives on AI engineering and stories and developer tools at other companies, check out deep dives into the pragmatic engineer that I wrote linked in the show notes below. If you enjoyed this podcast, please do subscribe on your favorite podcast platform and on YouTube. A special thank you if you also leave a rating on the show. Thanks and see you in the next