Developer Experience at Uber with Gautam Korlam

Vibe coding this is a brand new term what do you think it stands for it Vibe coding is just you trying to figure out how the system should behave when you're prototyping the good thing about VIP coding is you're able to like basically iterate much faster with the agentic loops but now you're just focusing on how would I actually achieve My outcome rather than the exact way to do it cuz you can always change those details later if you have the right abstractions you could swap layers out so I think that's changing the way people think about software because if you're someone who has taste and knows this is the outcome I want to see on the ux you may not even need to know a framework you should experiment with it till it feels right and of course like you need to massage it to make sure it doesn't make it hard or do all later on I think a lot of the software today gets you to the initial it looks great but then how do you maintain it is basically sort of like left for later time and I think a lot of people who are still coding with AI are early in the Prototype phase gotam Coraline worked at Uber for almost 10 years joining as Android engine number eight and was the founding engineer for the mobile platform team he grew from software engineer 2 level all the way to principal engineer and worked on developer experience and scalability challenges throughout his decade at Uber gam is now the co-founder of guitar and a gentic AI starter that automates code maintenance in today's conversation we cover how gotam accidentally deleted Uber's job on monor repo which had thousands of developers committing to it Uber's in-house engineering tools like monor repo submit Q local developer analytics and death pods ai's impact on software development why Vibe coding will spread and why gam believes Junior devs will Thrive with AI tools Uber's in culture is very unique even across tech companies and this episode is a perfect inside look of its early days if you enjoyed the show please subscribe to the podcast on any podcast platform and on YouTube Welcome to the podcast it's great to have you oh thanks so much G glad to have glad to be here thanks for having me it's good to reconnect because because we we talked a lot while you were at Uber but you know you were on the platform team and your work touched a lot of the systems that we worked on every engineer causes outage at Uber I I I did it people on my team did what was a memorable outage that you might have caused accidentally or deliberately so this is a funny story uh towards the towards the end of my career there I accidentally deleted the Java monor reper tuber there were thousands of deaths committing this yeah the the Java monor that that was one of the we had two two big uh back and monor rles right go and Java go and Java yeah so this was after we did the migration on like almost everyone using the Java monor repo yeah it was pretty funny cuz I the what the story is essentially I don't think many people know about this because I record pretty quickly but I was trying to test something out like a test repo and I was like hey I need to copy like the git uh push command cuz we created the repos on push so I copied the URL from the Javon repo I forgot to change it and then when I pushed usually at Uber they prevent you from Force pushing right so and it said hey I can't force push and I didn't really read the message then I was like why that's this I should be able to force push and I did with some special flags and there's only few people maybe like three people at Uber who can force push with that flag because I was part of the platform team and we didn't want anyone else to do it and I pushed and then someone PS me on slack hey so the repos say initial commit what's going on so so you you were a principal engineer at this point right like you were one of the most senior people and you were there for a long time right this is like not some you know you didn't know what was going on so I mean that's why I had the permission to to push so whole repo gone for until you verted minutes later uh it was tricky because we had to get it back from a backup yeah it was not too bad it was like a few minutes and I think we had an instant interview it was luckily at a time when we had a submit queue so the queue would PA automat Al and said yeah hey no work is lost right uh we had really good backups in place so we rewarded quickly and I was like ah got to be very careful with those like copy paste commands wow well and but I guess you tested the the backup actually worked you should have said that it it was just a test I wanted just to see if if the process actually worked yeah chaos engineering this episode was brought to you by Sentry buggy lines of code and long API calls are impossible to de debug and random app crashes are things no software engineer is a fan of this is why over 4 million developers use Sentry to fix errors and crashes and solve hidden or tricky performance issues centrica debugging time in half no more Soul crushing lock sifting or vague user reports like it broke fix it get the context you need to know what happened when it happened and the impact down to the device browser and even a replay of what the user did before the error centry will alert the right on your team with the exact broken line of code so they can push a fix fast or let autofix handle the repetitive fixes so your team can focus on the real problems Sentry help monday.com reduce their Errors By 60% and sped up time to resolution for NEX door by 45 minutes per Dev per issue get your whole team on sentury and seconds by heading to sentry.io pragmatic that is scent y./ pragmatic or use the code pragmatic on sign up for 3 months on the team plan and 50,000 errors per month for free you reverted thankfully not not you know like nothing crazy happened but this is a pretty big deal how you know like how can we imagine like what was the response of like your manager the in interview I mean yes sure you were an experienced engineer but come on this was a big deal right so the good thing was since we had the summit queue and we can talk about it a little bit more only thing it affected was the amount of time it would take for a commit to merge onto Main so usually everything everything would get serialized so all the did looked like was there was a little 20 minute delay while we recovered everything and then everything just flowed naturally so it was just like a random CI failure you would see but we had so much recovery automated in place that it was almost a non event and the only SLO we broke was the latency SLO no reliability slos were broken so that was pretty good like all the systems we put into place in the many years actually worked to save me that day okay and this this is a good way to to say into the next one Uber's unique engineering stack I I I worked at Uber I work for a lot shorter time than you but but we overlap you know you were there when I started and you were still there uh when I left Uber had a lot of Unique Systems can can you talk about some of them and how they came about and why why did Uber build so much of its stack well Uber and also you you built a lot of that stack right yeah absolutely so when I joined Uber in 2014 I was like Android engineer number eight I believe if I don't correctly and there was no unit test so I wrote the first unit test for the Android code base then I set up the artifactory yeah it was it was pretty funny we were shipping so fast that people were just you know integration testing using their mobile phones cuz it was just very very fast growth this was in 2014 2015 2014 is when I joined yeah so then I brought in artifactory so we could share stuff and over time what ended up happening was a lot of the uh stuff you take for granted today the cloud n SAS products for observability or uh hosting source code was just not built for our scale back in the day so we had to build a lot of stuff in house to solve some very crazy problems so we had commits going in almost one every minute I think and that might not seem like a lot but when you have thousands of Engineers working on code basis where you can easily go into merge conflicts that was a big deal I hear a lot of companies saying it or at the time it was popular to say it's not built for our scale what was that that scale I mean you mentioned the one one commit per per minute which I can actually attest to that that build systems could not handle at least Mobile bill systems could not handle that but but what other scale was considered just like way bigger than what commercial vendors supported yeah absolutely so I think the bill system part uh was definitely one because bills are taking just way too long I think we embarked on like a big appy right I think when you were there I think you talked about it before was funny I was actually in Amsterdam and that happened in between and then my boss calls me up hey this build time is like getting crazy we need to somehow bring it under control oh that was you yeah I was there and it was almost an hour something plus so if you actually seiz all the commits one by one to make sure there was no merge conflicts it would just take forever and if you did not do that our main bch would be red we had this graph where it was like red green red green red green and it was impossible to work cuz there was so so many teams trying to hit their deadlines and you were depending on so much work from the other teams like the networking library or the platform uh experimentation or a future library from some other team it was just not possible to do this without green ve essentially right so it's not just the pill system time but also the the fact that when you push something if you don't run tests against the other part of the code then you're just not going to have a good time because you will just be stuck reverting stuff someone would be on call all the time so it was a combination of factors that made it very hard to work with yeah and and also like we we did do that on that project specifically we turned off endtoend test I think for two or three days to speed things up and it speed sped things up and then we had so many regressions and it took more time to I think you know for two or three days we turned it off and for the next week everyone's fixing the regressions that kind of sneaked in underneath yeah I think we got a request at some point can you just turn off CI and as like you know what's going to happen if that if he actually did that and had to really talk people down saying you cannot just turn things off it's going to make things worse so there was the submit Q uh can can we talk about what submit Q was and why it was unique at least at least initially to Uber even even to this date I I'm pretty sure only a few companies have anything like that yeah I think submit Q is essentially a way for you to guarantee a green main so it's a way to kind of serialize your commits coming in and make sure that they play nicely with each other uh there are some more open Solutions now but when we did it at the time we had actually a paper that we published uh it was fairly normel because we had very low tolerance for main being uh red so we'd always want to guarantee a green main which means we had to figure out how do we actually test changes coming in and make sure that the cross dependencies between different commits were considered so it's just a way to kind of handle commits at scale and make sure that when everything merges everything still builds with each other right and that's a trickier problem than it sounds cuz we had to test things in combination discard some bats and do a lot of processing on the back but there was just one of the systems I think we had to build a ton more which I can talk about as well I was the consumer on this side you know I was just using it it was just a build system for me but I didn't realize for quite a while that you know like my team you know the 10 of us who are working on it we didn't really like have merge conflicts with each other CU we were working in different parts of the code but it took me a while to appreciate that when I was pushing something like at any given time there would be maybe 10 20 30 people on the same code base in Mobile working and each build would take 20 30 40 minutes and then behind the scenes I actually I understood only when I read the paper that you were you know looking checking like is this code path in Conflict if so which one should we build beforehand there were like all sorts of probability models there's a bunch of math in that paper as well right there's a lot of ml models too like I think we had to like estimate what would potentially cause a failure and speculatively try to go on paths that might be green to try to optimize and then backtrack if we didn't uh make that goal and we would make that model better over time so it's quite interesting I think if you didn't notice that's a good thing it means that it worked as it's supposed to do one thing that Uber did as an engineering practice was monor Repose uh can you tell us how it start Uber had multiple monor repos but it all started with the iOS monor repo and and I I I recall you were there can can you tell us like how the decision was made to go from having a bunch of separate repos initially on iOS later on Android and say like all right let's just do one big repository and you know why did this happen and what happened next so it's funny I mentioned I think early on we didn't have test at that point very very early on we had two apps Rider and driver on both IOS and Android right yep and we had another repository called Library which is the common components between the rider and Driver app which was just a subm module you would pull in and it was really painful to pull in as a sub module so we said hey let's set up some sort of cocoa Parts artifactory thing where you could download stuff from somewhere uh but as we grew we realized we couldn't put everything in one place we had a networking team uh experimentation team a analytics Focus team they all wanted their own sort of nice little playground so they they could experiment so they we went off and had hundreds of repos for a while where everyone was just you know unaffected by everyone else could focus on their priorities and deliver but then when you wanted to pull in these changes let's imagine the networking Library changed you had to update netking then the analytics library that relied on it then maybe the experimentation maybe some other future Library it was really painful to upgrade and pump and we were sort of on the hook because anything that the product teams don't do the platform team ends up being the kind of like catch all and you're like hey this is not going to scale I think someone on the iOS team was like uh hey we should just go to a mon rep just take a weekend famous last words I know but they took a while didn't take a weekend but the iOS team did it and it was very effective for them uh it was a lot of pain initially and I think happened the same time they were moving to a new language Swift as well so that was like double whammy they had to do both um they pulled it off and the gains and productivity were massive because you don't have to go bump libraries because we were not an open source you know make sure the interfaces are nice for external consumption it's one company one code base why can't I update my lities public API and just update all consumers in one shot it should be possible so mobile did it and then Android quickly followed suit we actually froze the code for a weekend and we did everyone over we moved everyone over and it was painful at the start because the build systems as I mentioned were not meant for hundreds of modules in a reposter we have seen some changes to Google to make it better and then after a while we migrated to a new build system Buck from Facebook at the time and we wrote some Auto conversion from the existing system to the new system so we had both the same time that was really successful because once things were fast The NPS improved again mon repo is good so the gist is mono non mon if you don't have tooling it's going to suck you at bigger scale you have to invest in tooling as your companies grow typically this need becomes more and more obvious as things start slowing down right and then later we had the J on repo and we migrated over to basil and we had the go monor repo and I think what we realized with the monor repo was that the main thing that helped us do was it was standardized which meant to make a big change one team could go to it like a centralized team could just take the B burden off updating this one networking library for everyone or updating Google Play services for everyone you just cannot do that in like hundreds of reposes it's just very bful amount of work right and what was the biggest push back that the teams initially had because I remember the the mobile ones that they actually happened and it was you know like very practical but then when the Java and the go was happening I remember a lot of teams were dragging their feet like I'm not sure we want to do this it's going to you know slow things down do you remember what the biggest you know like skepticism was and then how it turned out yeah the biggest skepticism is usually right now before mon repo I could break an API and not worry about it the tax is paid by someone eventually right this is deferred you still gets needs to get paid but the mon if I do it I have to think about my interface design more and my bills are going to be slower was the argument with the new bill system we kind of said hey the bills are not going to be slower but people would always come up with reasons saying hey if I did this upgrade then I had to do it for everyone so we said okay you know what we will make it possible for you to just upgrade your part of the cod in some scenarios but otherwise we want to have everyone be standard it does make things a little slower but then as a business it makes a ton of sense because you might not see the macro picture on your team that yeah you might be sling slow a little bit but everyone else gets like a huge productivity boost CU they didn't have to spend the time you did on each and every one of those teams to like get up to speed is it safe to say that monor repos often not always they they they bring kind of a almost necessary friction where like you know like making certain changes will be painful because they should be painful like because when they were not painful they were masking something down the road of that's say you know I update a dependency to to to something else and then it has to trickle down otherwise you'll have these problems but they'll only come out in weeks or months later yeah to be fair like you can also do stuff without the monor repo like I know companies like Amazon Netflix do M multi repo setups but they buil tooling especially to handle those cases where you have golden version sets that work with each other it's a little bit like a monor repo without being a monor repo right so you can make both work but monor repo is just generally easier in my opinion just having considered both but if you want to go the other route you absolutely can you still have to invest either way as your code base grow and what what were some other kind of Novel inovative things that I mean sounds like you had to invent some of these things at Uber because there was just not a solution that you could buy or or or it was just really expensive or not you know for not built for this use case absolutely so the other thing I think is interesting for us is as I mentioned build Time s a problem we only knew because we first had to measure it I think back in the day a lot of the measurement products for developer observability but just not built for the entire stlc so what I mean by that is like you might have like a CI Jenkins back in the day that would tell you how long your bills would take but you had no idea how much your indexing time would be in your IDE and we had very complex projects or you had no idea what was the time between pushing and code review so we built a system at Uber called local developer analytics we call it LDA for short which was like a little demon that used to run on your machine collect information about your system like what's your C CPU usage memory usage but also like integrate deeply into a lot of the CLI tools the IDE so we would for example know when you opened your project which file you went to where which part of the file you were actually like coding the most which files would have the most bugs and then you could also see like a funnel like in a traditional product funnel because we would care we care about like developers as like a end user you would see like analytics saying hey there was a drop in the funnel people could not create PRS because this part of the build process would error out more often than the others so you would go and like know what to Target and it was fairly norvel because I gave it a I gave it a few talks at a few different conferences and not even big companies had something like this back in the day and I think we still use that today to power a lot of the dashboards and the Deep analytics that we get from uh from the developers workflows yeah like what do most like you you've been a developer productivity you know you talk with a lot of people and know a lot of people but what do most companies do do they just like ask developers like how does it feel or I think that's a very important thing to do for sure because even if you have all the measurement if the development doesn't feel right to you you still want to ask that like you want to have a survey which we did in fact when I I remember when we started doing the surveys I think we had like an NPS score negative 50 something and when I left it was like positive 8 or something like that so I I call that as a win that we did a lot of effort so you do your survey as the very first step which we did but then you measure the easier things like your comit you time to like review code or time to like build code stuff that's out of the developers control you want to like minimize and also time spent in meetings like those are things that you could probably get easily but the deeper stuff is only worth it if you see that that those bottlenecks are adding up like if you're making changes but you're not seeing like people are happier because you'll always see developers complain build times are slow or ID indexing is slow that's when you know okay we need to measure things but now I think products like Vore jet bins have better analytics I think we worked with ch brain for example in trying to make them understand hey these are some of our problems it would be great to have Solutions out of the box for these and some of those did make it Upstream into products that everyone uses today so we call that as like we kind of had to forge our own paths and eventually all the vendors would catch up based on what the industry was doing yeah but it's interesting how back then there was nothing and you kind of like pinpointed I I guess one advantage that Uber always had is a we had a lot developers and there was actually a dedicated you know what was your team's name was was it developer platform we were called developer experience they had multiple names I think we were mobile infra first then mobile developer platform then developer platform that was the final name we had but funnily actually the team was not that big we had my team was about 10 people and we supported about th000 engineers and that's roughly the the breakdown so you would see that speaking to peers in the industry the team sies are about the same same you have like a very small code centralized team and then you have much bigger product organization that they support so it's very common and that means that you have to find ways to scale yourself uh and be effective and then there was another project that you I remember that was just taking off just in my last year it was called def pods what what was that so I think a lot of uh people might have heard the term Cloud developer environments so developer environments in the cloud uh so we call these D parts and what it is essentially is a container of your code your build system artifacts your ID indices in the cloud and what we did at Uber was fairly unique in the sense that we figured out some ways to make it boot really really fast and have pretty much everything pre-index and it is one of the most L products there because you could essentially Multiplex yourselves now rather than trying to switch branches reindex code or context which you could have 10 different D Parts working on different features and they could have everything warm you could Conta switch quickly you could kick off a build and go do something else it was actually great I think we put a lot of engineering effort to make that scale really well and it was a fairly small team as well actually a bunch of those folks are still in Amsterdam now working on the product and can you just explain because I think most companies just do not have anything like that like as a developer what can I imagine like okay I I I joined Uber that ODS is there what does that mean for me what can so when you onboard previously we used to have this like bootstrapping script you would run it's funny cuz do this now we have a bootstrap shell script that everyone just keeps like updating there was a shell script and then which usually worked well but sometimes it crashed and then we would have to you know ping the on call uh of your team but it was a pretty involved script like it it would take sometimes minutes to run out andot lot lot of magic under the hood and the problem with that is like it's hard to keep it forward and Backward Compatible with your system updates on Mac so it was definitely like not worth maintaining over time but uh the D Parts is interesting because I think just when we started seeing some of the solutions like vs code remote or CH prin projector become a little bit more mainstream we said hey why don't we move this development stuff into a container it should be easy right but it's actually much more than that it's these containers can be huge multiple gigabytes right because these artifacts and indices are huge uh we had to make sure we also use compute efficiently so we would have to have multi-tenant because developer workloads are very spiky they don't always have like a stable pattern like a server workload which means that you have to like understand when to scale up and down we have developers in different time zones so we have to also like provision machines closer to them so the build caches would be warm uh closer to where they are working otherwise you would download upload everything from a data center in the US but you might be working in India and the latency would be really bad it's so it's a lot more nuanced I think than just spinning up a container a lot of the solutions out there now are just focusing on content Rising stuff but not thinking through a lot of these other factors that are important so we went through and made a lot of improvements here some of that stuff actually one of the funny things I remember is uh I think jet bin had this like one shared index thing that they had a while back where you could download a index of the project but I think the way they would doing it previously you would have the full path of the system in the index so you would download a denormalized path and then you would normalize it to your system just populating all the absolute paths would take a really long time we measured it was like take take an hour to actually like index that Shar index for chat pains but then on torts what we did we just said hey everything's in our control let's just put everyone at home user so it'll be like closing the laptop and opening it again so we could just the cash there everyone was at home user so now no more like time spent like denormalizing the cash which meant we had like a sixc boot up is pretty insane for like a Dev environment so you would come in you would say Dev part start something and you're done you don't need to do any boot strapping not learn something and you would have one flavor per mon repo you would work on so a data engineer or like a backend engineer or a mobile engineer would have their own flavor and that's very optimized for that work so it's basically go on and say like the pot start uh and and then in a few seconds you have your web sorry your your local editor uh you're using your local editor but it's kind of like connected magically from your perspective so you feel like you're working locally but it's just all set up it works you can build immediately you can run your tests Etc yeah you would have like lots of course as well so you could run a lot faster than running it locally that's pretty awesome uh I previously did a coverage on did a deep dive on cloud-based development environments I think two years ago what one thing noticing is there's not much talk about these things even though they're just so darn efficient uh do do you see uh kind of excitement in this space or or is just AI right now you know blurring everything out because this this to me this this was a really big win yeah I think the the environments are still good I think a lot of companies do want to use this for efficiency sake the challenge usually comes with I think the the understanding that I have talking to PE and industries if you try to start with the solution rather than the developer workflow hey we have a container environment make it work for your developer workflow versus you have a particular way to work let's make it work in a container or VM so we took the other approach and that is why we were so effective like hey these caches had to be like super fast load other stuff can come in they sync right and not everyone would have that Insight if left to their own devices teams might device a doer file that has stuff but then how are you getting it up to date as your team CHS out more codee are your indices actually getting up to date otherwise you're spending a bunch of time after bootstrapping to reindex stuff or redownload stuff so we would push these updates every few hours completely transparently in the background so it was very much like a golden path that we had to like kind of happen up on and that's why I think if you have that knowledge of the entire developer workflow you can adapt it to each company's individual working style if you give a general purpose thing it'll work but may not actually work so well out of the box and you may not have the investment you may want to make to make it work for you yeah sounds like no you know free lung you need to put in the work and you know Uber had like so many years and of of like understanding investing like being there now speaking of years you spent nine years at Uber and when you joined you were an N2 is I right or I an n two and I joined one I guess yeah know what is the lowest level whatever the lowest level was yeah and then you when when you left you were principal engineer which was which is there's not many principal Engineers like like several like a few dozen or something like that so and you were promoted actually yeah maybe a couple dozen yeah not that many out of the closer to 2 3,000 uh people working in Tech and so you were promoted I think I counted four or five times in in nine years can we talk about your journey like how hey how did you do it and what did you learn on the way yeah absolutely I think one of the cheat codes is obviously joining early helps a lot especially for getting on a rocket ship uh but it's hard right cuz when I joined we didn't have as many people the platform teams were tiny it was so small it was funny because I jumped straight from entry level to senior so I skipped a level because the first year was just put through the pis of we need to get a lot of stuff done and I was shipping really fast so that helped obviously one jump up but it gets harder over time I think your roles also change a lot over time I think uh one of my good friends early on in the platform team um said hey if you can get into a niche and go deep it can really help over long term especially stuff that people may not want to do uh like developer platform developer tooling not Everyone likes to do it but if I said hey I really like this because I'm passionate about it that helps because then you'll become the go-to person for that aspect so as a company grew it was very obvious that hey I had to scale myself to larger and larger efforts so starting on mobile then I started working on P in stuff and then on more holistic entire organization stuff and that takes like kind of you know breaking your own comfort zone a lot like challenging yourself every year or two so every two years I kind of do some introspection and say hey am I doing what I want do I enjoy what I'm doing if I what can challenge me more and then I try to go for that and usually the path opens up if you go that way rather than trying to chase like a particular level because then when you're pushing the boundaries the next level kind of becomes obvious and and past the particular point it just uh also helps to have a lot of connection like Social Capital mentorship those are very important at like a big company right so I definitely went and got some mentorship with some more senior folks that helped a lot to understand how these principal Engineers operate and what's required is more than just the engineering it's also the business side of things how do we actually uh solve problems and focus on like the business metrics and bring that to engineering rather than start from bottom up and and when you say you know like you need to get some social capital and and mentoring like it can sound a little bit of hand wavy to people of like but how how did you how did you get that I'm not going to ask how you went about it but because it sounds like you didn't say like oh I want to get Social Capital but what were things that you did that you think actually helped build up and people saying oh you know like like godam I really trust this person I know him he's he's a go-to person Etc yeah so one thing that helps is I had this habit of just helping people a lot so when people come to me with like a problem with their environment I would just drop everything and say hey let's start to figure it out I'll spend a little bit of time time box it and if it fixes the problem then I have some social capital it kind of accumulates over time because they would send someone else to you and over time you can automate some of this stuff and I think I was one of the early people on the platform team to have office servers so I would say hey anyone can come in and just you know talk about your problems and we would you know just empathize a lot with everyone say hey I know this sucks you're working on it it helps a lot to humanize the problem right cuz developers just usually ping you on slack um and over time what happens is like as you solve more and more problems it's always good to like attempt and if you can't meet the expectation just say hey I can solve this right now then try to say okay that's not in my area right let me just try to help you out and that really helps build trust with the other other side and then when you go for mentorship it's like hey I clearly know you have the right sort of like uh mindset you have the right goals in mind or like the right intentions so I would love to help you kind of go to the next level so that's usually what helps get very strong mentorships yeah and I can just plus one a little bit on the helping people because you know I was in Amsterdam which is a we say it's a distributed site but we were just not in HQ we were we're relatively small office you know like compared to SF or even po also in headcount and we felt a little bit isolated and sometimes like a lot of times we would feel blocked by SF uh in in reviews and in anything and after a while I knew that if I wanted to you know you were one of the many people on the platform team but I think e either someone told me like there was this thing like if you if you cannot get if you cannot ret anyone on the platform team try got him like he usually responds and just thinking back it was just wild to me that like I did get responses from you know some sometimes and uh like unexpect for like unexpected like I I I didn't think I would get anything I was just trying but you did like spend time on this end and I just imagine that you had a lot of these things and I'm not saying you necessar did it all the time and obviously people like me were doing it sparingly but it it built so much Goodwill uh and also Trust of like wow like I just felt that whenever you did go on a problem you actually took it seriously uh like you you kind of did it and as you said often you would tell like we I cannot do this right now or it's not it's an non issue Etc so I didn't see every everyone in fact I I saw pretty few people doing it so like it's it's interesting how it feels like something that would not scale but I guess to some extent it it it did is is this like the stter mentality of like you know do things that don't scale yeah I think it definitely doesn't scale in all cases um but you can make it work I think it's funny because some people won't even try because it won't scale my suggestion is like hey let's meet the person and see it may not even be an issue right sometimes what you might discover is people are just finding it hard to find the right documentation because they just looked at the wrong guide and that's where they're coming to you then you can just go make that fix and then you have much larger impact but if you don't talk to people you have these broken windows like you have the big picture but then you lose touch of what's actually happening on the ground it's hard so I think my policy was always like hey I can always talk to someone spend a few minutes and if it's like a very esoteric thing I'll say hey I can get back to you but if it is not I would rather just point you the right way people joke when I left because they're like hey we should go build go llm because he would know the answers to like you know pointed things at least so that we could find the right information so I was sort of like an encyclopedia of where to find stuff and you were a principal engineer uh in towards you know the end of your tenure at Uber what is a principal engineer at at Uber and and also previously we had an episode at meta uh meta has archetypes above staff engineer do you think you fit into an archetype or or did you see kind of archetypes at the principal level at Uber yeah I I did listen to that episode that was very it's a very good episode by the way uh I think there is some resemblance to an archetype although we don't actually call it that so either you have depth in a particular area or you have a lot of breath and sometimes it's a lot of internal influence or you might have a lot of external influence right so it really depends on where you want to take your career uh I focus a lot on depth like as you can see my specialization is like d productivity and I have some amount of breath because of all the social capital and talking to a lot of people usually makes you understand what are some of the business problems that Uber is going through so that was actually a good bonus for me because if I never talk to people I would't even know oh these three teams are trying to solve the same problem maybe we can centralize it or do it more efficiently and I could come up with Solutions and people be like hey how did you know like that would work cuz you just talk to people right um and there were definitely like lesser archetypes probably than meta I would say but usually it's a bunch of breadth and depth and having people to sponsor you helps a lot so the mentorship angle is also part of that if you're working with more senior engineers and they see you actually grow they can give you feedback where you might be lacking and you might be able to kind of work towards that goal so I think that's one of the things I would recommend anyone who wants to go in that path with principal engineering it's a lot more you have to enjoy like understanding how engineering meets business than just pure coding or just pure uh what do you call it not soft skill stuff a lot lot more soft skills need to be horned at that point it's like going and giving talks helps and it's funny because when I was growing up I was not very good at like you know public speaking but I made myself over time more comfortable that helps also like build external influence and a lot of the early folks that are using our product right now are people I met in the community and then they kind of have that trust that they build over time so that so this is a lot of people who are using your product right now at at guitar right yeah yeah so I I I guess that that's a it's a a good reminder of like how you know it's I guess it's when you're doing something it it might hopefully it'll help you outside of when when you're at your current company where when or and and also help you while you're at the company yeah my tip is generally if you really focus on the people side of things you'll do really well in these in these higher levels because it's a lot of relationship management both managing up and down and shielding your team from some of the noise uh and if you don't enjoy talking to people it's harder I you can still do it there are people who do it but it's a very unique path and not everyone can do it but the most common path is usually like understanding and spending time like talking to people and figuring out solutions that are more creative over time and that also kind of makes you a problem solver not just like some who can just you know code and finish up things up but that's of course like you can have different archetypes as you mentioned so it really depends on where you want to take things yeah but I guess the principal entury level is kind of given that you're just you can code efficiently whenever and however it's just like you might not do it all the time by choice right it's funny because there was a graph someone sent me uh of like number of years of the company and number of comments per year and number of reviews and there was like a little craft like you know like an ASM totic graph and there were two or three dots at the very extremes and then said hey one of them is me the other one is you then I was like damn I should ask my boss for a raise so so you were still writing a lot of code and doing a lot of reviews I do write a lot of code even here at Guitar I write a ton of code I I love to review stuff because it really opens up your perspective of what's else is happening right I like to comment on documents and understand where there are inefficiencies potentially but it just over time I think you have a muscle to just do it but does not mean that you write tons of cod it just means that you have consistent highing backt so a really interesting topic is given that you were a principal engineer which was either the highest level or or maybe there was one level above it but there weren't many levels above that how was it like for your manager to manage you and I know you also talk with a lot of principal engineers and you have some thoughts on what are some tips to manage these very very senior Engineers like you know can can you help explain your relationship with your manager was it really a kind of a boss or more of a partnership on and you know what worked at this level I think that's a really good question because I think uh as you grow in levels you're more and more sort of like a peer to your manager you help your manager get stuff done too because you have broader influence sometimes uh it can be hard as an em to you know like push for a a particular priority because there's not enough senior people that can back right it might be just your directoral person that's sponsoring that but having someone on the team that's seen it enough can completely change thoughts for example to get some things practiced um one tip I will give managers is like if someone's asking you hey I'm at a senior staff or I want to become a principal after like the senior levels if they come ask you it's hard you should probably not ask that question to a manager because at some point you you'll have to figure things out right uh there's very few managers who probably can help you after a particular point because they're maybe uh done it before or have a lot of experience at the company but most managers tend to uh work at like senior or senior plus level and past that it's more like they're almost like the principal engineer on the team is load balancing a lot of stuff uh technically so the magic can focus on up Ling everyone else so the relationship is very much like a peer like more like hey Mr a boss but it does help to get feedback I think their managers are a very good asset when you want to do not like non-engineering stuff like for example you want to get uh stuff unblocked and the engineering angle is not working because you try to rationale with some team they won't listen sometimes you just got to take the uh hey this is the priority we need to get this done like to unblock yourself so that's what I would say for EMS who might be looking at uh higher level Engineers basically give them uh agency but of course like check in often and make sure that they're unblocked right because they can do wonderful things as long as your Aug is not getting in the way from them trying to actually have impact yeah this this is always interesting because I I don't think we really talk about this too much just because there's not many of these Engineers but but even when I was a manager I actually promoted someone actually two Engineers a level up of where where I was right I mean it was a bit easier for me because I saw examples but it it it's always interesting and I feel it kind of goes hand in hand like like for these things those promotions really help usually the manager as well and as you said for a lot of managers it's it's a first and there's no there's no Playbook after after senior or or staff it's it's all unique it's all based on the business needs for for for the company in that specific area there yeah but if you to keep up with the engineer then they can pull you up as well with them cuz then you can and focus on B pictar A lot of from an or orany perspective from the engineers cuz just talk to so many people then you might have opportunities that you may not have if you're solely focused on your team before we jump back into the episode I want to let you know that the audiobook version of my bestselling book the software Engineers guide book is out now you can get it on all major audiobook platforms like Spotify audible libro.fm Apple Books Google Play and also DRM free MP3 files I started writing this book after a decade of working as a software engineer when I was working as an engineering manager for a few years already at Uber here's a review from the book from senior principal engineer Tanya reallyy who is the author of the staff Engineers path from performance reviews to P95 latency from Team Dynamics to testing GG deifies all aspects of a software career this book is well- named it really does feel like the missing guide book for the whole industry you can get the book at Eng guidebook.com or search for the software injured guide guide book on your favorite audiobook platform I hope you'll enjoy it one thing we just kind of mentioned as As Natural is you always worked on a platform team but when you started uh I I'm pretty sure there was not yet a platform team there and then there was a split called the platform program split it was a very famous email that I think later we we saw I only read it can you talk about what was before there was platform and and program teams and and how did this split actually play out from from your perspective yeah so I I remember when I was interviewing to join Uber uh they asked me hey what do you want to do you want to do some platform stuff I said I want to build tools for developers and they're like okay you're hired cuz no one wants to build tools for developers the the longer version is uh the platform team sort of existed we just used to call it I think mobile had it first I think with than the other ones we just had unofficial platform teams they were working on core infrastructure pieces um the platform teams were essentially made so that the other teams could move faster and not duplicate stuff right there's a lot of stuff that goes into Fe building features and mobile was just very new back in the day so things were just not as well figured out as they are today so you had to build a ton of stuff internally so if you're doing this on every team then it's a lot of ways so I think we when we decided to the platform program spit we said program teams don't need to worry about a lot of the underlying layers similar to a cloud native environment you worry about your business logic your product metrics and we'll give you all the building blocks which could be a mix of Open Source wrapped in our own layer and our own architecture that makes so that you don't mess with the other features because everyone has their own way of building banners or their own way of like styling things not at all consistent for like a app like uber you need to have a common design language so you use common components stuff you would do on a website or take for granted to just didn't exist on mobile right so it it had to be done that way to kind of make things move faster so you don't have to rebuild like another widget or rebuild like another analytics processing thing and do something in the back end so one paino or I wonder how big a paino is but one one thing that looked like challenging to me I I was not on the platform team I was your your customer but you know we we always pinged you with questions and as you mentioned like a year ago or so there were 10 Engineers for about ,000 Engineers what did support look like and how what practices did you figure out cuz I I saw a lot of practices like office hours uh on call rotations of where to B Etc that I have not seen anywhere else like the platform teams were advertising these things it was it was almost like I don't know like like having a a vendor almost like there were like contracts there were actually slas it was like published how how fast you can expect things where where to where to escalate how to tax something with high priority Etc I've never seen anything like this and how did you figure out like support and how important was it yeah so the angle that I think we T about was like hey for everyone else the end user is the customer for us the developer is a customer let's not treat it as like another internal thing let's really be customer obsessed that that was the value we had it over that we extremely focused on the customer experience right developer experience verus the customer experience which means imine if your ubo didn't show up you'd be like very upset so if your build didn't finish exact same feeling okay I cannot ship my code right so we need to make sure that there is reliability and latency guarantees uh for that exact same reason so all the product I think like minded folks had like very much holding in on like product analytics we said hey let's think about what is important for the developers so we asked them things like build time they said hey I want to make sure my CI is reliable has no flaky tests can you make sure the flaky tests are not blocked me because it's expensive with like the build cut you have to chase the release train and that's not fun I'm sure you've seen your team complain about like hey I had this flaky test I had to like resubmit this diff and it's not going to merge in time for the Release Train yeah and and and just like staying lat I'm not sure I think the Release Train left on Wednesday night uh so Wednesday night was the cut off and or or morning I'm not sure but but I know people stayed longer on those days to cat the Release Train meaning getting their their PR as we call the diff merge merge merge on there yeah like it was as as you say it was it was it was a big deal I didn't actually realize that that you were like obsessing about it but it kind of makes sense because after a while it wasn't an issue yeah cuz that's when people complain a lot that hey I tried to catch the train but then something was out of my control the infrastructure was flaky so we missed this deadline now I'm on the hook as a platform team to ensure a heart fix happens which means a bunch of other teams are going to get disturbed because they have to retest all their workflows a lot of R effects right and this are not obvious unless you think about oh we cared about testing because the end user will see the impact if you don't test all these features again yeah right so for us actually the metrics were very important to have like we would have SL slos you would publish them we would try to keep them we would review if we like miss them and then do incidents management and like retrospective so we could like avoid those a lot of the good tooling came out of that was to ensure High reliability high like low latency which means you could ship product really fast as a company and the infrastructure would not get in that way right and that is why some of those metrics you see matters because once we act you talk to the developers we knew what actually mattered to them that's what we focused on so is it just I'm just thinking about this but is it safe to say that I mean I think we can agree that the the the platform team or the developer experience team at Uber did a great job but they did a great job because they you ran it like like a product team like you as you said you talk with the customers you you figured out how to map the how they feel or what they care about into metrics you then monitor those metrics you and then you kept iterating on these things like if if I replace this with you know like a paying customer is the same thing right like that that's what a great product team does absolutely like you have to run it like a product T because then the instanes are good on both sides because if the developer doesn't like the product then you should you should kind of be like okay what do I do to make it better so if the company shipping high quality product then we should also ship high quality product for our developers and and that's how we took pride in making sure that our developers could deliver uh to the end user so yeah you're absolutely right the metrics I think a lot of people might Focus about at a high level may not be enough you're right you have to talk to people you have to look at the funnel where are people dropping off like things like the onboarding for example for new Engineers where would they have difficulty like the dev parts for example was like Hey people would go look at these outdated docs and they try to set up an environment and then just spend a lot of time so having a deer just eliminates that and then later you might have a situation where a build failed and there's a very easy quick fix you should automate that you should not ask the user to like go to it if there's a linter failure in my opinion uh we failed because a lint should not complain it should either fix or not just telling them hey there's a problem is not going to do anything you're so much pressure to ship product that if it's not blocking me I will merge anyway right so if it's blocking me if it can be autofix why can't you autofix right that that's kind of how we took things I I like that what is your take on measuring developer productivity uh your your team was called developer experience I'm sure develop productivity came in this is a very heated topic these days it seems like for the past few years and there's a lot of back and forth of what can you measure what can you not measure should you measure diffs per engineer or per team you know you you probably thought a lot about this or have been in middle of this what what worked and what didn't yeah so I think the very first thing we measured was just sentiment so talking to as I mentioned The NPS survey was very easy because it was very quick signal to see if things are in the right spot if everyone super happy then there's no problem but that's really the case there's always you know problems that pop up um I think as you meture your measurement I think a lot of the Frameworks we see out in the open are sort of inspired by what Uber did early on and took some of the good picks out I think when people talk about like diffs per engineer they're not really saying hey you should commit more dis engineer is something preventing you from moving as fast as you could potentially because I can tell you that Engineers if basically nothing blocked them would love to write code right if you're engineer you want to ship code you want to feel plot about what you shipped there's no one will come in and say hey I don't want to write these mini tis I want to write code because it makes me happier right and and what we were looking at those signals was uh for was hey do you have too many meetings do you not have Focus time because deps were usually a matter of focus time so when we improved that we saw Focus time improved I think when we had Co everyone was working from home the D SP engineer shot up and everyone thought that hey people are working so hard it is amazing it's just because people just didn't have too many meetings initially and over time and everything went to zoom it kind of came back again down a little bit right it's it's usually a way to like make sure that you or infrastructure is letting people move as fast as they can and as you remove Road Blocks you're focusing on the right things and in fact actually one of the things we measured is time for code review more than anything else the time to review was the biggest B neck because as you mentioned you would put up a PR in Amsterdam might wait for review from SF that's already like a 10hour delay yeah and I think the P90s are pretty crazy like it would take days to get code reviewed right and that would just be the biggest blocker more than anything to like ship product and honestly like Ci time is like a blip compared to the amount of time it takes to review right so the in the big picture I think we work to improve turnaround time by nudging people by Auto approving PRS that had minor changes that semantically would not change the meaning or the behavior of the V that helped a lot because you didn't have to get another approval for making like a common fix or you doing a small stylistic change for example so that helped kind of unblocked a lot of those scenarios we had Auto approve or Auto Land which helped because if your reviewer approved then it would merge immediately so you don't have to rebase and worry about conflicts a lot of these kind of small things can actually have a really big impact but if you don't measure you wouldn't know so I think in terms of developer productivity when you talk to your engineers they might say hey I feel less productive because I can't do something then your metric should flow from that rather than just blindly taking a framework and saying hey this is a framework in the industry this should be it right may not actually work I mean it's not Al also the same for all kinds of developers is it safe to say that at Uber you know developer productivity was measured in the sense that there were these you know charts metrics it showed diffs per engineer on on on on your team you could see it and how it changed over time the time to code review Focus time Etc but is it safe to say that these were measured with the intention of finding bottlenecks and what to improve and they were not measured with the intention of using them for performance reviews promotion or or figuring out who you know who's the low performer like is is that a is that a fair assessment yeah that's the usual controversy people tend to misuse it the guidance is always like you know look at them at a high level and that is where a lot of these were aggregated right and this you can find ways to like descend and twice people from bringing those up right because you can easily catch bias we had for example folks on the performance performance company that would be there just to catch bias that hey are you just looking at diffs are you not looking at the Quality and sometimes like people may not have as much output because they have other contributions they're making they might be writing docs so they might be like influencing strategy they might be working on like unblocking teams or like figuring out product and business goals and dising them to like requirements and Engineering so it was definitely like uh I think uh initially people thought of it that way but that was never the intention on that was never the case in my experience at least on all the companies I've been we never looked at it as like anything more than hey it's just good to know that hey if you haven't WR any PRS that's a red flag potentially you had zero PRS in the entire year explain yourself like you will not like we we'll clarify with the manager hey what's going on are they what are they doing but we never say that as like a blanket okay that means nothing right so we always want to use this as a way to say if there are red flags we catch it that there might be something else going on but it was mostly used for unblocking uh teams from Shipping fast M yeah so sounds like it's just like if you have data and you can gather it and that data can actually help you build the right things I mean it will be silly not to do it and as you said start with from what the pain points are what people complain about I mean we always complained about when I was there like it was always about code review and how long it took code riew to take and I don't think we ever measured it I think I left by the time this this was there but if we would have measured it it would have just really shown that our team was we we probably have like you know like our median was probably more than a day because because we were waiting a a team on the other side of the Atlantic and then we figured out how to cut those ties and code is rarely a bottling especially now with like AI tools you could just generate a bunch of code and throw a PR like that's rarely the bottleneck you're bringing alignment or figuring out product requirements or making sure you're not breaking the experience takes the hardest so just measuring output in terms of PR is probably going to be harder as a justifiable thing after this new age of AI tools cuz people are just shipping so much more we use a lot of AI tools at work including our own and we ship a ton more than we used to back at Uber so you're you're now in the the business of of AI uh tools to to help engineering productivity and and Engineers work better how do you see these tools changing How We Do software development I'm talking obviously about The Usual Suspects the the coding autocompletes the co-pilots also the agents and and and also everything else that's coming yeah absolutely so we very thick in the space obviously the autoc compete is a huge boost because it kind of in many cases helps you sort of like less type less but think more right so I think if you probably use a bunch of these tools I do too it removes a lot of the grunt work but it won't remove your thought process because the taste in what you want to do is still there just not the how sometimes it's like okay this makes sense right um but I feel like a lot of the tools are either focused too much on the completion or they're only on the Cote review I think back from when we learned things at Uber we had a lot of inefficiencies in things we wish we could like solve with some sort of intelligent layer like I was for example going unblocking things like hey I knew exactly what would need to happen to like unblock a build or so having some sort of like an agentic understanding of things are happening in the IDE why did you type this character that affects like what's happening on code review and that affects what happens on deploy that affects what happens on like your incident for example there is nothing that's cross cutting so that's kind of what we are trying to solve with guitar is like have this whole stlc under the perview and not just focus on the point solution of I'm just an ID code complete or I'm just going to be on your CI reviewing stuff a lot of those Solutions currently are I believe good starting points and my experience like how I've seen technology shifts happen the much nicer or more important unlocks happen a little bit later into the cycle because since we're very early in the cycle with the AI at this point people are just experimenting stuff um but I believe that there's a huge value add for agents just because as you mentioned like there are some things that are so obvious when you think about it that hey we should we wish we wrote like a script to automate this right for support or for documentation those can be potentially driven by agents but then still have the human in the loop when uh things need to be approved that's kind of how we think about it and like as as you're using these tools like how does it change your development like like obviously we're we're writing less code CU is now like there there's just more suggestions or there's also agents but you know like what does it allow you to do are are you you know just moving faster are you actually having deeper thoughts I'm just interested in like you what what it means for for you the current version of these agents and we know we you know it's it's a service cycle we never know how long it'll go but clearly it's it's only been like no more than two years so yeah I think it allows you to explore more paths that you previously didn't have time to explore because you could experiment much faster and discard ideas much quicker because when you're prototyping you don't need the full riger you can put them behind a flag and then say hey this looks good I think people are calling it VIP coding these days like hey this has the right Vibe uh we can experiment with this of course it's harder when you have a lot of other constraints like you know you have to make sure the software is you know secure compliant a lot of people are not thinking about that when prototyping right so let's you prototype faster which is great and I think what's also happening is I feel now there's all of talk about hey Junior Engineers are are going to get replaced I think it's the opposite my take is that Junior Engineers are going to thrive because they are coming with new knowledge new ways of working with these tools and are going to be much more effective with because they don't have bias of working a particular way um they're able to like achieve things that for example it would take a long time to like even wrap up does doesn't mean that you shouldn't know the fundamentals in fact like uh if you think about uh 20 years ago they had like a DBA and a HTML and like a Java application engineer there was no dat engine a Web Master sometimes right Web Master there was no that job no longer exists no there's no like front end person there's no mobile person there's no like data engineer ml engineer there's so many more archetypes or like you know subcat I think with AI the general engineer is going to see a rise again which means I'm able to jump into for example front end without not doing much front end because I can understand that okay this is what I expect to happen and I would like to have uh this outcome with my with my software which means I can then focus on the other parts of the problem you mentioned Vibe coding and this is this is a brand new term what do you think it stands for is it just prototyping is it just like throwing out ideas or or is it a bit more than that um I think it's VI coding is just you trying to figure out how the system should behave when you're prototyping the good thing about VIP coding is like you're able to like basically iterate much faster with these agentic Loops that now you're just focusing on how would I actually uh achieve My outcome rather than the exact way to do it because you can always change those details later if you have the right abstractions you could swap layers out uh so I think that's changing the way people think about software because if you're someone who has taste and knows like this is the outcome I want to see on the uxx you may not even need to know a framework you should experiment with it till it feels right and of course like you need to massage it to make sure it doesn't make it harder to Evol later on I think a lot of the software today gets you to the initial it looks great but then how do you maintain it is basically sort of like left for later time and I think a lot of people who are still coding with AI are early in the Prototype phase I don't think at Enterprise scale that would work because I can imagine it'll probably mess with your entire abstraction layer you already put in so you have to really constrain it to work well so WIP coding is great for prototyping at this point but that's also why I think uh you can move a lot faster with less constraints because AI has a lot more freedom to explore what one of the the know like problems that we're seeing and if if if you're you know using it you'll also see as like there's this add Osman he called it the 70% problem that you know like Vibe coding and these things that they get you started but then it can get just really confused and stuck and and people see this like people who are not really technical they kind of get stuck and that's where experienced Engineers really shine you said previously that you know you think Junior Engineers will will do great there how do you think less experienced Engineers can deal with this that's one and then the other is uh like is this where we might see you know senior Engineers actually you know Thrive and spend more of their time yeah I think the main differentiator there is like when things go wrong understanding why they went wrong which means strong CS fundamentals strong sort of system knowledge of hey how does the entire thing work end to end so senal Engineers usually tend to have a better bigger picture so if they adopt these new tools actually they can get a lot more productive I can tell from my own experience that adopting these tools I'm able to get a lot more stuff done because I know for example this is how I EXP my system to work which means I can delegate to I used to give it to like a more Junior person to like you know understand the system now I can just give it to Ai and I can check the work and I can come at it but if you're very stuck in the Bas and not want to use these tools you can still do it it's going to be a lot slower and that's totally fine too it's just a matter of like how quickly you want to ship and for things that are not critical you may want to still experiment to see what's out there because the things the way they're going the tools are getting better every day which means means over time a lot of the mistakes that are happening today will start to disappear there'll be like other agents that'll check the work of the first agent so they're not going to be as many mistakes um you still want to have a human in the loop before you check in otherwise you will not have full guarantee that whatever you're shipping to the user is going to be good but you definitely want sceners to focus on the high level system level knowledge and the agentic loops can be more nuanced like more focused on smaller parts of the puzzle after n nine years of of uber working on developer experience you've now co-founded a company called guitar how are you thinking about continuing this developer experience and also what's your approach to to AI what are your bets on where the space is going to go yeah that's a good question so when we found co-founded guitar with my co-founders Ali and Raj we basically saw that a lot of the inefficiencies we had tools were not available back in the day at Uber we saw because we had a golden path for developers now with agents I think you we talked about Dev Parts previously right like hey there's a golden path for a developer to get stuff done so what if an agent had a golden path to get things done for a particular Enterprise so we're thinking of uh guitar like the agents we're building at Guitar uh we just are going to launch a new product pretty soon and you can check us out at guitar. the agentic AI we buil uh is actually inside the IDE on the code review system inside your deployment it's across the board so there's no reason why you can't autocomplete like a deployment shock and not just be stuck inside ID right and having an understanding of things like hey how are things going in production which part of the code is slow these dots are usually what Engineers connect right the context is What flows through your brain um and if there's a standardized path for an agent to like Evol software and maintain it it becomes much easier to focus on the business value and one of the things that people dread doing is hey maintaining stuff like updating liaries or fixing T de or adding coverage we feel like a lot of that grunt work can be done very efficiently with agents so that's where our focus is that we don't want to be just like a code complete or within your uh Point solution we want to have it across the entire stack of software development and we have that experience working on this for many years talking to PE in the industry there's a lot of potential for disrupting the space and the stuff we wish we had where hey I wish someone was on call during this time to fix a small thing we can make that happen because we can give the agent powers to like unblock people when you might be asleep which means you'll be on call less often so we thinking of it from a holistic developer experience standpoint and producing more code is just one part of it it's about doing it reliably maintaining it and making it understandable so newer developers can understand what's going on cuz if you just let AI run a up on your code base and not have an idea what's going on then that's the recipe for disaster so we feel like there's a lot of opportunity for uh making the enti experience end to end uh much more uh standardized so that the golden path for developers and agents is like really well laid out now like you know when I'm listening to this like one part of me is like you know if if I'm a business owner I'm like oh that sounds great because over time I might just have less developers because I'll have a really capable agent and as a developer like obviously one thing that pops into your head is like wow you know like we might have a smaller team hopefully I'll still be there but I might have fewer colleagues or you know I might be unfortunate one to get the cut uh like let's let's let's just run with this that this future is coming and we will have a lot more capable agents in this case what are what do you think developers will do and what what are skill sets that are worth investing in so that you will still be an efficient a great developer in in in the future you know seen as a highly efficient developer 10 years from now you know your your managers will be like oh my God like we we cannot do work without this person we need this person here and we should probably give them a raise because they're underpaid yeah I think as you said there might be less people in a business but there'll be more businesses right that's typically how a lot of these technology ships go if building software is easy then the value unlock will be like actually The Taste and the experience for the end user like what do we actually provide if every looks the same because you prototyped it in like a a website builder then that's not going to be your differentiator right like even I think during the mobile era a lot of apps came up to do things like to-do list or you know like reminders or like you know just scrolling social media but then eventually some clients are great ux they won out because they're just a joy to use so knowing what matters to your end customer is going to be differentiated for engineers to like hone in on so that means understanding the business really well right and then the other thing is like how do you build something for scale essentially as as you like grow the business you don't need as many Engineers probably and that's okay and that might be fine because if you're at a smaller company with lesser number of Engineers if you have equity for example you might have more of the pie which means that's a good way to kind of incentivize people to experiment with more ideas and usually competition is good in that way right so you may not be bound by those restrictions that you have to always is being like a big company I think we'll start seeing smaller more efficient teams who are going to be more productive and will deeply understand the product and that's going to be I think a good way to like compete with other products in that space like the taste the art that goes into like crafting gr ux or great end user experiences is going to matter more than how you wrote the code right now if I'm thinking because I think it's a good good think back that for example even 15 years ago 10 15 years ago we all we already had this right like WhatsApp at 50 they they were barely at 50 people when Facebook acquired them for $19 billion which is crazy this was no no wayi know nothing but clearly they were a very efficient team Instagram I think 12 or 13 people also they they they built an app that was already 30 million users with that small like even today that would be a big deal so it's like to to get that skill set uh of you being able as an engineer being able to do a lot with these tools with the small team you mentioned taste you mentioned art uh what other things that are worth focusing on that will that should should probably help you know like do a lot more with less yeah I think uh having an understanding of the system layer is good so I think the scalability part we didn't cover so how does your app or software scale to more users what parts of the system are less efficient more efficient because all code might do the same things but less or more efficiently yeah and how do you maintain it right so that's the other challenge a lot of people run into as software systems grow is it easy to upgrade because you have less external dependencies is it harder because you have so many versions the software might work the same today but okay three years on the line if you can't upgrade and move faster your competition is going to basically move faster than you because they started on a clean slate we see that all the time I think there's a lot of Legacy technology or database systems people are trying to move to Modern systems it's a big cost right and that's very expensive a lot of engineering time wasted uh so if building for basically maintain a AB ility is going to be important as well right and building for like EI either AI agent maintainability or human understanding so that you don't have to like you know look at outd dos or do an expensive migration uh I mean for example if I could wave a magic wand and move all my micro Depo St mon depo and everything just work that would be awesome I don't think any system can do that today yet maybe it would in the future but if that's a reality because you designed your software to move that way that's great right so having that knowledge know is not because AI agents are fundamentally based on Expert knowledge right so if your knowledge is like understanding of your entire business because they can't really tell you how should you write your code for your end user right what matters to one business may not matter to the other right some parts of the software might work well for one segment but not work so well for the other segment so really understanding those business values will differentiate just AI drivent code versus how do you actually use that code to make your business like actually thrive so again it might not happen right or or not or it might happen slower than we expect Etc but what I'm hearing is like even if it does happen having expert knowledge helps having taste and H and you know you kind of get taste from seeing things or failing things so is it safe to say that it's probably a pretty good strategy to try your best to go and work at companies that are either building stuff at scale or they just have kind of taste you know there's startups that are known to to do these things so that you can build up one of the muscles of like scale taste if you can switch between them because in the future I assume that the the some of the kind of repeated like the the successful people who you who have a pattern will will will probably have the pattern of like oh they worked at company that kind of did pretty groundbreaking stuff at their time and now that're we have these tools well yeah they're they're doing even more groundbreaking stuff and they probably built a bunch of their experience yeah the experience matters a lot like knowing a lot of the stack helps like when I was at I didn't do much work on front end but now I'm at Guitar I'm doing like front end I'm doing infra I'm doing a bunch of stuff and I'm using agents to help me like we use our own agent to like rewrite a bunch of our stuff and I was like okay now I understand why this works that way I'm a general engineer now I have strong fundamentals let's say I can see the system I know this is how it's supposed to work now I up level myself without spending it's kind of like that Matrix movie where Neo gets that you know upgrade to like no kung fu yeah but doesn't mean that he can beat the master yet you know like he still has to train up and understand it yeah I I I like that so so with that let's goose with some rapid questions I'll I'll I'll just shoot out some some things and then you go what is your favorite programming language and why rust rust since when yeah uh since guitar actually I was like initially like oh my God rust has a bad reputation for being hard but it makes things so safe it's hard to have I think like a if you design it right I mean you can still have them but like you can really understand the type safety is there to help you out and you start treating it like your friend then it unlocks a lot of stuff for you this someone who's who's you've spent a lot in Java and go right like you've probably spent like half your career and those yeah it's funny because when I graduated college I told my friends if I would never work in a company that does Java and I did Java for nine years so you know never say never uh what's an AI power dep tool that you use un like uh we use our own agentic thing quite a bit um Jimmy which is Agent Jimmy yeah CU guitar you know um we use cursor as well for like you know the auto complete but we feel like Jimmy gives a lot more end to end stuff for us so we've been using it veryy heavily uh but you can also try it out hopefully soon and then I use a bunch of other smaller like claw directly obviously for question and answer stuff uh how do you play with like some of the newer deep seek research and models and stuff like that because I've been hearing good things about them for more deeper work but haven't had a chance yet and uh what's a book that you'd recommend uh my favorite book uh I don't read too much but I read this book early in my career called head first design patterns which is a a little uh old at this point but the patterns are really good because it really makes you understand how you can layer things a a lot of software today is just abstraction or abstraction or abstraction if you think about them so really helps you understand what pattern makes sense when and when to use it how to transition between them uh I would highly recommend that if you're trying to go into like a more systems level approach and want to design software for maintainability that's very crucial making it pluggable composable and like replaceable under the hood like a lot of the migrations we did at doer was Zero downtime with like no code freeze when we did the Java migration there was no code freeze so that was aing achievement yeah yeah and it banks these days still you know like they will shut down their thing for some migration for like a whole day on the weekend in in 2025 it's it's a big difference as as you say well gotam this was I really enjoyed this conversation it was good to to to reconnect and just see how developer productivity empowering developers you're still like just running with it yeah I I think this is the area I love like working in this area I can't see myself doing anything else I've been doing it for 10 years I hope to do it for 10 more we'll see but uh I hope I can make more developers productive I hope you also found this episode about Uber's internal tools developer productivity and ai's impact on software engineering as interesting as I did you can find goam on social media as linked to the show notes below and check out his company guitar guitar. for more deep Dives on Uber's engineering culture check out the pragmatic engineering articles Linked In the show notes below if you've enjoyed this podcast please do subscribe on your favorite podcast platform and on YouTube this helps more people discover the podcast and a special thank you if you leave a rating thanks and see you in the next one