Developer productivity with Nicole Forsgren (the creator of DORA)

s is a satisfaction metric just ask people right are you satisfied with the tools you use are you satisfied with something cuz people can usually tell you pretty quickly like I've seen data across an entire tool chain that was pretty fast like it was good for the company but you ask the developers and they're like this this is nearly impossible performance you know outcome metrics we definitely want a is activity all accounts anything that's account right we usually have a lot of activity metrics C is communication and collaboration this can be looking at meeting schedules it can also be looking at how well apis are working or if they're breaking and then ease efficiency and flow those types of things those types of data points and metrics give you additional Insight Nicole forren is probably the best known expert on developer productivity she is a co-creator at the widely used framework Dora and space lead author of the book accelerate Nicole started her career as a software engineer at IBM where she experienced firsthand the challenges of being given unrealistic deadlines by management who understood nothing about developer productivity seeing how developers burned out around her she entered Academia researching devops and developer productivity she then joined Chef soft Ware and then co-founded a boost trrap startup called Dora Dora was acquired by Google and nickel worked on developer productivity here she later joined GitHub as vpf research and strategy and she is currently leading research initiatives of Microsoft she's also working on a book related to developer productivity one that we'll touch on in the podcast in our conversation today we cover Dora and space examples of using these Frameworks and advice on how to make the most of them measuring developer productivity whether measuring PRS make sense and why this area remains challenging to quantify traits of high performing enging teams and why time to onboarding can be very telling about the efficiency of a team and many more interesting topics if you're looking for advice on making your engineering team more efficient or are interested in ways to measure developer productivity this episode is for you if you enjoy the show please subscribe to the podcast on any podcast platform and on YouTube thank you this greatly helps the show to spread to even more listeners and viewers with this I'm very excited to have Nicole on the podcast I want wanted to put a pointed question at you yeah um you've you've been studying developer productivity uh both at at the the theory level but also very practical level what is your take on the importance of PRS or diffs and the background is here that there's now A Renewed conversation about this uh there there's been um this viral thread going around on the concept of ghost Engineers who might not be producing as much uh kind of like code on a monthly basis and you know that's something that that can be measured and also companies like uber and meta they do have tools for managers and directors to track the number of diffs per team they don't say that they're going to break it down per engineer it is possible and also there's a new DX core for framework which is something that uh is is going around and they also suggest uh this is that the number of diffs at the team level should be measured I'm curious how important is this the diff frequency and how good of a predictor is it for productivity or unproductivity at a team at an individual level I love this question thank you so much um I will say PRS and diffs uh both are good signals and are terrible signals um which is probably not helpful but like let me kind of like dig into this a little bit more when we think about using the word productivity in particular many folks especially in leadership are thinking about the traditional economic definition of productivity which is the rate of output right what what do you produce and then sometimes that'll be a ratio right the rate of output uh per inputs and so as we know that doesn't apply very well to software or it's really hard to disentangle right is it a feature is it a shipment is it a full product is it what right it's it's different than in like farming I grew up my grandfather was a farmer right like how much do you put in and then how many potatoes did you get out yeah like that's fairly straightforward and when we think about GDP that's often what it's looking at but in technology things are things are just different because many times you'll get many many many more things when we think about cameras and phones or photos right we could take a handful of photos now we can take millions of photos and it's basically free and so many things in technology are just flipped so with that context um I will say they're important in the PRS are important in a few ways one way is that it does give you a relatively reasonable view into what work is getting done and what work can get done through the current system and processes that you have right uh with a couple important caveats and disclaimers right many senior Engineers don't have very high uh PR output or code commits now for those of us who have been in software for a while this isn't surprising right because as you become more and more senior you're doing things like unblocking folks you're doing things like architecture and design work you're doing things like supporting reviews and mentoring and maybe like pair programming with Juniors and so your commits your PRS yours finger quote right aren't showing up this episode is brought to you by DX the engineering intelligence platform designed by Leading researchers measuring developer productivity is a decades old Challenge and many technology organ ganizations still struggle to get it right DX takes a scientific approach that offers complete Clarity into productivity and its underlying factors it combines data from development tools with self-reported data collected from developers so you can see both the what and the why the insights from their platform cater to leaders at every level of the organization from CTO to platform teams to Frontline managers learn why some of the world's most iconic companies like Dropbox plade twilio versel and gust still rely on DX just visit get dx.com pragmatic engineer that is g dx.com pragmatic engineer yeah and then you're also pulled into recruitment you're leadership meeting performance calibration oh there's this team on fire we need you to help out you know the higher you are staff and principal level it happens all the time exactly exactly and so I think it's really important to think about not just what we're measuring but in which context and who we're measuring because that's not I wouldn't call that output I would call that one aspect of throughput right because it can be really interesting and important to see if there are blockers why is it if there's friction why is it is it an overly complicated code base is it a really really difficult to use uh text stack and developer ecosystem is it that our PR assignment process is just like bad right either it only goes to one person or it doesn't like get shared out very well and so one person or just the assignment is the blocker so there are there are a handful of things to look at there um it's bad as a just straight solom metric but I think everything is bad as a solom metric yeah what we always want to do is take kind of a constellation of metrics that help us think more holistically Bally about the context in which developers are working what are the tradeoffs that they are making so I've been uh working with some folks to um kind of kick up we we started the Eng Thrive effort of Microsoft and it's a way to understand and start conversations at several levels in the business about how can we help our Engineers Thrive and we use a handful of metrics across many dimensions and it was you know inspired in part by space uh the space framework because then we can look at qualitative feedback right Microsoft uh you know famously has insat uh which is kind of like a satisfaction uh qualitative survey that's like every six months right I think I remember when I was there it was still going on and some of them are coming out 15 years ago yeah some of them are quarterly now so that we can make sure we stay on top of any Trends any changes but then we also want to capture some objective data from our systems because that can provide additional insight and you're not asking people for it all the time you're not interrupting the work and so there you're looking at you know examples of quality and outcome metrics you're looking at examples of any delays that may come up now I'm just giving like broad examples this isn't necessarily what end Drive uses but uh you know like build time or uh deployment pieces or security outcomes or quality and stability outcomes PRS and the data that are around PRS right so it could be number of PRS it could be time for PR it can be the time that it waits from review those can be helpful in this constellation right because then you're also seeing and you can think about a story of why people are making decisions to do the work that they're doing and what are the impacts and implications and I love that because it can help Inspire larger questions around when we're making Investments now I'll say Investments whether it's money or capital or time where should those go right how do we want to balance uh feature delivery and and customer value delivery with internal Investments because those are so tightly related right I like to joke this is our hot AI moment right and everyone's like we'll build all the AI things just everything's AI you have to go faster it's real hard to build the AI things when like your pipeline's trash like it's super hard if your developer experience just isn't there I I agree I also feel that the reason I've been thinking why is this trending right now like people seem to be really touchy feely about the fact that you can measure diffs this uh this survey was specifically about remote Engineers you know like like you're doing less and I feel that what is happening is a lot of things that what what you're talking about what people talk about developer productivity it's like you have a team that's doing well everyone is you know doing their work and we're trying to figure out how to do better what's happening where are people stuck where are the bottlenecks but there's another question which is Performance Management which is uh I have a team and I'm a non-technical manager or I might be you know removed from the layers I'm the director or CEO and I just want some stats that tell me who's performing well and not and there's an assumption for some reason for people who are not uh working here that I can just look at some numbers may this be output or some scores and I'll see people scored and my question to you how important or is it possible to you know run a good engineering team without a manager or a lead who hands- on and kind of understands this context and you know like knows that what's what's going on is is kind of involved cuz I'm kind of seeing that there's a real desire from people investors Founders who are removed from technical details to get some of these stats you know some of these developer Frameworks so that they they can tell without understanding anything oh this engineer is good and productive that engineer is not a is this realistic and B is this something new or or have the has the business always been asking for something like this I'll I'll start with that last one is this new I would say it's not new right like we've we've been we the Royal we that's good business and researchers have been watching people work and trying to uh measure how people work and trying to assess how people work with summaries and numbers for a really long time right there's something in research called the Hawthorne effect which is um when the what effect I love it uh so uh this gentleman was working with uh I want to say it was like some type of production for process and he would come into the office he would you know like flip the lights on do all like watch people work count things and he would leave and like they would count things and the numbers would be different well so he was like it must be because the lights are on cuz I guess there's like if I'm remembering correctly right like he turned the lights off so they start turning the lights on but it's not like consistent turns out like people are doing more like when you're there when someone there when they know they're being watched right athletes tend to perform better when there's an audience right us develop first we'll definitely do as soon as we know what's measured we're like all right we're optimized for that absolutely and I think this is where we have some real limitations around things you touched on it a little bit when you said like we can measure PRS so many times people will default to the measures that are easy and quick to operationalize number of commits number of PRS number of lines of code number of yep what can we grab quickly and I think this is uh one thing that prompted space the space framework is yeah counts can be useful yes if you have access to data that you can quickly operationalize and grab sure right inserting caveats around like what's the accuracy the reliability of the data how are you Gathering it are you doing cohort analysis but also that will only show you a few things and the few things it will show you are the parts of the business that are easy to operationalize and it will ignore everything else and so we often want to think about you know s is a satisfaction metric just ask people right are you satisfied with the tools you use are you satisfied with something cuz people can usually tell you pretty quickly like I've seen data across an entire tool chain that was pretty fast like it was good for the company but you ask the developers they're like this this is nearly impossible we are going to break soon because so much of it was in systems so you'd think it was automated but every single handoff point had to be manual because they were not integrated and that's one of the biggest weaknesses for uh operationalized data is missing that handoff the the boundaries between systems and the boundaries between processes so people can tell you pretty quickly um act so performance you know outcome metrics we definitely want uh a is activity all accounts anything that's account right we usually have a lot of activity metrics uh C is communication and collaboration this can be looking at meeting schedules it can also be looking at how well apis are working right or if they're breaking and then ease efficiency and flow and so that's where we're looking at time handoffs um you know th those types of things those types of data points and metrics give you additional Insight if you can grab at least three different categories you're going to be in a much better position now you also asked and I'm I'm kind of tying into the earlier question which was can you evaluate and understand how well a team's doing can an enge lead or an enge manager do it if they're not like there I think at that level right when you've got a tech lead when you've got an Eng lead or an edge manager being absent and unaware of context does no one any favors right yeah at some level though I think there can be benefit to having a holistic set of metrics to help leaders guide decisions around which type is an example what type of blockers or barriers or friction points are affecting most of the company would that be a good use of a central resource what types of barriers um show up that are probably out of the realm or scope of control for a development team to fix right sometimes their documentation can be but it's something like a central build system no no individual team no is going to be able to fix that or or it will be heroic effort but the larger the company the the more impossible it is exactly and so this is where I think there can be some value right especially when we're doing things like cohort analysis and and kind of holistic measures and having it be a discussion where people can get curious and can and can talk through what they think they're seeing and then go confirm because it's it's just not realistic for an executive to meet and be fully embedded with every single engineering team that exists and every single product team that exists and every single marketing team that exists to try to figure out you know get a good feel for the business this makes a lot of sense and I really like about space how to me that was the first framework that kind of made it really clear that this thing is complex and and we can measure a lot of things and we should measure multiple things and sometimes they will you know tell us different stories but we need to look at the whole picture but before we go too much into space on what what happened before before space there was Dora uh you mentioned your your first start it had a massive impact on the industry the the book accelerate uh I think had had a huge impact as well I see it on the shelves of almost every engineering leader and if you've not read it I I think it's a great read to start with at least and in the Dora framework there are four key metrics the deployment frequency for D lead time for changes change failure rate time to restore service and still today a lot of engineering leaders who I meet they read the book they kind of hold it up and say like I've read the book I've I've cracked it what what it what for us what we need to do to be a highly performing team here are these four things we're going to measure it and if we have high scores on it we'll be there now clearly things have a v sense but I'm interested uh what where do you see the usefulness of Dora so teams who are optimizing for this and getting to this thing you know how far can they go and what is the point where it's kind of time to look at other things and you know like for example how space and and other uh things evolve from here Dora is a lot of things uh Dora is a how I kind of refer to the entire research program but a lot of folks just when they hear Dora they think about the four metrics and so I think understanding that there are both of those are important and here's why the four metrics end up up being you know through a lot of statistical tests over 10 years now like even when I'm not doing the analysis it ends up being a very good indicator a really good signal for how well your development pipeline is working right is it relatively fast um how many how many things can you deploy uh what are the quality outcomes at the end of that right and so for folks who are just starting out that can be a pretty place to start uh now I will say sometimes folks just hate me because they're like well this doesn't work here you know I'm in air gap systems I'm in wherever find ways to accommodate for your environment I do work with folks that work in air gap systems that use Dora and so instead of the final and by by air gap systems can you just tell us for for the ones of us who are not familiar with that term yeah so air gap systems are where you have some some Gap in your deployment process usually right before like final deployment where uh there is no communication between the two sides right there is there's no input you can't just push and it deploys all the way through um so some folks that I work with that have air gap systems is the US Navy right uh they do it so sometimes it's it kind of makes sense why they would have that right many times it's for security purposes but sometimes it's because you literally have to take something out to a ship or a submarine and install it there right that's a good way to ship on a ship yeah right um and so there you know we say say okay well then what is what do we want to consider our deployment pipeline when we want to think about optimizing and improving this what should that be now at some point you do want to consider flying out to a ship or a submarine and installing it yeah but for the teams on the ground you can go until that pre-prod environment because that's within the scope of control and that for all intents and purposes the way they're thinking about it that is what they're thinking about now this is the same is true for um like iPhone and Android apps you can't push to the App Store every single day that's not going to that's not real right but you can go to pre-prod and that can give you great insights so don't don't get super caught up in like precise definitions because we can we can always accommodate right now your other question is how far should you go and what else is there now sometimes folks also tell me that they hate Dora the Dora 4 because it just tells them bad they're doing but that's where the rest of the research program comes in right because there there's a BFD a big friendly diagram it kind of points to the many things that can improve these four metrics things like improving automated testing things like improving uh continuous integration things like improving you know your your Cloud deployment and your and your Cloud usage so it can help it's a signal but it's not the action right how well are we doing and it can be be good because as you generally start to improve you start to improve if there's a huge drop off you should probably take a look at your system right so from a signal perspective it it's pretty good it's not everything though and one thing I always try to point out is that Dora only goes from commit to production uh now back when we were starting this this was like uh 2013 that end to-end software systems had not been studied very much and measured because it's so so difficult to measure them and so we focused on that part of the engineering system because we thought we could have some really good impact deliver some good insights um also because that's the part of the process that can be highly engineered right when we think about writing code we want things to work better and be faster but also sometimes we just need to take 30 minutes to go walk around outside to solve a problem sometimes we're sketching on a notebook to solve a problem sometimes we're having discussions around prioritization once you commit though you you should have opportunities to optimize all of that system it should be high predictability and low variability and so that is also a nice way to think about that because if your uh tool chain is really out of whack if it requires a ton of manual intervention if all of your tests and your builds are failing if it's highly highly variable sometimes it's super fast and sometimes it absolutely breaks and you you can't figure out why it's a great place to start because you can engineer the heck out of it but it's not everything developer experience ends up being incredibly important now it's related because if you can't get feedback on something for 2 weeks and then you find out that you know the thing you committed you thought you were done with you were no longer done with I have to drop all of my work I have to reset my environment I have to get my head back in the right head space I have to figure out everything that's happening so fast feedback is important but the rest of the developer experience the holistic developer experience is also important can I set up an environment quickly can I provision the resources that I need quickly can I write code and do local build and test quickly with good insights and so that's where identifying the opportunities to really understand and get to I like to look at this as a you know constraints problem theory of constraints where are my biggest blockers because I can't fix everything and I sure can't fix everything in the next you know month do do I understand correctly that Dora started with kind of developer productivity if if if you will as as you said like until we ship just to get some data from there and a lot of the research that has been happening since thinking about space thinking about the devx framework thinking about some some new things coming are are starting to look at not just this but also the other part or the other piece of the puzzle I'm just just putting it here as you mentioned developer productivity and how all of this uh is affecting one or or the other is this a fair way to to say where the space is progressing or how do you see things evolving since Dora U to together with space and and some of the newer research that you're embarking on yeah um so a lot of people have studied various ways to improve the active writing software right I don't want to say development because development is a lot of things but like when you're coding in an IDE what are ways that we can improve that um but I think there has been kind of this continued Evolution both with the tools that we have with the processes we have with the things that we've learned prior to think about this uh from an end to end type of a view right so not just commit and not just writing code although many people focus on that and they're the experts and they can tell you everything down to the millisecond of how long things take and like at what point uh you lose focus and you break concentration so I think all of that all of these individual pieces are very very important and for for some folks myself included my team uh we like to look at the full experience to to kind of test what that looks like and I think it's fair to say that that is part of the evolution of space is you know we had learned so much about the you know commit to prod and how else do we want to think about that and I had been chatting with so many companies for so long that kept saying well how do I Define a metric then how do I pick the right thing or once you're in Dora and let's say Dora you know the analysis some kind of constraint analysis tells you that build is the biggest problem or PR is the biggest problem then the question is well how do I measure it and so space was useful here because it can measure the entire developer endend tool chain or you can just come up with space metrics specific to PRS how satisfied are people with the pr assignment process uh what's the outcome of PRS right like what percentage of of PRS get through within you know a day a certain time frame or number of viewers activity metrics how many PRS do people uh commit how many PRS are they assigned communication right like what does the communication piece look like and then efficiency and flow how long does it take where are the delays so so you can apply space to a specific component as well and turns out when I went back and looked um Dora is an instance it's an instantiation of space oh wow that's that's kind of reassuring yeah so deploy frequency is a count right that's an activity metric uh lead time to deploy is efficiency and flow and then change fail and time to restore are both uh performance metrics so getting into developer experience what like as you mentioned this is becoming increasingly important uh especially with midsize companies and larger companies maybe startups don't care as much just yet but what does this term mean to you what is a place that has good developer experience I think developer experience is what a developer goes through like what's their lived experience I'm I'm very being very reciprocal in using the same word uh but in research we like to talk about like what's a lived experience right what does it feel like and what is it like to do the job daytoday um now many times people kind of conflate like a good developer experience with just using the word developer experience but I do think it's important to think about them at least a little bit separately because the experience is just what the work is Right could be good it could be bad that's what it is and then if we want to improve it we can look for ways or we can identify ways uh that take away from from or in inject negative like not wanted things in that developer experience which often looks like friction which often looks like blockers and barriers which often looks like confusion right if we can't find information if we can't find the docks if we can't find any of the things then that gives us you know a a an improving developer experience or sometimes there's a degradation in developer experience when we roll out sure people will think about this right when a company rolls out a new HR system or a new expense system or a new budgeting system it's going to slow me down and that might not be what people typically think of as my role as a developer but if I need to ever interact with these systems with a type of regularity you've got an initial learning curve which is understandable but if that delay or if that friction remains for an extended period of time that is going to affect the way I do my work it will affect the time that I have available just just as you say an example that came to my mind when I worked at Uber when I started started things were pretty kind of free you could you could access a lot of uh systems and and there was some logging of accessing sensitive data when you went to profile you had to just right quickly why you're doing it and then boom and then as Uber grew obviously they needed more compliance so what happened is when you wanted to access that systems or or even data as as I was debugging customer issue I needed to fill it out and my manager or my director had to approve it now suddenly stuff started happening we're like okay I need to just you know fix this bug and I couldn't CU my director my manager was uh traveling and it was like two days because then the skip level and suddenly I lost contact I didn't get back and I mean this policy change uh that had nothing to do with developer it it was just really like hammering down I never thought of it as the developer experience was there I was like oh there's just you know things but the result was like things were tasks were getting done slow especially these debugging tasks it's an interesting way to think of it I I don't think we thought of it like that I love that you bring up not just the time delay but also the cognitive implications right how does this affect your cognitive load in terms of having to come back to it later and regain context but also maybe this is just me but like I'll joke that I just end up having to use computers in Anger too often right because when it keeps breaking I can handle so much frustration and still yeah be at my best in terms of creativity and sometimes it inspires me but at some point I'm like not today we're done right like I've got to jump over to some administra that needs done anyway and it just doesn't require the type of problem solving and creativity and Innovation that great work does and I just I can't do great work when I'm just surrounded by friction and interruptions and broken systems and unnecessary process some of it's good but also how can we find ways to minimize that process right security and compliance is super important are there ways where any environment that gets spun up automatically has all of the or most of the preconditions in the settings done versus a developer every single developer having to do it manually every single day and every single time like multiple times a day now in your experience because you you you talk with a lot of different companies large and small you also work at at some very well-known ones again pretty pointed question do you think developer experience in general is better at some of the largest companies you know the the Facebooks the microsofts the metas ETC or at small startups uh which you know like these big companies that I mentioned they do invest a lot in in developer productivity and sometimes smaller companies look at them as Envy they have platform teams they build custom tools they they have teams that measure all these things and try to improve it startups you know small like two three five person startup has none of that they they just kind of do whatever they can so do you see a difference between them because sometimes startup move pretty darn fast yeah I will say yes and no um and I'll a joke I used to make all the time when people would come come to me with door and they would say well this can't apply to me because I'm in a startup or there's no way this could apply to me because I'm in a large company there has to be something different I'm like well startups tend to be faster they have less bureaucracy they have less levels large companies have more funding they have more resources they have more ways to help improve what that developer experience looks like so uh tldr no excuses um but what do I see I see a mix right so I've been at a large company that had I'll Google right probably the best developer experience I've ever seen it was incredible I was commit C on the first day it it was great um they have really taken time to invest in that and prioritize that and research that right I worked with some incredible researchers there um that did great work here jaspine uh Emerson Murphy Hill he's at Microsoft now they care deeply about that and it's evident through almost every system that they have they do not tolerate friction I've been at other large companies and advised especially I've seen you know as I kind of talk to large companies some of them are very good and some of them are very bad like it's I don't want to say bad it's not a value judgment but it's incredibly difficult to get work done there why because many times they're working in large Legacy systems Legacy code bases they're working with layers of bureaucracy that slowly built up over time and now it just makes it incredibly difficult to do work yeah startups um have often a slightly different set of constraints right so they can move fast they can pick whatever tool they want they can just go but that also means they have less infrastructure in place right they have to build that as they go they have to kind of acquire that um if they have to go through security and compliance there's not a team dedicated to security and compliance at Dora we were a tiny little like three person startup that pretended to be like a 10 person startup we did stock two compliant because some of our customers required that getting a stock to and basically a two two and a half three person company that's a lot right like that congrats we I I I did that at we did that at Uber but we had a massive team dedicated people it stopped work now shout out to jez because he did uh the Lion Share of that work um but there was no development happening through those those that week or two right it just doesn't happen unless it was like emergency bug fixes or something yeah so there's there there're always trade-offs wherever you go one thing that keeps coming I just keep asking it because I I i' I've also been thinking and and covering and you know being involved in developer productivity here and there for like many many years now why is it so darn hard and you've been doing this for a lot longer than I have why it it just feels you know like we're software Engineers we write code we have an output okay we're people as well but we're kind of pretty organized people it feels eventually we should we should Converge on something that's kind of you know like easy enough to understand and and the the space should go down a little bit you know we should understand more and more and there's less ambiguity it doesn't seem to be happening how are you seeing it why is it hard I think it's hard for a lot of reasons right one is that most of the work that we do is let's say invisible right like you can't walk down to a production line and see where something breaks um y that's part of it and the systems that we use and we work with are increasing in complexity right when we think about like Cloud systems that changed the way we had to think about uh distributed systems and and orchestrating them and architecting them now we have ai ai also changes the way we need to think about the infrastructure and the pipelines and the tooling and the Model Management right how we think about that and how we account for that just it's ex it's existence and its emergence it's like continued growth changes the complexity of the work that we need to do and the supporting systems that we need um another thing that can be challenging is many times leaders and exacts aren't feeling the pain right how many CEOs have been developers and are currently so some of them are developers but also how many of them are currently using our systems to do work and using them for at least a week or two right they're not managing their calendars they're not which they shouldn't be but if they had to feel that pain I think the motivation would change right this is this is so interesting because when I talk with David Singleton who used to be the CTO of of strip for seven years he told me that every few months he takes a few days where he does development work which I found very interesting because not many CTO at his level do it he had thousands of people underneath him at the same time stripe is one that invests very heavily into platform teams into developer tools that they had one of their AI systems early on to help developers and for I I was thinking and he told me like look if you're not doing this you're not he he said what you're what you're saying but very few people do it so I think that's just a really interesting points I think anyone listening who's in a leadership position especially in a se- level position if you're not doing it you're you know there's companies that that that are doing it a lot better and you might be Miss some things right and in that case because maybe for whatever reason an executive team can't be carving out you know several days to do development work uh Courtney kler uh is at Starbucks now I met her back when she was at uh Nordstrom she had a phrase that has always really kind of stuck with me and resonated and it's honor their reality if you ask a handful of developers and Engineering managers what it's like to work in systems you don't even need a survey right like if you want to send someone if you want to chat with a handful of them you need to honor that reality you might not agree with it but that doesn't mean that isn't their lived experience of doing work and sometimes that's going to be the best proxy you can get but getting it is super important and listening to it and being curious about it is super important this episode was brought to you by Sentry buggy lines of code and long API calls are impossible to debug and random app crashes are things no software engineer is a fan of this is why over 4 million developers use Sentry to fix errors and crashes and solve hidden or tricky performance issues centrica debugging time in half no more Soul crushing lock sifting or fague user reports like it broke fix it get the context you need to know what happened when it happened and the impact down to the device browser and even a replay of what the user did before the error centry will alert the right div on your team with the exact broken line of code so they can push a fix fast or let autofix handle the repetitive fixes so your team can focus on the real problems Sentry help monday.com reduce their Errors By 60% and SP up time to resolution for NEX door by 45 minutes per Dev per issue get your whole team on Sentry and seconds by heading to sentry.io pragmatic that is s n y.i pragmatic or use the code pragmatic on sign up for three months on the team plan and 50,000 errors per month for free now most people listening who are software for engineers or engineer managers who are pretty Hands-On or or stay close to the the teams they will agree with with both of us that this is important Dev develop productive is important it's hard and it if you're above a certain size at mid size or above you kind of need some investment it's nice to have dedicated folks who help out May that be provisioning infrastructure standardizing things or Building Systems the number one question or not the number one but a verific question I get from people especially injury managers senior manager directors is I see this as a problem like I think we should invest more into developer productivity or or internal tools that help us with developer productivity platform teams I need to get funding for this and i' like to make a case what is your suggestion of doing so and I'm asking you as someone who's who's advised a lot of companies and seen them what are ways you've seen engineering making the case uh that there should be an investment in platform teams that directly indirectly address some of these topics because you know the business the the the usual push back will be injuring expensive enough we gave you enough resources you should just start it out as you are yep um in general I would say some of the best ways to communicate include both data and stories right if we can have some numbers that can be incredibly helpful it can drive a home if you only have numbers it it becomes abstract it's just an another spreadsheet or a number on a document now if we only have stories it can be easily dismissed as a one-off that's just that one thing someone is just like a disgruntled engineer right someone some team just doesn't like it was just that weak um and many times resourcing will come down to like what are the tradeoffs and you want to take a realistic look are your engineers having to uh you know rewrite like roll scripts every single time they deploy how much time does that take so go around to do just a rough estimate like at Chef we did this early on and we just had a spreadsheet this does not have to be you know super Intense or rigorous we had a spreadsheet where at the end of the week we asked engineering managers to like kind of chime in on like about how long certain things took because then you can do some kind of like rough back of the napkin math to say this is how much time we're losing in aggregate now if you want us to repurpose Engineers we can but this is the impact on the product because we can come with a recommendation but it's going to be someone else's decision and the more information we can give them about the trade-offs that exist in the system the more valuable it will be and then pair it with a story now it can be an actual story it can be a quote of a comment or it can be a very true amalgamation like almost a Persona of what a day in the life of development looks like now and then what it could look like later and then it comes back to being a decision around like where are we going to invest resources we're always in a constrained environment no one has unlimited resourcing maybe a couple AI companies right we always have constraints so what's the best way to do this I will say one additional thing to keep in mind for folks uh kind of kicking this off or making decisions you know about whether they should be investing in this is to acknowledge both to yourself and to leadership that there is a Tipping Point in this tradeoff where further Investments exploit you know dedicated Investments won't deliver that much more progress yeah which I think is important because there's you know I've talked to exex before who were like I'm not going to take this meeting it's just a black box everything's broken all the time so being upfront about the fact and saying I know that there are so many problems and we could like spend all of the company's resources fixing all of it I know that's not what we want to do I am trying to op optimize our workflow so that can deliver value to our customers and I think this is going to be a very important leverage point and once we get to a point we'll continually reassess where it's good enough we will retain these Investments right we'll keep a platform team but we won't need additional investment until it starts negatively impacting the business and that kind of opens up a a door to have a a real conversation around like what are the constraints that we have right don't just ask for head count yeah I I I love the both that you're saying making the case well first of all making about trade-offs because I think that's a lot of us to work with the fact that that mentioning that you can later repurpose teams and I think just the reality right now in in in the market it's all about efficiency uh there there's now push with a lot of companies where they're hesitant to hire more software Engineers so I do think folks will need to get a lot more creative and also just accept the reality that we might have you know we we've we've had a really good run of the last 10 years where we like so many places got almost unlimited investment in these areas as well and which meant that Engineers didn't have to feel some of the pain we might need to you know just decide that some of this pain it it is here let's solve it let's try to you know work around it but let's solve the most important pain point so I I feel there might be a bit of a turning point that a lot of companies not all of them were like what what you've been used to you know getting resources to uh just building teams that that help it either help yourself or or sort it or look at tools that can make it change up your workflow so is is this also a little bit what you might be seeing y yeah absolutely so talking about highly productive companies in now in in 2025 some of the most kind of productive companies that that that us a teams engineering teams I would say that just get stuff done what are some of the common traits that you see across them and are there some common practices that they use maybe may they be a startup or a a company sorry A Team within a larger company many times times what I get instead from a company is how do I make my company High performing right but it really is very contextual to teams right because there are team Technologies there are team onboarding there are team cultures there are team Norms so it can be a group of teams right but I often see very different I say like performance profiles right across a company now what makes them great and you can be great anywhere I've seen a great team working on mainframes I've seen great teams working on you know SAS software I've seen bad teams working on SAS software and I've seen bad performing teams working on we've all seen it we've we've all seen this right we we might have even been that team may Pro maybe yes that's a yes um the thing that I see in these teams is is is a few you know key characteristics and values um psychological safety is always in there right I'm sure several of us have heard about that you feel safe taking risks you feel safe asking stupid questions you feel safe uh doing so many things another part of psychological safety is depending on your team and also knowing that your work has purpose and that can come back to asking stupid questions right sometimes stupid they're very important questions as like how does the project that we're working on right now relate to the group or the business goals yep and does it I was at a company once where we rolled some of these things out and a few Engineers quietly back chn came to me and they're like I don't think this aligns to any of the key you know the exec okrs what am I doing and so we were able to have a discussion around it does it's just worded differently and you're not only impacting your line of business you're impacting three others or yeah I don't see how it aligns at all you should chat with your manager because this might be a good opportunity for prioritization yeah um so psychological safety is one a very related one is uh getting curious being very open to asking questions and learning more and I especially see this in large companies being open to the possibility and frankly the the high likelihood that the way you do things now is not the best way to do things just because something was difficult or impossible to solve 5 years ago doesn't mean it hasn't been solved and so when you open yourself up to the possibility that there could be better ways to do things there may be other teams doing better than you learning then you can learn what that is it it offers an opportunity for improvement right like I've worked with teams uh kind of in larger older orgs where I came in this was a problem and they're like oh well but that that's not one we're going to look at because like that just can't be solved because last time they really put their heads down to try to solve it it was seven or eight years ago and it's actually been solved now like I can point you to conference talks right and so being willing to consider and ask several questions ends up being super important and then tactically like onboarding onboarding ends up being a great indicator right how long does it take a new hire to commit code um and how can we improve on boarding practices and environments because it often puts a spotlight on things that are really really difficult I was working with one really large or once where they're onboarding time for a brand new College higher engineer was the same as a senior developer who just switched divisions in the company because the tooling changes the documentation changes the contexts they needed to get wow was basically non-existent those two things shouldn't be the same just just to be clear it was a long time right we weren't talking a day it it was probably like we were talking several months wow okay that's okay that that that's a real unexpected I mean that that's just terrible let's say it how it some of the best teams were at 60 days wow which not great right it's bad it's bad it's bad and when we worked with them on some general interventions right what are some of the concrete things they could do to improve it there were basic things like improving documentation improving how easy it is to find documentation um creating a dummy exercise for a dummy poll request very very early that has has to be completed within the first week or two now what we found is that even if if you uh no so some research on this has been done at Microsoft my colleague Brian ha did a bunch of this which is really incredible and shout out to the teams that you know partnered with us on this even something like a a dummy poll request right not dummy but like fix a title right ends up being hugely impactful it increases productivity some measures of uh easily automated measures of productivity um through the whole rest of the year right there was like a 30 to 50% increase because you've gone through the system now people push back they're like what if you're gaming the system great this is the best way to game the system because then someone has gone through every single step of the process they are now more familiar with it they've done it early on something that was fine but like kind of didn't matter like they didn't have to have full context of the entire code base to get familiar with the tool chain and surface this is just so interesting it was great and they could surface obvious problems that other people uh I when I joined Microsoft I joked cuz some people just knew where every body was buried they knew where every doc was they knew where every everything was but it's because they'd been there for 14 years and they had a list of bookmarks bookmarks are not transferable right uh now I mean the same was true at IBM the same ends up being true at many large companies and even small companies because if there's just a handful of you you can ping somebody and ask yeah when I used to work at at Uber I had a a dock of like the things to do and I think most people had after for a while cuz there's so many systems but what what you said is just so fascinating especially about the onboarding cuz I didn't think you know as we were talking developer productivity I thought about a lot of things and I think a lot of people are like this they don't think about how onboarding can predict but as you said it's not just the onboarding but the internal transfer and one super interesting thing that might tie back to this is this a famous thing a famous law that a lot of engineering managers will quote is Brook's law which says for late project if you add more people it's going to be even more late but this is true at most companies where onboarding takes a long time you know like the project is like estimate is two months if you add like three more people they'll take a month on board and suck time away Etc but if you're working at a company and you know like a company that's kind of known for super fast onboarding um externally at least is meta uh new hires will often you know push something a code on day one day two they have monor repo if you move teams you will probably work on the same code base if you have a organiz a where changing teams means that the new the the person who changed teams in a day they will push code to a different part of a stack Brooks law might not entirely apply assuming you can you know pick up parallel tasks which a lot of people don't realize because I I once had this experience at a company where this was actually at Uber where we had a people said like here if you get more people on your team can you get a dumb faster at first I was a Brook's law but my director said like no no no like those people they're onboarded like they they know it's the monor on mobile monitor they can do productive on day one and we added more people and we got them faster and later I was thinking how did we because this is it 50 years ago it was Co 50 years ago but as as you said I think this is the part where you need to be open to things potentially changing around you for the better and that's where the question kind of becomes when someone's onboarding to a team are they onboarding to the entire system and the entire environment in which they work or they onboarding to a slightly new team with like maybe a slightly different culture and learning a different part of the code base but everything else remains unchanged if that's the case you know maybe not uh similar productivity on day one but you know a handful of days to really dig into the new new part of the codebase new functionality but everything else is just done whereas sometimes even if you're within the same company you're onboarding to an entirely new system you have maybe the same tool chain but it's configured entirely differently so it's a basically a different tool chain you have different request processes you have different documentation if everything's changed that's onboarding that's new higher onboarding they just already have their login yeah related to this a culture turnarounds a lot of companies that you will turn to experts like yourself and and seek out different methodologies know that we've got a problem we're not very productive we look at our competitors other companies they seem to be getting stuff a lot lot faster we want to turn around the culture we want to go from a you know like do whatever it needs to be done have you seen successful turnarounds and and if so how did they happen and over what time frame and you know let's talk about like companies that are like like you know like not not not not a tiny startup but like company that like 100 plus people or or even bigger and and more interested in like kind of the the generic learnings because what I see is you know people at at at the they think like oh yeah we'll just hire some Consultants you know we'll become a tech first company in in no time I've actually worked at a company like this a long time ago financial company that kind of advertis itself and on the ground developers are you just rolling their eyes like okay yeah we're getting you know another agile training or whatever that is and you feel not much is changing and there's usually a disconnect between the two and and there are some companies that over time by the way like they do manage to become a lot more Tech first companies like old school companies who you'd be surprised you know like a lot of banks are are here right you I've worked with a lot of banks that have kind of undergone that transformation successfully yeah part of the government for for the ones that succeed what do you usually see in terms of success the time it takes and and what do engineers feel on on the ground you're right I end up talking to a lot of folks about it I think it's important to realize that if you're trying to change the tech culture that's not going to be possible and by that I mean you need to think about changing the culture of the company or the culture of the organization you can do things to have it and help it influence the technology culture but it's really difficult to try to just carve out the technology culture and change the way people think about software without changing the broader culture and I will say one you know I think notable cultural transformation is of Microsoft right SAA Adella has has really changed what Microsoft is like compared to you know the Balmer years or I I used to work at the Balmer time it was very different oh my gosh it's very very different now there are a handful of things you can do uh in general and then specific to Tech I'll probably focus on the specific to Tech ones because I'm sure other people are familiar with you know communication and commitment and uh those types of things when we think about technology I think we have additional opportunities because there we can look at uh some previous uh research that's been done John shook did some great work we used to think that to change a culture you would change uh or or to change the way people do work you would change the culture you would change the way you talk about it and then that would just trickle down into the way people build tools and and and you know kind of execute their work what he found is that when you change the way people do their work that bubbles up to cultural impacts and I think that's where having an environment where these changes are allowed and supported is super important right from that broader environment um but you know as an example you know people talk about new me all the time but I the cart Factory but I love talking about it kind of situating it in companies if you're in a company or in an organization or a team or a situation where your tool chain is just incredibly slow it's very high friction there's a lot of process to go through you can talk about doing agile and it's going to help to a point right people can get their mind wrap their minds around it they can think about you know what it means but actually changing the core culture isn't going to happen if you're working in systems that are incredibly slow yeah now if you introduce new tools and new ways of working that are actually much faster that actually make it easier to communicate with others that make it easier to surface and fix problems early that that make any prioritization decisions that you're talking about fairly easy to get done now you've changed people's lived experience you've changed what it feels like to do the work and that's paired with larger Communications larger education larger you know commitments to what is happening now commitments is also important because are you committing to improving the difficult pieces of the technical stack in the workflow to get to the desired outcome are you investing in the communication that's required right you have to say things at least three times for it to kind of land and it needs to come from several areas and several levels of the business and so I think those two together can be incredibly powerful but also needs to be explicit and and and just being a little bit more specific uh say I'm either a a staff and Senior plus engineer senior staff engineer principal engineer who joined a tech a company or an engineering manager a senior engineering manager not a C Level right so I I can't set the things but in that sense what is it that I could do like I'm I've kind of boed I took I took it in I didn't rush to you know like copy paste whatever I saw the previous place you know this is a frequent criticism of big Tech employees joining startup braing process but we're we didn't do that mistake I took it all in I realized there are things that are just really inefficient uh I can see you know if I was a CTO I I could do stuff but obviously I I cannot do that what are ways that I can improve the engineering culture the the efficiency and is is it going to be what what you just said which is around the lines of starting to change my team's lived experience for example and hope it bubbles up yeah I I think it's going to be a mix right just on a smaller scale right so what is within the scope of your control and it'll start by I I love that you said you know it's it's when you've joined you know you've done your first 90 days you've done a listening tour you've been able to kind of identify what some of the challenges are it can be super important to then you know write up a summary run it past one or two people to make sure like it kind of makes sense yeah sanity check meet with the teams say this is what I'm observing this is what I'm hearing from you does this resonate does this sound right is there anything I'm missing and then end that without a big call to action but say I would like for us to look for opportunities to improve this and I would love for you to be involved if that's something that you're excited about give people you know a little bit of time to kind of like marinate in that kind of Digest and then go back and say what are some ways that we can improve this these one or two things really stood out to me what does that what does that mean for you and then depending on this is where some resourcing comes in right because we want to change the way that people work and so that could be creating new education if it's just about you know changing a process it can be changing it can be improving the tech and so we can have discussions about either you know kind of carving people off or maybe setting up a hack day sometimes a hack day can be great to just deal with paper cuts yeah and that already the thing that's great about that is you get this quick win it's visible it's a quick win it impacts people so they see it and if it doesn't impact me personally maybe it impacts my co-workers so I'm at least seeing progress and then we have continued communication right again got to what you said which is you know talk with people and basically make allies because when you started a new company you have no real Social Capital Beyond maybe a fancy title or you know if you have some impressive credentials but not there and I think what people forget a lot of times cuz I've seen so many times where someone on paper you know at a high comes in at a high level tries to make changes there's a push back because you know like like we just want to do our work like right like what is this person trying to do and then they eventually they they they leave eventually disappointed that this company doesn't want to change but what they miss doing is first of all not realizing it takes a longer time you do want to you you do want to build a team who who supports you you understand them they understand you and now a team and then later another team joins and uh do this so I I I I I feel like you've kind of conveyed this really important thing and the fact that it's also not a Sprint in a Sprint you I mean you can do something but it'll probably be washed away with like a sand castle on a beach right yep um and I will say I have found uh having aligned it Ines ends up being important as well right if you say something's important acknowledge it and and follow up with actually being important so if you're just on the team sometimes that can be a little tough if you're just an engineer like a like a an IC you just started out of school right um but you can thank people you can highlight what they've done you can send a thank you email and CC their manager and as a manager you can say this is something that I find very important and we decided together is important any work that's done here even if it's not uh Feature work is going to be valued and then reflect that in any uh performance reports or one-on ones because it can be really discouraging to to go along with like some transformation effort or or fix effort or whatever and then it just kind of to your point right it just kind of washes away it just goes away so so far everything we talked about might have might as well been two years before because you know before chat DBT came out before any of the AI tools when mainstream but these days software Engineers are are using AI tools coding ID assistance and and many others it just it is changing software engineering as a whole how has it changed your thinking about developer productivity and and how how we should think of developer productivity and developer experience with AI I'm getting so many questions now about how can we improve the way developers work how can we make sure that we don't you know detract from their work does this impact space or D metrics um so you know quickly say dor metrics in general won't change right like their utility I expect to remain about the same because we're looking at how well that kind of outer loop uh is running in terms of space I think it's still incredibly applicable right because we want to know how satisfied people are with the development experience we want to know about performance outcomes like quality and security we want to know about how many things they were able to get done um the way they communicate I think efficiency and flow is also super important because it's changing the way we do work right when we're writing code we're writing code but we're also now reviewing a lot more code than you know it's almost a review exercise instead of just purely writing yeah um there may be opportunities I think to expand space right because trust ends up being a much larger Factor uh to everyone right so I I studied uh trust with s admin and people doing you know High consequence highly uh complex work and and trusting a system and trusting the outcome of a command and a new tool was incredibly important developers didn't see that as much because you know we go through school we use an IDE we use a compiler right does the compiler work like do are people people are not asking does the compiler work and I understand like the underlying math is very different but suddenly you've introduced this question to an everyday all day workflow is this the right output can I trust what I'm seeing um so I think it's good to kind of capture that and watch now what does that mean kind of more broadly I think there and we're seeing a lot of opportunities to kind of reinvent development in software engineering right we're seeing it in the IDE right now we're seeing it in in tests we're you know we're asking questions around code quality there was a great study done with GitHub and uh Office of the chief Economist here that found that four developers that had been seasoned developers right they have at least five years of experience they were finding that you know they're saying that the code is easier to read they have good approval ratings they're seeing uh over a 50% increase in unit tests getting passed so it can have some really good and this is just from devs using you know tools like co-pilot or or other AI coding assistants yep yep it's just from interesting and and these were the reviewers saying that you know the the PRS that are coming in are just seem like nicer to work withh um and you know you can have a a PR review cycle that is done you know we're used lters why not have an AI take a look and offer some suggestions you know that doesn't mean approved I I think there's already like like startups that are building it but I cannot see this not being the case feedack Loops right right some folks it's not quite on their radar though because we are so many times when we think about a developer we think about someone like sitting there typing right like it's hackers or something right it's very IDE focused but there's so much more to the development tool chain right yeah um one thing that my team and I are focused on that I'm like real kind of bullish about is how do we think about the rest of the software process how do we think about roles like release engineering and deployment because historically that's been very difficult and someone something that a person makes a decision on and it's basically a risk-based decision and they have to take data from the builds from the documentation from the downstream system from so many different places and make a decision this is an opportunity for l LMS to really jump in right because the person is still making the decision but instead instead of spending days coating all of this data and trying to summarize it you can instead get a quick summary with links out to what you want to look at so you can confirm or see if there's any like nudge or Edge case or something and then you can make a more informed decision so I'm personally excited about several ways that that AI can kind of help bolster and support and you know reinvent and disrupt right the way we do the full engineering life cycle earlier you you told me that some of the most productive Engineers uh that that you or teams but but also Engineers you you've seen are curious but also they're they don't hold on strongly to beliefs they're open to things changing and you gave the example of like you know like a team tried an approach 8 years ago and it failed but now it's actually it was possible do I correctly that it's probably a smart approach for a lot of us Engineers to just kind of put away a lot of our reservations because llms are are just fundamentally changing a lot of things they'll they'll they'll turn out to do some things wonderfully some things will they'll just crash and burn but unless we do this we'll probably be that team that kind of close itself down to to how things are is is is this the kind of advice you give to uh Engineers or you know what what what would you tell people CU because there are some experienced Engineers they've been around the block and there's some people are skeptical like okay they're they're just a fancy autocomplete we're giving them too much credit here it's not going to change what we do at the fundamental level or there are concerns that it will introduce errors or security bugs or other types of things that like I would never do or te would do right um yeah I I would say if if you're in an organization that is you know deploying or allowing uh tools like co-pilot and tab9 and cursor um really take it for a spin see what works see what resonates see what lands if you if you don't like it don't use it right if you only like it for specific types of use cases and specific types of tasks leverage it for the things that it is good at and don't use it for the things that don't fit into your workflow that don't fit into your mental model but give yourself a little bit of time to open up what that mental model is because it can shift and it can shift in ways that are very very valuable right it can it can open up time you know now you don't have to do like all of the setup in your code you can just focus on um really solving the problem really looking externally really trying to figure out where the blocker is or what the longer term Architectural Components should be now when I talk with developers and also what I'm building with with uh AI I I see two types of usages one is you have just specific talking about the coding so you know I know what the problem is I know what I want to do and I need now need to do the work one of is you have the inline AI assistance uh which you know Show automatic code completion may this be co-pilot a cursor or or some other ones and there's a and you know it kind of helps you move faster as as you go it continuously gives you that additional context but it also kind of takes away your attention a little bit as soon as you type it it shows up things an interesting thing that I I talk with uh with a developer who's actually building a game he uses AI very differently uh he says he actually goes to one of these tools chat GPT or Claude or or one of the others and he goes there with like a skeleton code pastes it in with a Todo and a generates and I asked like hey why aren't you aren't you using it in your ID and he said like I actually I find that I can focus way more when I I don't have this thing interrupting me all the time but I'm like no here's I need help and I I want to get your feedback and I'm iterating with you versus going I'm in my flow state are you aware of either some research or even just personal observations on how developers can preserve flow is it important or or can you actually have a flow State either way is it something that everyone needs to figure out for themselves um I just want to say that story you started with I think is incredible because someone played around with AI and they figured out the workflow that makes sense for them that works for them right it doesn't have to be you know line completions inside the IDE it could be writing up a chunk of code and then asking it for review right um I will say anecdotally I've seen folks sometimes adopt that type of workflow or they'll they'll turn off autocomplete or Auto suggestions you know as they're typing I haven't seen any hard date on that although I I wouldn't be surprised if there's some coming out in terms of flow um there are a few things that we've discovered and notied so there's a paper that some colleagues of mine did here in MSR around uh they use the cups model so it's how does how does coding change right you write something you wait for feedback you have to reassess it you have to decide what's coming next um and some of that work and early work we did with the GitHub team even influenced how the timing of suggestions happens right we ended up slowing some down because as you pointed out if you're getting like suggestion after suggestion after suggestion it ends up being an interruption um when I think about you know your question around how do we think about flow I think everyone will kind of need to play around with this and see what works for them and what that means right because when you're coding in an IDE I know sometimes I'll turn it off because I just want to code right I did that this weekend because I wanted to play around with some code um and I wanted to be kind of driving and and doing all of the writing but that also is I think one helpful way to think about it is imagine a world we're here right where there's just a different way to do the work and maybe that means you know you call it something else in your head right maybe instead of coding maybe I'm driving and reviewing or or I'm guiding right because if we can think about it differently then we're not comparing it to something that's not the same thing and we can find what that new workflow is I I like it I also like your your initial suggestion of driving versus guiding it is just very different but I I I do think that a lot of us Engineers just need to accept that it's different it will change you we'll look back this 10 years later people will look back people are starting their career now they'll be like oh yeah you know like a little bit how anyone who's a coder it's like yeah stack Overflow it always existed I mean interesting enough now we've kind of stopped using it but there was a time where it was just a a given and there were people before and people after so I think we're we're we're going to have that it's it's it's it's a good good entirely reminder and also we're just in beginning of this right like this whole thing it's just wild how how much it's G in such a short time I I don't think there's been any technology in history of software engineering that has spread across the whole industry so quickly and and it's changing how uh how people work changing their productivity their you know their what what what what what they're doing and and still work in progress yeah absolutely I I totally agree right when I think about you know big impactful uh technology shifts in Tech cloud is one for sure but it didn't hit almost the entire industry within a year right like and specifically like llms right to to the AI folks out there AI has been around for years anytime you're doing any type of autocomplete in an email or you know intell code this is all AI uh I think llms really changed you know these these Foundation models really changed uh the way we can do work and the types of development we can do so closing off with some rapid questions I I'll just ask and and you fire off whatever comes to mind the first one is about coding full-time versus being a researcher you were a software engineer and now you're you're a researcher what is the the one thing that you missed from coding full-time I miss just like hacking something up and being in the code there's the thing I love about research is that it's about identifying and solving problems that are hard to solve where we don't know the answer um coding is very similar and I I love that I kind of miss the thought process of figuring out the best way to write the code uh what's the best way to structure the code how do I want to like do some of the work right like joke like do the type right I I miss those parts of coding um and luckily I kind of shifted into something where it still kind of scratches that itch of where are the big problems what's the strategy I want to adopt right there's strategy in writing code there's strategy in uh Communications and product Direction and and research uh research program but even if you're not coding you're working with people who do code yes and I like I said I still play around with it but but missing I think a little bit also is like the rush of writing something for customers that I know is going to be a production system um I had Folks at IBM a couple years ago contact me and say that one of the systems I wrote there as an intern was still being used and I was like awesome first of all sorry second of all it's there's just a different kind of excitement around writing code and seeing it deployed and pushed and shipped to customers right which luckily I get a little bit with some of my research I'm realizing now as I tell you this like I've I've found another proxy I've found a way to kind of duplicate some of my coding just another discipline well and you still code a little bit so yeah what is the next book you're working on if you can tell us yeah uh right now I'm working on a developer experience book um and it's kind of geared toward toward folks who want to think about you know improving developer experience they want to learn more about developer experience could be uh an IC or or an engineering manager it could be leadership so we cover things like um I'm writing this with ABI Nota so we're covering things like uh what type of data should we be thinking about how should we communicate data um how should we communicate like the ROI right a lot of this is communication um how do we want to think about a long-term analysis and data collection kind of uh program uh this is not a stats book but you know getting people pointed in the right direction right examples of survey questions examples of types of data examples of ways to operationalize because um my kind of puristic is once once I've had like at least 100 people ask me a question that ends up being in the same flavor of answer right it's very very similar something written something like a book can be really helpful because then everyone can kind of cover Basics and then think of the next level questions that they're going to get to eventually but it's not a good use of of time if I'm a blocker if we're trying to do something over an hour what's a practice or a tool that helps you personally be more productive on the day-to-day basis dayto day so day today at work um I don't want to say a tool um I've got an incredible PM who has absolutely uplevel all of the work that we're doing uh Jason in shout out um having a very good PM TPM could just be a game changer I had an incredible one at Google too um was just so so good um but when I think about technology tools um not in my work life because I kind of maintain this I for sure maintain like a security boundary uh Claude claude's incredible I'm able to use Claude to do many many things and that I would say has been the most notable impact it it is very interesting how the the competition between llms it's just so great because you know open AI obviously started all of this and now there's anthropic clot sonnet uh in in many ways you know like jumping ahead but but I feel it's just so great for the ecosystem there's so much competition everywhere and they're all inspiring one another to become just so so much better so I I feel like as a someone using all of these things we're just in the middle of some of the best time in the sense of like getting all of this these things for free because you know these teams want to like outclass one another and we're just benefiting from it and you know like you said it's so early that everyone is still learning and I'll jump back and forth because some models are you know better for something else but just even having someone to like change every month right yeah it can change all the time but having like a a rubber duck that responds to some of your questions sometimes I just want the rubber duck but sometimes I really want to think through something like what are my limitations what am I missing what are the holes okay no that's not actually a hole what else would it be ends up being really useful and it's super super fast and what are are one or two books that you read and would recommend it's Tech related um Inspire by Marty Kagan is great I'm still making my way through parts of it because it is not a small book yeah it's it over there um it's it's really kind of solidified and been and articulated things that I've kind of known but like in a very fuzzy way and it's also like introducing some new things for me to think about which I really appreciate awesome um another one that I loved uh that was just kind of like practical I just like to dive into like random things um but outlive by Peter AA was just I like to N I want to like nerd out and learn something which you know my partner jokes is like I heard a lot lot of good things from about petera from from people working in Tech so well and for a fun book I always love Enders Game it's like an oldie but goodie so if I just want to like Escape I'll just grab like a a trashy beachy read or I'll grab like Ender Game or Ready Player one or something CU it's just it's just a fun story that I can just race through there's such fun stories I I feel the movies were were fine as well but nothing beaks the book yeah and then like for for news and and articles tresy McMillan cotton and an Helen Peterson are probably the most high value reads that I get outside of tech awesome well Nicole thank you so much for being on this podcast this has been so fascinating interesting and I I think just like motivating both on how much we've come with developer productivity and also just how just just when we're getting closer to figuring things how things change again with with with AI and you know teams behaving differently it's an exciting time yeah it's it's really fun this is a great time to be working especially in Tech thank you to Nicole for this deep dive into the ever evolving field of developer productivity you can read more from Nicole on her website or follow her on social media both Linked In the show notes Below in the pragmatic inine we've done several deep dies on developer productivity check these out Linked In the show notes below or by searching developer productivity in the pragmatic engineer newsletter if you enjoy this podcast please do subscribe on your favorite podcast platform and on YouTube thanks and see you in the next one