Last DevRel standing

Welcome to changelog and friends a weekly talk show about dev rel and zerp a massive thank you to our friends over at fly over 3 million apps have launched on fly including us and you can too learn more at fly.io okay let's do this what's up friends i'm here with a new friend of mine jasmine casas product manager at century she's been doing some amazing work her and her teams over many years being at century and her latest thing is just awesome user feedback you can now enable a widget on the front end of your website powered by century that captures

user feedback jasmine tell me about this feature um well i'm jasmine i am a product manager at century and i'm approaching my three-year anniversary so um i've spent a lot of time here i work on various different customer facing products more recently i've been focused on this user feedback widget feature but i've also worked on session replay in our dashboards product with user feedback i am particularly excited about that we launched that a few weeks ago essentially what it allows you to do is it makes it very easy to connect the developer to the end user your customer so you can immediately hear from your basically who you're building for for your audience and you can get basically have a good understanding of a wide range of bugs so century automatically detects things like performance problems and exceptions but there are other bugs that can happen on your website such as broken links or a typo or permission problem and that is where the user feedback widget comes in and it captures that additional 20 of bugs that may not be automatically captured i think that's why it's so special and what takes it a step level above these other feedback tools and these support tools that you see is that when you get those feedback messages they're connected to century's rich debugging context and telemetry because often i've seen it myself i read a lot of user feedback messages are cryptic but they're not descriptive enough to really understand the problem the user is facing so what's great about user feedback is we connect it to our replay product which essentially basically shows what the user was doing at that moment in time right before reporting that bug and we also connect it to things such as screenshots so we allow we created the capability for a user to upload a screenshot so they can highlight something specific on the page that they're referring to so it kind of removes the guesswork for what exactly is this

feedback submission or bug report referring to now i don't know about you but i have wanted something like this on the front end pretty much since forever and the fact that it ties into session replay ties into all your tracing ties into all of the things that sentry does to make you a better developer and to make your application more performant and amazing it's just amazing you can learn more by going to century.io that's s-e-n-t-r-y.io and when you get there go to the product tab and click on user feedback that will take you to the landing page for user feedback dive in learn all you can use our code changelog to get 100 bucks off a team plan for free now what she didn't mention was that user feedback is given to everyone so if you have a century account you have user feedback so go and use it if you're already a user go and get it on your front end and if you're not a user well then hey use the code changelog get 100 bucks off a team plan for three-ish months almost four months once again sentry.io all right we're here with our old friend multi-time recurring guest too many times to count i don't know swix three four seven eleven times on the pot i'm not sure but you're back it's been a little while good to have you swix welcome back

thanks good to be back i've been always a loyal listener and it's just an honor to be invited on every single time like it never gets old well we love your enthusiasm and your

availability you know i i can hop on with you on monday and say hey you want to come on the pod tomorrow and you're like let's rock and roll so it just means i you know i keep myself

relatively free no it means i currently do not have a real job that that's what it means it means i don't have i i did move a meeting uh but that's that's just that's just because it's easy for me to move stuff because i i could i'm my own boss now effectively remind me where you're at in the world again yes uh so i i am now no longer uh having a real job i run that's i always call it two and a half companies it's the latent space podcast and newsletter sort of media empire which you can talk about later the ai engineer conference uh which just finished two weeks ago and i also have my own venture-backed company small ai which i'm working on with a couple engineers so like yeah it's hard to describe because i don't work at a regular employer i kind of split my time between three business ventures but uh that's just that's just how my attention is uh you know is spent i think each of them independently have different time horizons of success and you know hopefully uh they all have a common theme of like me being a prime mover among the engineering field how do you make money so the thing that actually currently makes money is the conference that i run we have now run successfully run the 2000 person conference for the for the first time ever and uh you know same deal with most conferences you sell tickets and sell sponsorships you pre-commit to a whole bunch of expenses up front and then you freak out for three months uh you know hoping hoping that you sell sell enough tickets to cover your costs and then and then we do and we we make some money back on top of it and you know that's the profit well happy to hear that you're running in the black

there a lot of a lot of folks run conferences in the red or very very near the red so that's

awesome yeah we had we had a lot of help because andre karpathy basically tweeted about us and uh you know we immediately sold out like an hour later so i think like for me it's a long-term game of like i want to build like the cube con equivalents like the industry like the definitive industry conference for ai engineering which is the thing that i've decided to sort of place all my chips on so we don't have to talk about that this time but you know i think it's relevant to dev rel in the sense that when i was a developer advocate or dev rel person i spoke at conferences and i on a couple times we actually even organized company conferences like that's the pretty much the peak of what you do as a dev rel and i think for me like i think any dev rel who like believes their own bs enough should actually go out on their own and and do this as a as an independent business venture because it is worth much more to dozens and maybe hundreds of companies than it is

to a single company yeah that's interesting well i remember last time you're on the pod you we had just experienced the chat gpt moment i think it was probably a month or so after that and you said like this is it i'm going all in on this i'm learning right you were writing what you do which is one of the reasons why we invite you on because you learn in public and we learn from you and with you and you had made multiple transitions you know kind of front end stuff dev rel you were thinking back end for a while i know you're at temporal doing workflows and back ends and then it's like all right here's where i'm going to really dive in and that was a while ago so definitely want to catch up with you on that stuff but the reason why you caught my eye this time around was a post about dev rel and about the zero interest rate phenomenon which it seems like we're learning now post zerp that a lot of things that we were living with and thinking they were normal perhaps were not so normal they were kind of bubbly or frothy or maybe symptoms of or side effects of all of this cheap money which was in our industry and that quickly left our industry when the macroeconomic situation changed you have a post which we covered in news just the other day yesterday as a record but a few days back as we ship dev rel's death as zero interest rate phenomenon where you ask and answer the same question is dev rel dead and i have heard a few whispers of this like okay is dev rel dead or dying or what's going on with dev rel as a as a thing and so that's the opening of the can swix take it where you want first of all you say no but maybe why is it not dead i'll start there i think to claim something

is dead means like there's no longer demand for a role like this like like this and it's objectively not true i have friends who are desperately trying to hire dev rel with full knowledge of all its faults because they are close friends of mine and i have complained to them about the the failings of dev rel so full knowledge that like a lot of dev rel is completely ineffective a lot of people are really bad at dev rel they still have the job full knowledge of all that they still need it right so you cannot say that the role is dead if people just really demand it still right but it is uh people the the fact you know just like with any technology the moment people start asking is it dead it's not quite dead but it's you know it's less cool it's not a good sign right redux isn't dead but people have been asking is it dead yeah you know like it's it's it's it's declined and i tried to quantify it right i think i maybe it's my approach is like the first time anyone's actually tried to say like oh yeah you know it's not dead but you know how much of it has died and my number is 30 percent it's you know over over zerp it increased 200 and then it declines 30 since the peak and where do you get that how'd you quantify google trends that most objective amount of data a good proxy for perhaps

being truthful right in terms of search like what's the trend yeah search trends yeah but

also you know and that seems to coincidentally line up with the common room industry survey of dev rels where about 26 percent of them have said that they've been involved in layoffs and anecdotally we've seen a bunch of different layoffs including my old company netlify including planet scale including a bunch of other companies out there off zero as well and these are layoffs without replacement so like straight up we no longer have dev rel and that is one form of dev rel being dead right companies that used to heavily invest in dev rel i actually i'll put this on record because i couldn't find an authoritative source so i'll be the authoritative source microsoft 2018 to 19 hired something like 200 dev rels they call them cloud developer advocates some of the top names in our in our industry spent really a lot of money gathering all of them and like two years later half of them are gone you know it's like not polite to talk about power politics within microsoft but there was definitely a big power struggle in there trying to build a dev rel org and not succeeding i think that was a like a really early precursor because that wasn't served right 2018 19 wasn't served but uh it was a pretty pretty early precursor and people having very inflated expectation of what they could do for a business throwing a whole bunch of money at it and then realizing that just money alone and just like number of warm bodies alone doesn't actually solve it like you actually have to have taste and you know a clear message and a working system that you know that it scales uh healthily and

effectively and that's excited to build yeah there's a lot of facets to this and one of which you mentioned maybe offhand that i think about a lot i think adam thinks about it a lot because we talk to a lot of dev rels of course dev rels would love to be on our shows and all this kind of stuff right and there's a fine line between a dev rel and a shill you know and like i think there's a big difference between good dev rel and bad dev rel at least from where we're seated like we can just like see through certain things and then other people were like yeah we'd love to have you on you know and it's like there's a big difference between the two it's it's clear and i wonder we're like do you think first of all you seem to agree with that that's generally a true sentiment sure yeah absolutely and then so secondly it's like when it comes time to die but then also have life right like there's still value in the position it seems like and maybe this is just like staying the obvious or shallow but it's like the good ones are going to stick around and the bad ones are the ones you know no offense to any individuals but the ones who aren't good at what they do they won't provide much value in the first place right

yeah there's there's some amount of that but also inherent in the job it's a high burnout job regardless of the macro in a sense that it's mostly like a mid-career job you know there's there's very very few chief dev rel officers right there's some cdxos out there but like there's a career path to like vp dev rel in most companies like it's just uh it's a job that you that you that you sign on stepping stone it's a stepping stone or you've just decided to opt out of the rat race altogether and all you want to do is make content or a lifestyle right it's a lifestyle business and and there's and travel yeah uh well for most people professional travel actually starts becoming a drag like you it's nice to see your friends every once a quarter or something but professional travel uh actually is a drag like nobody actually really wants to do that in that role unless you actually just love travel but like that last like that's what my point is

i've met a few and maybe they're just saying this but it's like they just love traveling yeah

check back with them after like two years of it yeah you know definitely yeah over time that gets

old for sure you know there's only like even like you go to sedona or some special place for vacation like man i want to buy a house here which is how i felt when i went to sedona i was like man i want to live here but if i lived in sedona and i lived in orlando florida for a while too and it's like after a while orlando is just orlando now yeah i can go to universal studios anytime i want but like do i i went there like twice right maybe three times over a three year span like it's just not vacation places it loses its luster exactly it loses its luster in terms of like well i don't think this two years of travel constantly is really the game steve clapnick i think is probably the best example that i've known of before you know well before even any of the zerp maybe it was even like maybe it was zerp i don't know it was for your money maybe that was part of it but not this phenomenon we're speaking of when it comes to like covid and like literally lots of free money out there this more recent occurrence of it steve clavnick traveled i think so much and he was outspoken about this way way back in the day and just got burnt burnt out bad on it right because it's just you're not built for it time zones travel you never know where you're at in the world it's like you're constantly in a different time zone your body your circadian

rhythm can't even keep up with it yeah so you know i want to preface this with i think maybe some people's impression of devrel is like it's it's a lot of travel that that number has definitely come down a lot uh part of it is cost cutting part of it is environmental concerns and part of it is people just don't want to travel that much so actually when i say people burn out of the job it's actually not only that it probably is not even majority travel it's actually just the the grind of constantly dealing with people who are new to a technology so this i call this the eternal september effect right that there will always be more beginners so you're actually if you care about for example viewership numbers you're always writing the 101 level intro to whatever and like that is your life for like for as long as you want to do devrel uh because because that's the highest tab content that that's possible right uh the best intro to something that's that's your the sort of pinnacle of your success is to write the best intro to something you know so a lot of people don't want to do that forever uh they want to they want to have more seniority and more impact in their in their work and impact being maybe financial impact rather than impact on the industry because devrel has does have impact on the industry uh and and they choose to move on so like people leave their realm not not only because they're not good at their jobs or they just aspire to something different uh and and that's normal as well like this this job uh way more than other jobs and startups has more churn inherently and that's that's normal i just want to establish that like it's not a judgment thing of like oh you sucked at it so it's like you know you had to leave yeah so so like this was a very overall this was a very tough piece to write because obviously a lot of my my friends are devrel like i obviously i had that job for a for a long time and i'm like a lot of people know me for for that and it comes comes with you for advice on that and actually like so this post the idea for this post uh was actually one calendar year ago i tweeted it on my sort of private alt account and i had to wait for everyone for more people to start saying it for me to be okay publishing what you could read today because me saying it too early would have pissed off a lot of my friends who had that job and and like i think for me to find the words that would accurately try to say what i was thinking without also like being too inflammatory right like i'm trying not to bite the hand that feeds me but i'm also saying like hey like let's call a spade a spade like there was a mania in devrel over this earth period and now everyone can see it let's put a end to it let's try to learn some lessons from it now

that it's okay to say it out loud what's up friends i'm here with michael abbens staff software developer at one password michael i want to talk about how one password helps secrets management secret sharing with teams with orgs with production even really easy even so much so that when you spin up a new person onto a team or you add new people to a team or you even if you take away people from a team you know you want to focus on access and you also want to focus on security so if you want to remove somebody you want to make sure they no longer have access to your secrets and then the flip side is when you add somebody to a team you want to make sure they have access to their secrets so help me understand how one password helps teams be efficient with

secrets management well so when you have a dev team storing their secrets in one password then it makes it really easy to bring new people on they already have so much to do and so many new things to learn you don't need them to have that extra burden of creating a dozen different accounts to be able to access your payments or your errors or your monitoring you can have that all set up for them already within one password and each of those particular services gets pulled into your local application your production application at runtime through the one password cli from there you have your new developer downloading the repo they don't have to spend time setting up their environment all of the different services are ready to go because they're already stored in one password all they have to do is use the one password cli to spin up and run the application and it's ready to go it's like developing in the future well we must be in

the future michael because we use one password just like that all of our team secrets all of our personal secrets all of our application secrets they're all in one password and we're using exactly this process to make our lives easier so friends go to one password dot com slash changelog pod they've given our listeners an exclusive extended free trial to any one password plan for 28 days it should be 14 days but no it's 28 days but make sure you go to one password dot com slash changelog pod to get that exclusive signup bonus or head to developer dot one password dot com of course to learn about one password's amazing developer tooling the one password cli one password for ssh and get cicd integrations service accounts and so much more once again one password dot com slash changelog pod this microsoft area you mentioned that was preserved and maybe even after it like i think one of the reasons maybe there's churn is that it's a hard even to this date kind of hard to define what exactly is dev rel and when you have a challenge defining what it is you have a challenge in defining what it should do what the function should do which maybe is the reason why people flunk out of the job or move along or churn because maybe even in the microsoft area that you referenced where they hired lots it was challenging to define what exactly are you trying to do because if the if the job primarily is focused on a function that sits between the company which usually is a tech product of sorts a sas a dev tool dev service etc maybe even an open source company a cause company or an open source project they can have dev rels as well if the function is to nurture the relationship between the company potentially the purchase of a product and the developer community there's a lot of ambiguity in there in terms of what you can do to be successful and it might be challenging even even as a manager to manage dev rels right like that's like what do you really do here what can you do here what is success in your role and when you have lack of clarity as an individual it's kind of hard to maintain what a good friend of mine michael burt coach michael burt says the prey drive right you have to have a reason it's your because goals because i'm a dev rel these are the things i do in my role or because i want to do these things within the developer community i have clarity when you have ambiguity in your role or lack of clarity it's kind of hard to kind of wake up every day and be motivated and get something done or when you're tweeting or doing social media or doing these things that aren't really seen by peers or adjacent peers like engineers or marketing or sales is like that person is dev rel and they're just tweeting like is that work so when you feel like you don't really you know mean like when you're not clear it's kind of hard to kind of just get up every day and just do what you do well unless you're a self-motivated person so i guess all that to say how much of this churn is because the management or definition of what the role is and what success is of the role

is well defined so they can be successful i would say so i worked at aws says dev rel i've never worked at microsoft but i do think you should have some faith in the big corps to really define roles before they hire for them because it's hard to open headcount in in these things and so they had their definition and you know i i think you have there's more security there just because like these these things are very very well uh defined at least internally for that stuff what's less defined is the startup side of things where you know i just got a a bunch of funding i'm gonna allocate you know one person out of my 15 person team to go be that that sort of public face for my company what is your job do whatever seems right and i had that job that was me at netlify i had no manager for a year it was fantastic it was also absolutely zerp because i just did whatever i felt like doing it was fantastic and then they you know eventually we got adult supervision with uh sarah drasner but uh but also like that i i thrived like the you know is that the most intrinsic motivational job i had just like figuring out the meta game and i think you know a part of it is with with marketing with with anything with um with people you know the true tau is a tau that cannot be taught and the true dev rel is that not the dev rel that can be written down uh and and the moment you try to write it down and try to systematize it the game has already moved you know six months ago right to like the new game right if if you think like here's the way to success and we're gonna scale this for the next five years tough luck like people move like people in trends and and you know move quicker than that so it can be a really tough thing to nail down uh that said i do have a piece that's just fairly fairly popular i'm told it's like required reading within google of uh measuring dev rel and i think basically you know the definition of any job you know just from the outset is that so you treat dev rel as a black box money goes in what comes out of it so i have three major buckets it's community it's content it's product and either you're producing in one of these areas or you're not at all and i think most uh expectations of dev rel is is like one of these three things and you know i can i can go into into those things further but like you should have some definitions of like what the visible output of dev rel should be from there and then that basically becomes the job and if it's not successfully captured in those three buckets then you are then you're probably doing a different job than than what most people think of as dev rel i think that's well said i think that

it is really tricky because i almost the more formal and described and delineated the role becomes you're probably on the lower end of the value chain in terms of like actually being able to execute on it well for the reasons that you just stated it's like you almost need this the formula that works really well is like the small startup and i think you talk about there there are companies who are doing really well with no dev rel or minimal dev rel but they all have like a charismatic leader or somebody who's already very online and very good at being online that just continually brings them more and more interest more and more community more and more relationships and that person's almost a unicorn to a certain extent like can you systematize what they do and then hand it to somebody else and say go and do this i'm a bit skeptical maybe it can work but i don't think it's going to work on a repeated basis into the hundreds of employees

right into the hundreds of employees on a smaller scale yes if we're talking about like the twillios of the world which i feel like they have successfully done that that is it becomes much less personality driven and much more about a repeatable process that can be scaled across every major city and it might be also a relic of maybe seven eight years ago when things were maybe more more more sort of in-person centric now that i think yeah online has has taken more and more and more and more of our mind share and time and you know remote working and all that the sort of online first dev rel meaning that it is just no location means that there's just there's just no there's no need for repetition and therefore more centralization in a single person and so so i think that's that's important to note i think the other thing also like i so i agree with this whole you know the the sort of low level junior beginner person thinks about it as like oh i will produce three blog posts a month or something the higher level think about think thought the thought process is i am persecuting a mission i'm promoting an idea and the output is three blog posts but those are downstream of me promoting the idea like i'm starting a movement you know for me like for me when you know when i when when i talk about the stuff that i do uh for latent space ai engineer conference is small ai i don't think about it as i organize a conference i think about it as i am starting my own industry and the visible output is i start a conference and so i i think people who think about it on a higher level have a more coordinated approach to their actions even though the actions still look still look the same they have a more cohesive outcome because there's a broader plan beyond that instead of the individual units and i i do think like for the for the junior dev also i i do some advising on on the side just for fun and for the beginners who come into the job like they very much do like the one-off like hey like we gotta get this launch right like we gotta we gotta max out this launch and we gotta like get all the retweets like make this the most awesome launch possible whereas for me it's not about the launch it's about the the long-running campaign from the past the launch through like whatever is next and i think getting people to see that whole journey i think is is is something that i think levels them up to the next tier and i think that's what the founders can uniquely do and you talk about these unicorn founders that's what the founders can uniquely do apart from the hired guns which is the founders you know started the company because of this this whole mission and being able to be to authentically tell that story i think is very rare i think uh maybe you know one possible positive example is lee rob at vercel it was successfully taken on that sort of voice of next

year's a great dev rel by the way when i was thinking about the good ones he was one that was in my head it's like i think the guy does a great job he actually prompted this this post on

the depth of dev rel because he he was like yeah like i've been thinking about this a lot because he and i chat offline quite a bit but yeah i mean like you know like it used to be gearmo primarily promoting next year's and vercel and now you know now i think lee rob is kind of number two in the company as at least as public facing figures go you know being be the voice of the company and i think that's that's absolutely like a rare success story i think it's it's very much a two-way street about the founder being able to trust uh whoever they hire and then the higher the the employee being able to rise up to the task and very often those those two things don't don't happen in either direction so that's what it is the last thing i want to point out as well in just in terms of like why people fail at the job or why measurements fail this is something i've been thinking about which i did not sufficiently capture in my measuring rail post which is that often dev rel just like any other content of media business is a hits driven business not a consistent output business i can like i i used to often say like i'll write 50 blog posts a year but it just one of the 50 will actually be the one that people remember and it's very very hard to run any business it's very hard to learn from any information or any market reaction when like most of the stuff you work on is going to flop and it doesn't mean that anything is wrong it means you should still keep going even though most of your stuff is flopping because it's the stuff that's going to produce that one hit and it's going to justify everything else right no i 100 agree any

hit space business is like that you you uh well you say okay well all i need is hits then i'm just going to oh yeah just just don't fail it's like yeah exactly like well see that doesn't you don't know why it's i mean even with this post swix you said i'm not sure why you guys invited me on or

i'm not sure why this one resonated with people there's many reasons why i'm not sure i don't even

know why but i was like oh this is interesting and uh well it hadn't talked to you in a while i could probably answer that question okay well i think that we talk to a lot of dev rels we have a lot of friends and fans and people we're fond of that are in this space that have i would just say tumultuous times right and so it's it's a hit because we don't want to see people or companies shrink that particular function size because that means friends of ours are out of work or they're changing what they do they're moving into adjacent roles or different roles maybe directly into marketing where dev rels sort of adjacent to it but sort of in a lot of cases under it sometimes even under product i think we care because it shows the health to some degree of our industry if one of the core functions is withering or failing or churning or not right i think it's an indicator of how healthy the market might be right i think that's why we care about this particular function so much because this is literally where product and the in the future potential buyer might be like one thing you reference in your measurement is the sean ellis question which essentially is how would you feel if you can no longer use the product and so if you ask this question to a free user a free tier user you say how would you feel if changelog was no longer a thing tomorrow would you be you know happy unhappy somewhat disappointed devastated very disappointed yeah devastated thank you swix i think that if we had a large majority of people saying devastated or very disappointed versus the other two or three options then that means that we've got some version of product market fit or we're very beloved and so we should find a way to exist or live if that were you know us on a deathbed jerk geez what this is terrible point is is that i think that we have a lot of people who are in that space we care about and any unhealthy measure in that in this particular space shows signs of an unhealthy market that's why we care that's why i care plus we're looking for answers

and explanations too right i mean we see things going on some of us talk about it some of us don't and it's tumultuous and it's scary and sad and then you're looking for answers you're looking

for like well what was going on and it's like well here's a post that surmises that it was this and now okay well that rings true i mean it rang true with me which is why i put it on changelog news and then i was like and i like the end of it like well given all this like what now or what what can we actually move forward because we know that it's not dead in so far as it's not a valueless thing there's huge value in having high quality developer relations around your product or service and all that that entails but the free money is gone which was allowing it to bubble and as we've discussed now what you know in that in that post zerp environment what does it mean for dev rails what does it mean for everybody else is it still a job that i should go out and seek you know is it not is it something that i you know like what are the now what so let's key in on that point and what are some of your thoughts being deep in this area of here we are 2024 halfway done who knows what's going to happen by the end of the year but i don't think we're going to get back to zero interest rates and by then we might see one maybe two cuts from the fed this year maybe not maybe zero but for folks who are either in dev rel currently or considering it or trying

to get back into it like what are your thoughts for them yeah yeah so i actually tried to leave solutions out of this post because it's been covered elsewhere and you know i haven't identified as dev rel for maybe a couple years now and there's there's a bunch of solutions out there i do think just just the street the straight job hasn't really changed i think what the removal of free money has led to is basically we can no longer get by with lack of accountability in dev rel it's probably a good thing it's probably something that we needed and so what i tried to do in the post is to list out the smells of what zerk dev rels looks like and so i tried to use that as a checklist for people in the industry of like if you were doing this there is no longer any appetite for this like it is no longer okay to do like let's just call it uh free tier dev rel for example like you only talk about how to use your company's free services and have blissfully zero knowledge of anything paid because that you know that doesn't serve the company's needs and also actually you know more to the point doesn't actually serve the customer very well because you don't know your product and there was a lot of free tier dev rel in zerf because it's easy to talk about something that you can adopt for free it's easy to get applause or something that's free the hard part of your it was really challenging your skills is saying like why you know your company's products are actually worth real money and people who obviously were successful at that were probably more valuable to the business anyway so yeah there's there's a lot of thoughts so i at the end of the post i linked to lee rob i linked to sam julian who used to be uh vp dev rel at off zero and i linked to myself as uh you know independent thoughts i think everyone's basically the common consensus let's just say is that dev rel moves into developer experience you know which is kind of a annoying rebrand that every industry likes to rebrand itself you know like in devops you know there's this sort of ongoing rebrand to platform engineering same thing for dev rel same thing for data science right specs data science we can talk about that uh after but i would argue not but okay save it save it we can save that for me you know i do think like basically there's a maturation dev rel that i'm looking for where you know you don't have the one size fits all dev rel that does the sort of full stack of production to publication to sort of idea generation and you have a sort of front middle of back office dev rel this is definitely for for more sort of scaled up organizations you know i was leading a team of nine uh at my previous company and you know i definitely saw that need to grow you know more structured process around dev rel and then i think understanding for people who choose the developer experience path understanding how you interact with the rest of product and engineering and having the buy-in to actually have features like for for lee like he straight up just became vp product like there was no we will coexist with products no like dev rel just took over product like that that's how they solved it um very very few other companies will actually let that happen because products usually has way more political power than dev rel that's just how it is and so dev rel then then gets shunted to marketing and then loses all power from there i i think i think like having dev rel become pms is the path that like i think i see the some of the really motivated people interested in impact do and i think i think that is that is the right way to do things but dev rel as a title is going to continue to exist as a primarily sort of marketing and community and docs and you know function much more than product just because product is its own beast it's a much more established industry by far and much more politically powerful and therefore a harder force to have any impact on i don't know

if anything i said is controversial leading product is tough that's a tough role what do you know about how things have changed for lee rob because i we've talked to him several times but i'm not familiar with the details of how the dev rel folded into product how did that actually play out how does that roll out now like you said dev rel took it over what does that mean

i mean so he he was promoted from dev rel to product uh so there is still dev rel at vercel it is just far far less visible than it used to be and probably for the better i don't know you know they basically just had attrition without replacement and that's just how the team sort of shifted its priorities i mean they needed a vp product and lee proved to himself and into the company that he was up to the task and i guess they promoted him i i can't really speak for his personal experience you know just because i only you know i only hear tangentially from like from him and other people but i don't hear the full story so you can talk to me about that that

was less on the specifics of his specifics but more like how they as an organization achieve that because leading product and leading dev rel is uniquely different but also not exactly far off like to build the best product you have to have a a connection with the people that you're building it for which is a function of dev rel right a connection to community but you also have to have business mindset like where do we actually make money where do our users really get joy where is our business trying to go not not just where's the product trying to go which sometimes is is similar or the same but not always and so i would i would not suspect it would be easy for a dev rel just to take that over unless they've got some prior leading product product management experience or have just they're just elite rob where they just like slay it you know i i don't yeah again

i not not not really speaking about his specifics but like i do think that if people care enough about developer experience then it basically is a shadow product team anyway this is something i've talked about again and again which is like kind of the existential problem with dev rel which is that you're supposed to be the voice of the user you're so you're it's supposed to be a two-way street you know you you spread the good word out that's the dev evangelist role and then and dev relations role takes that good word that you get the feedback from developers and brings it back into the company except most of the company doesn't want to hear it because they already have backlogs and you're just adding to the backlog and you're not welcome here go away so it's a really good dev experience person would prioritize and justify and go like you know here's here's what our developers are telling us just like listen to me like i i am i'm good at the people and i i understand what you're uh what you're shining a light on though is that friction

between dev rel's job and product's job yes right and so rather than fight the fight merge just take it over yeah right yeah this is becoming more close so i asked the question because that i was less trying to understand lee robb's personal specifics but more this function of because i think you kind of clarified it there where there's that friction point if you're just kind of going out there and you've got less respect or less political power with product and direction can you even do your job well if when you go back to the table you say hey i'm out there fighting the fight i just flew 10,000 miles last month spent you know three weekends on the road and here's here's the here's the wisdom and everyone's like the company paid for right right we are paying for this right and then and then product is like nah we already we got different marching i've been talking to users too but in a different way and so yes we're gonna pause your thing because we've got enough backlog already and i've already led this direction here so yes it's almost just wasted it's absolutely wasted yeah it's absolutely then you go back to the developers that you spoke to i was trying to be kind about it i suppose by saying almost yeah and then you didn't deliver for the people you spoke to either right like you couldn't actually get their request represented in a way that gets it so you're ineffective on both sides that can

be incredibly frustrated i'm sure yeah i call this a two-way umbrella for the company to the users and from the users back to the company and you just have to filter a lot and so i call this an emotional burden and when i tweeted that i was definitely feeling it yeah and yeah i mean this part this comes with the territory but you know if you want to actually change anything about it instead of just tolerating it you take over product and this is something i actually ended up doing a temporal i ended up being the pm of the typescript experience and actually i think it helps that sort of two-way synergy because after i was done being the pm then i also then flipped back to my dev rel role and started talking about the stuff that i did right so like if you are heavily involved in talking to users and designing the thing then you can very authentically say like i designed this and like here's here's how you're supposed to use it and

people believe right and if you're that highly invested you might as well just be repping your own product right i mean yeah that seems to be the move right because now that's how you it's easier than convincing the product manager to do your things is just become the product manager and that can be very difficult unless it's your own company in which case you wear all the hats and you bear all the burdens but you also get all the upside yeah it's like i mean it's exactly what i'm

doing but i would say like it's a very tough job to hold all those things in one go and i think it's a very privileged position to be in to help to do that for a company that has a lot more resources than you so i i'll just say like yeah people are interested in entrepreneurship you want to be able to build and you want to be able to sell this is one of this sort of dev experience dev rel combined with product role is probably the one of the best jobs out there in developer tooling

i agree i think uh putting dev rel or whatever dev rel function is even if you can't don't call it dev rel under product makes a lot of sense because i think the reason why dev rel kind of gets this shill as you mentioned earlier jared or this bad rap or this sort of pejorative feeling is that you feel like you're out there trying to sell and that's not your job i think the job of dev rel generally is trying to showcase the vision of where the product is going and get that resonance from the community and see if it's landing and also create advocates out there that who become passionate about where you're going so that you can become essentially take that wisdom you've got back to the team and say this is what we're doing this is how people feel about it this is where they're not getting it this is where my demos and my tutorials are not landing this is where my one-on-ones are not they're falling short it's because of this part in the workflow or whatever it might be their job is not to sell their job is to tell and share the story which if you do it right does sell but you're not trying to sell even in our ad spots i don't know how much you care about these things swick how we do our ad spots i literally tell these people that i sit down with more often than not ceos of the companies and i'm like i don't want you to sell okay if you're if in this conversation you're trying to sell we're doing it wrong i just want you to share your story can you share your story for me and not that story on my phone sorry about that it is to just don't don't sell i don't that's that's my job to tell people where to go and to be excited about your thing and to give people waypoints don't come on here and sell same thing for dev rel don't go out there and sell just tell people what we're doing and get that feedback on how we're doing it and is it working and how do we change to make it work

if i could make one tweak instead of just tell people what we're doing you should nerd snipe them like that is the way to hook developers it's like tell something tell people what a more hard problem you worked on and tell people like the backstory to why you worked on it or like what's the sort of intellectual history behind these ideas and like where why is this the thing that is inevitably what everyone is going to be going towards yes like whether or not you build whether or not it's you or like you build it in house or you buy it from us or like you you you know someone else builds it doesn't matter the industry is going this way are you are you with us or are you you know part of the last century whatever like that is the kind of story that i i like to tell which is not just tell us what you're doing but put us in a broader narrative of like where is where where are we in that moment in history and i think you you get the nerd snipe definitely try to show a little bit of the behind the scenes i think a lot of the people a lot of like standard marketing advice is benefits over features and i think the there's a little bit of inversion for developers where you want to talk about features because you let the developers figure out the benefits but like go down to the the implementation details because people love to learn about that so that they never have to touch it and then go like here's the benefits of of that of that but if you only lead with benefits like you know we will accelerate your digital transformation by 10 percent in the you know in the next quarter i don't care as a developer right show me how it

works and tell me something cool one flavor of that that i think would be interesting which maybe we've done maybe we haven't done and we definitely do on some of our shows is like tell me how hard this particular thing was to build like what you have to go through to build this thing exactly and that's where the nerd sniping comes right the nerd snipe is so effective

for selling the product but also selling you on working with me right like come join us we work on cool tan tan yeah exactly you know and that's why i'm like always like i i just i cannot tell people enough to do this because i think you have to kind of repeat it to them a lot that people want to be nerd sniped like they want to work on hard things and if you if you just emphasize the nerd snipe you accomplish both goals of doing any public appearance which is recruiting and selling

nerd sniping for the win so riddle me this how has data science not been rebranded into ai engineering or data engineering or you pick your flavor of the day it seems like the data scientists are just doing what they were doing before mostly maybe there's some deployment things going on now and just like changing the label on their on their business card i think there's

definitely there's definitely some rebranded data scientists that are transitioning over to generative ai really well but i think there's a qualitative difference in the kind of people that are doing really well in generative ai that have no shared history no shared skills no shared language with the data scientists right like there are many many successful engineers that do not know python right and for data scientists not knowing python not knowing pandas is like what are you even doing here you're not part of our club okay so you're saying the the opportunity

has broadened the industry to where you don't need to have the same background right as a

traditional data scientist and and this is not to say that data scientists demand has gone down at all like i there's this there's been a secular growth trend for decades i would just say like this is just a different type of skill set that makes you successful in this era rather than the previous era and whether or not you're successful here relies a lot more on your creativity and full stack product development skills than just pure data science because data science comes in later when you have data to work with but now when you have foundation models that you can just prompt and make an MVP so quickly you actually need to be creative and quick to market rather than being deliberative and analytical being analytical actually slows you down and makes you too conservative you're worried like what are you doing worrying about costs when the cost of intelligence for gpt3 goes down 90 a year that kind of stuff makes sense so what are the attributes of an ai engineer i have a i have a convenient blog post to refer to people so please read it out loud to us now so i yeah i mean obviously it's not to be annoying but i do actually have a blog post uh for this and you know that's part of the the sort of meta game i i do preach to people about the learning in public which is like anytime there's a frequently asked question you should have a canonical blog post for it not just because you can be that annoying person to send it to people but actually because you can actually spend the time to think about it so you have a more complete thought um i think kelsey hightower often says you don't really know what you think until you write it down and the reason he's so thoughtful is actually he writes a lot of stuff down first so uh there's the rise of the engineer post that just celebrated this one year anniversary and you know it's it's the it's the start of a lot of things i'm doing and more recently uh we actually published a hiring engineer post that actually published some sort of reference job descriptions for people and it's i like the sort of framing of offensive and defensive ai engineering defensive meaning like being able to create systems that fundamentally work on top of non-deterministic ai models right lms as you might know like uh hallucinate they're non-deterministic and they actually fail a lot like the the p99s of latencies like are ridiculous sometimes just for whatever uh inference load reasons your selected api provider might have and so it's effectively like how do you create a reliable service on top of fundamentally unreliable foundations sounds very familiar that's distributed systems and like a lot of the the sort of same language maybe slightly different tooling emerges uh coming out of that that's defensive though and then there's also you know let's just call it preventing against regressions or optimizing cost and that's a lot of like fine tuning for smaller models and all that all that good stuff but really offensive ai engineering is exploring new frontiers right like this capability just came out how can we put that to good use in in a sort of end user product way that immediately clicks for them and generates a lot of revenue i think the image companies have have actually had the most success out of this mid journey is is my sort of favorite example of this making something like 300 million dollars a year with 50 employees completely bootstrapped like no vc funding so for people counting at home that's six million dollars per employee and you know i think there there's and there's more examples i can list in there but it doesn't really matter like if the new capability comes out is it the optimization guy or the creative technologist that wins it's the creative technologist and for me it's like okay like you know mostly engineer most engineers are not creative technologists but are they products people can they can they think about like how do i how do i use this capability that just emerged to solve problems for a customer in some way like they can be more creative there so i'm trying to basically explain why that is qualitatively different than the data scientist role which is mostly analytical which is still very important it's just like a different skill set that like if you just don't have that gene in you of like being creative as a product thinker then you

won't be as successful as someone who is who are some other people besides mid journey who are oh god very successful well because from my perspective i've seen aside you know set chat gpt and the likes aside you know generally generally use chat bots like that's as a category set that category aside obviously huge success right lots of value etc give you that mid journey give you that one but the companies that have brand new products that are making moves in the marketplace that are have gone beyond demo and hype to actual product people are paying for i don't have my thumb on that pulse i'm not seeing much of that i'm sure you're seeing more of it so that's what i'm asking

about specifically yeah so my bar for let's let's have a bar right for like it's easy to get to production on a small use case that nobody cares about so production to me is not good enough so let's have an even more aggressive bar of it must make a hundred million dollars a year like that that's that's that's at a point where you can ipo as an independent company right like that you know maybe it may be the bar is 200 but you know that's this is a factor of two so let's just let's just say a hundred million dollars what ai use cases have made a hundred million dollars a year right so obviously we talked about mid journey i have four and then like the fifth one's more speculative conveniently this is another blog post called the anatomy autonomy if people are looking this up but generative text for writing jasper ai and writer.com both have about 100 jasper reached the 75 million arr because before they imploded and writer.com i think is comfortably

at 100 how do they implode did i miss adam did you miss this what happened with jasper

so i i don't know what their revenues are today but effectively they got rugged pulled they got acquired well they imploded before they got acquired they got the acquisition was the was the exit yeah um i don't look i obviously i'm just saying secondhand stories from other people so don't quote me on any of this but effectively the founder the founders we run a podcast being

transcribed so yeah but i'm transcript they just said don't quote me on this

the founder sold a whole bunch of secondary and it just peaced out okay so he basically yeah lost interest in developing the company but then also uh it seems like they so they built a very successful business on top of gpt3 before chat gpt and then a whole bunch of people found out after chat gpt that they weren't actually doing that much on top of gpt3 and then they migrated to chat gpt so they were basically killed by chat gpt is the common narrative i don't know how that how true that is because their focus was very very strong on e-commerce on facebook the reason you don't hear about them because you're not on facebook they are they did very very well they went from zero to 75 million revenue in two years very few people have done that but anyway so since then the emergent winner in that sort of generative text for writing categories is uh writer.com they seem they seem to have figured out that sort of post chat gpt navigation which is not hard like focusing on users and building differentiated features is on top of the model is the job of ai engineering and you just have to do a you know more creative more dedicated job staying on top of it and not being defeated by open ai's like first move into chat bots right so i don't know if that's a fair like i really i really want to stress like i i don't know like this is this is not my industry like i i don't know uh this specific writing case like whether that's a fair characterization of what jasper went through but it is uh interesting story but like so fair amount of revenue there copilot now i think 200 million in arr so like well past right uh and there's a bunch of other smaller co-copilot competitors all with like decent revenue many of which spoke at the engineer conference that i that i held um so you can go go look at look at that chat gpt i think something

like two billion a year i ruled that one out exactly so those are the four categories that

we are like very very sure like makes sense right there's a bunch of like sort of co-pilot for other knowledge worker type things like harvey is is now the emergent example for like we are co-pilot for law and every lawyer needs this or you are behind like fine right so for every knowledge work profession there will be a co-pilot for x right and each of those things will easily make 100 200 million dollars because you are replacing a whole bunch of junior workers uh for that we can talk about the replacement theory issue but like there is real revenue here there's a real case for generative ai it does not have to get smarter to be useful okay the fifth category beyond all this is the agents category which is the most contentious one right it was a complete bubble last year this year the the bubble company is cognition devin also spoke at my conference the first time they ever spoke at a conference i like them i actually i have access and i use them we can talk about cognition if you want they're not the only players in this game of like the fully autonomous agent this one happens to be code related but there are others that are not code related i do think that whoever eventually cracks this will be able to make significant revenue but we haven't seen it yet obviously but you know the bar is for everyone listening is can it make a hundred million dollars and if that's not good enough for you nothing is good enough for you right like you know if your bar is higher than mine then you're just going to wait longer to see the results but this is happening in in progress and you know you can either criticize it from from afar or you can just get in earlier and track the progress as as i'm doing hey friends

i'm here with brandon fu co-founder and ceo of paragon paragon let's b2b sas companies ship native integrations to production in days with more than 130 pre-built connectors or configure own custom integrations brandon there's a certain level of pain that a product team or an engineering team has to endure to let's just call it rolling your own integrations help me understand that pain that angst for those teams help me understand that true pain of delayed integrations for a product not integrating or having to roll your own integration this seemingly slower route

to integrations i think for context one of the reasons we start a paragon is that today the average company uses over 130 different software applications so that means if you're a b2b

software company selling into the markets there's over 130 of your customers applications that you probably need to connect your tool to because customers today expect that any product they buy is going to work seamlessly with the hundreds of other applications that they're using of course we see this when companies come to us and they say hey we have a backlog of 10 or 20 or 50 integrations that you know our sales team has told us we're losing deals because customers are asking us to integrate with all these different apps and we can't deliver on those integrations or maybe our competitors are integrating with these tools and the problem that that results in for product and engineering teams of course is how do we build and maintain these integrations in a way

that's scalable that we can not just satisfy what customers are asking for us today but we can maintain those integrations in a way that's scalable for you know the next hundred customers the next hundred integrations that we need to build so for engineering one of the challenges

obviously the backlog and prioritizing uh time for certain features or integrations but then there's this other side where you gotta really learn every single api and everything is hand rolled custom maintained and over time that kind of gets i gotta imagine kind of taxing on teams

what do you think so most engineers know that you know every api is completely different can be completely different in terms of how they handle authentication in terms of how they deal with

different record types and so it becomes this problem for engineering teams to basically have to become experts in other people's apis and what could be dozens or hundreds of different apis and to build those integrations we've seen can take as much as three to six months per integration for a developer to write the code to build that integration and it depends on the use case of course in the type of product that you're integrating with but of course that becomes a massive challenge at scale when you're looking at how do we scale our product to support you know 10 or 20 or 50 different integrations so again paragon was really designed to solve that problem and to distill the complexities and the nuances and the differences between hundreds of different

sas apps into a single connecting platform into a single sdk that your engineers can install in your app and then easily connect your products to all these different sas applications in the

market okay paragon is built for product management it's built for engineering it's built for everybody ship hundreds of native integrations into your sas application in days or build your own custom connector with any api learn more at useparagon.com slash changelog again useparagon.com slash changelog that's u-s-e-p-a-r-a-g-o-n dot com slash changelog i would love to hear more about cognition and devin it seems like they were unscrupulous in their marketing with the upwork

god okay i will defend them here so yes uh the the headline on hacker news reads cognition or devin debunked right uh very nice alliteration there out of the nine videos that they produced one of them was uh overstated claim which i agree they should not have put out and the claim was that this

bot could make money on upwork autonomously yeah paste in an upwork job and it would just do the

rest and make money for you right that was obviously there was a human behind that being the the bridge from upwork to the bot and also the bridge from upwork from the bot to slack which it does not the devin does not have slack integration so the stuff some stuff in the video was not the true devin experience or they they failed to show you know it's it's how like when people market games they tell you if it's like in-game render or if it's just you know some some artist rendition of what the game should feel like and that one was definitely the artist rendition of what the what the game should feel like eventually but yeah i mean like one video was was inevitably inevitably produced the guy who made it owned up to it and said like yeah sorry i you know should not have done that but like that doesn't take away that there was this is the still the most significant agent we've ever seen outside of open ai prior to this my reference for best agents outside of the self-driving cars that we have in san francisco because those things are the best agents in the world second best agent in the world was chat gpt code interpreter since then we have devin and then since then we have uh claud artifact from entropic which which you can talk about but devin is really good like it's a really really good agent actually a really good generalist agent not even factoring next to the code writing ability and i hope that people don't throw out the baby with the bathwater because unless you've actually tried it you don't know what you're talking about you're you're just reading headlines and you're just repeating the the last headline that you just read well we can't try it because we sign up for

a wait list and then they don't give you access to it and so what do you want us to do besides

speculate we can't maybe spend less time on things where like you know you're just going to repeating headlines i don't know i'm not spending any time on it i watched the upwork video i

watched the debunking video he certainly debunked what they did and there was no question to it so i'm i understand that you're okay with nine out of ten times i tell the truth but when i'm coming out and trying to make a splash and i'm lying in my marketing material sorry i'm just gonna go ahead and just remember i'm just gonna go ahead and remember that that was a bad idea and they

should have done it yeah still like it's a good product you know i have to square those which i can't i have to take your word for it yeah i i have the fortunate ability to say i have no vested interest in devin you know i i just like they gave me access i used it i was impressed and so was patrick collison so was fred ursum from coinbase there's like a bunch of people who cannot be bought who like it you know sure that's deep that's good that's good i'll take your word for it i'll take your word for it so can we like but can we talk about like like i think the technical design of it can be replicated i think the the the real question the thing that people really should be talking about instead of the video which was a mistake one-off mistake the most structural issue with devin is can it be cloned how thick is their moat this is a six-month old company that is now valued at two billion dollars right which is absurd by any any any stretch of the imagination so now that's the real question which devin has to answer and you know the rest of the uh the sort of ai engineering industry has to answer there's a project called open devin that is trying their very hardest to replicate it i've interviewed both of them you can check it on on my podcast i would say that devin is still ahead who knows how long it's going to last but i think that the the sort of structural merits of like what devin has innovated in terms of how agents should be interacting with each other what are the necessary components of agents that is going to stick and if you if you focus too much on the marketing video you are going to miss the actual lesson to be learned from devin which is that hey your agent should have a coding environment should have access to a browser should have a plan and should have uh i'd say like a a terminal and chat interactive chat bot where you can sort of observe what it's doing and correct it in real time and it can respond to you in real time that is the ux that has wild all these people wild myself i have never seen it in any other agent before and i think it's going to be the state the standard or state of the art for all agents going forward because it's so good what are the

odds of something like that which is very general as you said just gets sherlocked by open ai in a

way it has been but not by open ai it's by anthropic right which is the the other thing that i mentioned so that's uh claud artifacts is the other thing that people should really think about they definitely looked at devin and were like oh yeah we're taking a bunch of that they did not do the the browser access because these guys are way too worried about safety as compared to me and as compared to scott from devin so uh claud projects is basically an advanced version of chat gpt's code interpreter that can render and a working web app i often say the the sort of spicy version of this is that claud entropic did more for code agents in two months than replit has done in two years uh because it's basically repli so for the record for my repli friends that you know obviously they did not build a full sort of rebel environment in ide anyway so but like still like you can do very significant programs in claud now that you could not do in chat gpt code interpreter you can do in devin but devin is slower than claud uh and less less generally capable than claud uh it's just very very good and like for the first time people are actually openly talking about anthropic being better than open ai like opening eyes has lost its crown as like the undisputed number one which is wild like i i did not expect a year ago to be living in this world but now we do live in a world where it's sort of like a multi-polar world where there are multiple sort of top powers in in this space it's very very good and you can try it unlike devin yeah no that sounds good that does sound good love some competition for open ai of

course there's been you know turmoil over there as well and there's been interesting things going on

inside and around open ai maybe for for the engineers and you know listening i would say like the progress here has been at the model layer you know so claud artifacts is built on top of 3.5 sonnet which is the current world best model but also there's a significant amount of ai engineering that was required to build devin and to build artifacts and i think that if you want to see what the future of sort of ai engineering should look like you should be trying to build a clone of this thing that's what i'm trying to do because like i think a lot of engineering will look like this will will look like how do you wire up a model to the real world to produce projects of significant value that you would otherwise have had to assign to a junior engineer i think that that is that is absolutely the sort of gold trophy that people are going for right now and obviously like the step beyond that is artificial general intelligence but this

is a pretty good second place a two billion dollar market cap in six months is absolutely amazing and i think that i mean it's a bubble sure yeah clearly but still the fact that it took six months tells me it's gonna take less time when more people are applying to that and so like is there actually a moat there time will tell i guess yeah yeah time will tell they're

trying to build one uh i think the yeah so this is a question of business and less about tech and the moat is really user data right the more people you can get coding with this thing and the more you can observe how people interact with these agents like devin has a six-month head start on everyone else on how people work with devin like agents and if devin like agents is like the you know the goal of this thing then like they will have the the best rlhf feedback data uh on the planet for for specifically this task uh it's the same motivation that opening i had with chat gpt which is you know they kind of locked into this but now they have the longest series of like chat oriented data sources that human feedback data that you could you previously had to pay a lot for and then so that the mode is the data mode but then actually it also becomes you're investing ahead of where the capabilities are so you're sort of you're sort of saying like i will build code i'll write manual code to build out the capabilities that i don't have yet but as the capabilities grow in the fundamental model you can just kind of swap them out for your sort of handwritten code and be more generally capable with the scaffold that you already built ahead of time so i feel like i'm being vague there but you'll see this in the form of the model's ability to interact with the real world a lot of times you're writing integrations you're writing like it'll interface with open api like screw that man like the future is just models like surfing websites just like anyone else would surf websites and interacting with them exactly like you would interact with it but right now right now we have to use the crush of code

uh in future we don't like yeah models surfing websites how does that sound adam uh dangerous cool amazing awesome uh yeah it's a it's a new world yeah on the grander timescale of this like

well this is happening what within the last three years you know what's what does 30 years of this do what is 300 years of this do like we are birthing a new life form uh i you know i do think about that that timescale as well and so like it's it's just an exciting time to be alive and to observe this uh i i don't think it's useful to try to resist it because it's happening anyway i think this is why alignment is important because the people who have believed in this way earlier than anyone else is the alignment people they took ai safety more seriously because they knew this was coming and the rest of us are just waking up now and you know the current mindset is how do you control something that's smarter than you because it's going to be and so i think that that is probably the right mode to think about it the relevant paper for people interested is the weak to strong generalization paper from open ai which is written by young like it before he left for anthropic i do think like if you're worried about the safety elements people are working on it they are trying to look for similar minded people you can go apply for those jobs

well there's always the plug right the plug just pull the plug yeah like the legit electronic plug unless we give them the the new life form we're birthing as you've just eloquently said uh which i'm taking it back by but also want to dig into because it's like wow are we really creating a new life form kind of like what is it that would give it autonomy the agi-ness of it i suppose and this is the holy grail question everybody's doing we need to uh we all need to partake in some marijuana before having that conversation i know so a lot of ai discussions

tend to devolve into existential risks and agi discussions and part of my goal with defining ai engineer is to create a space where those discussions are the side discussions and not the main thing because we're all here to engineer we're all here to build for today's problems with today's capabilities and i think that's a lot of how i think about my impact in this field which is how do i guide people in a more positive direction that basically nobody's against a lot of dev rel is the future is here is not evenly distributed but a lot of engineering here in especially in ai is the future is here but it's not evenly distributed and how do we distribute it best to everyone else you know i've come from singapore one of the my favorite stories to tell is that the singapore government is embracing ai really well for their older folks the the people who don't speak english the people who are disabled the people who need the natural language interfaces to the many many digital forms that are coming up in our lives and applying this ai technology to that civil service i think is is like the best form of like how we do ai engineering so yeah i mean you know we don't have to go to the freshman dorm room conversation of like you know are we bootstrapping a life form that's fun to discuss happy to engage with that but like why i try to keep it to the engineering conversation is to let people have a way to ground

their conversation and what can we do today but you're the one who said we're literally creating a new life form yes i do open the topic i'm sorry that's okay which i think is uh well and i'm so i've been silent for quite a bit because i'm listening quite well and uh i'm just slurping up all the things you're saying and i'm also feverishly trying to find where mark racinovich said in our conversation with him mark racinovich is the cto i believe of azure right jared correct so we met mark at microsoft build 2024 where it was just like basically all ai everywhere all in on ai as we said and mark said and i'm thinking like well mark is you know part of microsoft and they're one of the largest companies that benefit well from open ai's innovations sure the discussions around cognition as well devin but mark said i am not i'm paraphrasing as i couldn't find the quote and i was hoping i could find it but jared please fill in the blanks mark said paraphrasing again that he is not worried about ai taking over developers jobs but then you just say we're literally birthing a new life form and then you're speculating slash revealing to some degree the the agent os of cognition and devin and what you think would be a good outline for anyone trying to copy what they've done meanwhile saying they have you know a leg up in terms of time frame six months a lot of time in today's world but realistically not a lot of time and then you throw out what was it two billion three billion what was the valuation two two which is just incredible like one what is that number based on is it based on the somebody was is willing to purchase it for is that the valuation is it is it based on like tain yeah funders fund

invested at two billion dollars i think they gave them a few hundred million or something

gotcha okay so they're the valuation is based on venture capitalist coming and say okay well we'll give you x at x valuation i guess i'm just camping out there and sort of sitting back thinking like gosh is this really a new life form we're birthing and if so i think we got to talk about that sure well maybe maybe i could uh square the circle here because swix was talking on a very long time span and i found the exact mark rasenovitch quote thank you and he said i can tell you we're not at risk anytime soon of losing our jobs so maybe that harmonizes your stance swix what do

you think yeah absolutely we should always be clear about what time span we're talking in and uh there's there's a big difference between the near term and long term i just think that you know in the grand scheme of things if most agi timelines by the way are like by 2050 we shall have agi right that's within our lifetimes guys like it's time to panic if you really think this is going to end the humanity it's time to panic seriously like we have eliezer yutkowski saying we should ethically the right thing to do right now is to bomb all data centers in the world because humanity ends otherwise he said he said this in the new york i mean i guess i guess if um

if we're in charge of this in terms of innovating it and creating it how can we not have failsafes in place to be in charge of if it goes wrong i mean i think that's where it has to come down to because i'm i jokingly said pull the plug but i literally mean if we control the physical hard wired plug into the wall now jared if that book i mentioned in the intro that i shared with you a while back which i can happily share here too if that happens then we're in a different world i speculated a good intro to a book or a movie and i'm thinking more movie than than book but all good movies tend to begin as books sometimes they get they're bad movies of good books but anyways i digress i was speculating that this intro scene to this movie was a very beautiful cinematic scene where you see this human being and it's so strange to say things like this a human being is happily racking and stacking these servers happily organizing this hardware happily instantiating a new machine into the rack meanwhile the entire task was given to the human by artificial intelligence so the boss you said before live above or below the api i think we're kind of like nice callback that's like a callback i forgot about that a version of that is like above or below the ai you just take out one letter oh because at that point it's like well in the future this dystopian potentially non-dystopian future where subjects of ai but only if we allow it but if we're in control of the hardware and we're the physical beings for now because you do have boston dynamics out there creating robot dogs and the latest version of atlas like at what point do we lose i guess that's why he's saying bomb the data center

gosh i gotta back up pretty much so the the question is um you know can we pull the plug on these on these bots uh so that for what it's worth this is my favorite joke in uh in this in this category which is uh sam altman is very well known for carrying around a blue bag and everyone's jokes is that like the button's in there that like if you want to if he never needed to push the button it's in the blue bag i don't think we can because the secret's out that it's mostly possible to simulate intelligence inside of neural networks and even if the current transformer paradigm doesn't really pan out for that something else will because we evolved from non-sentient life forms we think uh you know unless we're created from in the span of seven days so if we can evolve something else can evolve too and we are currently speed running uh evolution of of this this particular life form so like i don't think that's necessarily a negative for us except that in every prior incidence of less more primitive civilization encountering a more civilized more advanced civilization the more advanced civilization accidentally wipes out the more primitive civilization and right now ai is not more advanced than us but it is growing much much faster than us it is spreading much much faster it learns much faster than us and so we need to figure out how to contain this or eject it from our solar solar system so it doesn't doesn't doesn't affect us i don't think you know these are i don't think that's i don't think that's possible so we have to contain it we have to align it like that that's the only way all right so yeah like and also like i also don't think that capitalism and this safety is this sort of top-down safety are aligned in a sense that in order to control this if you really are concerned about safety you have to nationalize all ai labs and then you cannot stop there because what uses nationalizing things within one border you have to nationalize all borders so you have to take over the world and control all all the developments if you if your intention is to really control from a top-down basis of all ai safety so that's not happening yeah borders are right a big concern this is the the classic like china's gonna do it if we don't do it well there's this um there's

a show out there called west world have you seen this show west world swix yeah great season one

and two okay you gotta watch season isn't there season four there's gonna be a season four no so i

will i would like there is no i think there no actually i think there was going to be a season four but i believe it was canceled i don't know it's an hbo show i can't i gotta check in on that so in my opinion the entire show is worth watching for season three alone and i think you only really need to maybe even watch recaps of season one and season two to watch season three i don't think you're really it's almost standalone in my opinion and i think anybody out there listening to this right now head nod into season three is is nowhere i'm going and i don't want to ruin any of the plot for you all because you haven't watched it but i would say go watch it a lot of what you're talking about here is represented some way shape or form in the intelligence and the autonomous beings let's just say that are out there in the world doing different things and it's very captivating from a cinematic standpoint and i think if we're 26 years away from 2050 i had to do the math there real quick if we're 26 years away from agi or even the beginnings of it and cognition can create what they created in six months or some span of time less than a year i gotta imagine whatever was in west world season three is closer than we think some version of that's close than we think it could be 2070 yeah plus plus minus 20 years i don't even know how to do math these days like yeah i mean like 30 more years after that 20 more years after that it's got to be close if you get to that speed of creation and then i would also say the other thing i've learned about is von neumann probes it's this idea of a self-replicating spacecraft so shooting it out into space is not going to be helpful because they they might they might allow themselves to escape on a von neumann probe which will just self-replicate it will begin to ore and mine planets to create new materials to create themselves to just replicate and come back and do whatever now they could be peaceful if you've read the books i've read anyways yeah no no real

response to any of that apart from uh you know it'll happen like really like if you want to be a player on this stage you either need to be a political leader of a world power or you need to be ahead of a major ai research lab basically the rest of us don't really get a say this is not

where something where democracy has any sway over we are below the abi below the ai line yeah i think

we're above it in the sense of like we do get freedom from like currently we do get freedom from like the mundane tasks like i no longer care about doing like really minor features because i can just tell cursor to do it for me uh and it does it really well and you know if cognition pans out or what something like cognition pans out then i will have a lot of prs done purely by agents and that's great but like yeah we will live below we always have and always will live below the power line you know then you know that's that's separate from the api line of uh people actually deciding the sort of future course of humanity and like uh i think where engineers really make or break here is whether or not we choose to join them and enable them because they still need us to execute things though the one hope the one note of optimism you know between the very short-term future which is where we are today in a very long-term future which is uh when agi is here is that i do think that ai engineers are the last job to exist because they are the job mathematically to eliminate the other jobs like you need the engineers to eliminate the lawyer you need the engineers to eliminate the i don't know the executive assistant right so like if you're if you're worried about job replacement go be an ai engineer because that will be the last job and then then we'll be post abundance and then and then we can explore the stars but until then you should be an ai engineer there's the sales pitch there you go if you would like to destroy all other jobs become an ai engineer and you will be the last person

standing i do have to bring out uh my favorite tv show jared silicon valley well you lost me

at westworld haven't seen a single episode don't know what you're referring to in season three so

go ahead man in the final episode of season seven sorry season six episode seven called exit event of silicon valley they're locked in uh you know the pied piper offices and they're dealing with what they're dealing with obviously it has to deal with ai because that's the conversation we're here right now and jared says okay is this a good thing or a bad thing somebody tell me how to feel and gilfoil says and this is my favorite line ever abject terror for you build from there so that's my advice for everyone that is not a political power or whatever you just said you had to be swix to have any say in the future this because abject terror for you build from there

yeah uh i i don't call it terror so much as we live in a point of history right history is happening we're we're lucky to be alive to witness this in this moment we have some minor sway on it you know but like history is bigger than us and it's going to happen it's going to take course and and i don't know like uh to me like i think this is part of the east general eac message which is that if you're pro-life you don't have to be only pro-human life you're pro-life in any life form you're pro-consciousness in any any form and if humanity happens to be the sort of bootstrap load sequence to what actually is what life is supposed to be which is sort of uh you know more more reliable sustainable faster learning machines than us uh then maybe that's the natural order of things i don't know i would like it to not be the case because i like i like humanity i like my body but we do live in a world

where that is a possibility and this is why we almost outlaw conversations on artificial intelligence on this podcast because of this this is almost why we outlawed almost i'll mention one

last thing maybe as a as a positive part parting thought please be positive i'll help you be positive we used to basically completely throw in the towel on interpreting the model weights you know gpt3 was 175 billion parameters absolutely like just meaningless numbers like 175 billion meaningless numbers and i used to just think of mechanistic interpretability as a joke i will say anthropic has done a crazy amount of work here uh recently to make features of models interpretable and if we can study the brain of these things as they think then we can control them very very effectively and i have gone from this will never happen to oh i didn't know that this is possible and that's where you should read the paper scaling monosemanticity from anthropic which they demonstrated they can do it on cloud solid which is a mid-sized model we think it's something between 15 and 70 billion parameters if we can do that to 15 to 70 billion parameters from a starting from the standing starting point of like less than 100 million parameters last year we are accelerating our ability to interpret models faster than our ability to grow these models and that is a good thing like we will fully understand and map this brain before it is bigger than us and so like we will be able to control it if that is true uh that like the trajectory of interpretability this year has been an unmitigated success story and it is going to get better and we might actually overtake our ability to grow these brains and that will help us control these programs much more effectively than basically any other method possible i did not know that that's very cool what was the name of that paper again uh scaling monosemanticity it's a it's the third in a trilogy of semanticity papers uh the first one is superposition i covered that in my sort of ai news newsletter uh yeah this is where i plug my newsletter for like go subscribe if you want to keep up on this stuff because uh yeah this is that's my sort of daily pick of what

the top thing to know is all right swix well fun times great conversation dev rel ai engineering agi the end of the world all the things we expect nothing less hook us up with links to your newsletter to your pod to all the things mentioned and we'll make sure they hit the show notes for folks to follow up and connect with you on the interwebs yeah thanks for having me on uh i like

this and friends format because then we can just talk about whatever is top of mind instead of sticking to a specific company or a piece all right well that's all for now but uh we'll talk

to you all on the next one bye friends okay agi is coming 2050 market on your calendar i know i have because i'm kind of scared as guilfoyle said abject terror for you build from there i don't disagree honestly i'm a little bit terrified of like what will happen when the reality of an agi comes especially when you contrast that against boston dynamics latest hd atlas update i mean it's kind of terrifying to think about that kind of robot potentially having agi built in okay enough doom and gloom dev rel is here to stay we've defined it we've described it but zerp is real and zerp affects everything and by the way there's a bonus on this episode for a plus plus subscribers learn more at changelaw.com plus plus it's better it is better again learn more and join the fold at changelaw.com plus plus okay big thank you to our friends and sponsors at century at one password and also our friends at paragon and a massive thank you to our friends at fly.io over three million apps are now launched on fly as we said before we're one of them learn more at fly.io and thank you to bmc breakmaster cylinder for those awesome beats that's it this

show's done we'll see you next week yeah so like basically started a podcast february of last year we had a very good initial success right up to when we had george hots on we actually hit number 10 on the us tech charts so we we actually like overtook the a60z podcast since then like we haven't really had a ton of hits like just like slow steady wins i would say and i feel like it's a little bit of a rut you know i think latent space has done well serving as niche of like we are sort of the ai engineering podcast uh tobias may see from the data engineering podcast and python podcasts try to get into the ai engineering uh space and ended up giving up on it so i think we you know we we have stood the test of time you know we're maybe like it's mildly differentiated from practical ai which which is the show that you guys run and i'm wearing the shirt of today i do think like there's the meta game of like all right how do i run my existing operations better which is i need better editing i need to do better on youtube i need to figure out my audio quality a lot better uh we can talk about all that then there's also like this that's the defensive side then the offensive side is how do i grow like how do i how to get to the next level and for the growth element it's very obvious book big names that's it like that's basically it it's the only thing people care about is it's just hearing from big names and i can go get them but like i just feel like then i'll just kind of devolve to like an interview show just like everyone else i'll just be one of many interview shows so yeah that's

that's where i'm at that's where you're at well that's further than most people get so uh congrats on the success