How HubSpot scaled AI adoption(product.hubspot.com)
product.hubspot.com
How HubSpot scaled AI adoption
https://product.hubspot.com/blog/context-is-key-how-hubspot-scaled-ai-adoption
49 comments
The monitors thing is funny to me because I love using dual monitors at work, and my coworker doesn't, and this forced AI adoption would be like if I forced him to use dual monitors.
I thought of this exact scenario when I made my comment! I'm sure many people benefit from multiple monitors but some probably don't at all.
It’s the same trend of executives claiming RTO increases user productivity according to their data but could never show the data.
People that don't code think that something can do code 98% correct is surely better, without seeing the hard enforcement of a 2% error without manual intervention. I think all of the jokes about stupid computers and weird behavior of languages when you use them out of spec (famously labeled wat) gave people the wrong impressions about why those problems exist or how they can or cannot be fixed. You'd think they'd be able to transfer that cynicism quicker to models since they are also dumb computer things but apparently they say "hello" and that is tricking them out of that?
It's crazy how fast we went from big data and every exec needing some massive dataset with cooked up numbers to justify even the smallest decision, to this, where nothing matters except for ~vibes~. Does AI increase productivity? Does it improve or degrade quality? Who knows, but number must go up.
>Nobody’s doing anything like that for other productivity tools even when they’re proven to be helpful.
Isn't mandating IDE usage a perfectly reasonable and common thing?
It's a productivity tool after all.
Isn't mandating IDE usage a perfectly reasonable and common thing?
It's a productivity tool after all.
I've never seen an IDE "mandated", I've seen officially supported development setups where you're on your own if you do anything different. Is that not the standard?
Your job will look pretty funny at you if you want to code everything by hand while everyone else is using VSCode.
I've literally never seen this and I've been around for awhile. There's always people with some bespoke vim or emacs setup while everyone else is just using Jetbrains or VSCode or whatever, and nobody cares at all as long as that person is getting their work done.
Still an IDE right?
I'd never have worked at a place that mandated a specific IDE. Luckily it's something I never encountered, if it's ever done this is the first I'm hearing about it.
But you'd have to use an IDE right?
No, it never occurred to me that anyone would care. I don't think many of my bosses or teammates could have named my choice of IDE/editor even after working with me for a while. I'm sure I mentioned it occasionally but I don't think anyone would care enough to remember.
I was usually most productive with a text editor as opposed to an IDE but I'd sometimes use an IDE as needed or when I wanted to try something new.
I was usually most productive with a text editor as opposed to an IDE but I'd sometimes use an IDE as needed or when I wanted to try something new.
Is this the company that hacked and attempted to extort a journalist who wrote about them?
I was really looking for tangible, actionable advice since I'm facing slow adoption in my org. This post seems to hide behind the "secret sauce" that it claims made all of the difference.
I wanted to reach out but I couldn't find your email. Mine's in the profile if you want to chat.
Out of curiosity, do you have thoughts on why the slow adoption?
The mention of "HubSpot" triggered a memory... https://news.ycombinator.com/item?id=11369632 - wow, can't believe how long ago this was! Anyways, not surprised that a company where 1 + 1 = 3 would be big on AI.
I think this became this book: https://www.goodreads.com/book/show/26030703-disrupted right?
I remember this book well.
I also remember thinking “this guy kind of seems like a self-impressed jerk,” while reading it.
Not that HubSpot didn’t earn their portrayal as a hype-driven business run by clowns, but the author lost me when he described himself getting into a passive aggressive Facebook comments argument with coworkers as an indication of how stupid they were… when all I could think was “you’re all idiots, cut the FB drama and get back to work.”
It seemed like he was kind of looking for a fight the whole time. Like… if you’re shocked a marketing tech startup runs differently than a newsroom, that’s on you.
I also remember thinking “this guy kind of seems like a self-impressed jerk,” while reading it.
Not that HubSpot didn’t earn their portrayal as a hype-driven business run by clowns, but the author lost me when he described himself getting into a passive aggressive Facebook comments argument with coworkers as an indication of how stupid they were… when all I could think was “you’re all idiots, cut the FB drama and get back to work.”
It seemed like he was kind of looking for a fight the whole time. Like… if you’re shocked a marketing tech startup runs differently than a newsroom, that’s on you.
>measurable but modest productivity improvements
No mention of how this was measured.
No mention of how this was measured.
> We pulled metrics on code review burden, cycle time, velocity comparisons before and after adoption, and production incident rates.
This is not how statistical analysis works. You need to know how the comparison was done to know if it's valid. You can't merely tell me the data points you've used as evidence of the success of your methods. With claims like these, a citation is truly needed.
I'm very skeptical on this because I know there is competing research suggesting AI use makes tasks take longer but feel less burdensome. Also, you'd need to account for regression rate over time. Also, you'd need to ensure your methodology is correct. It's not trivial and great claims require great responsibility.
I'm very skeptical on this because I know there is competing research suggesting AI use makes tasks take longer but feel less burdensome. Also, you'd need to account for regression rate over time. Also, you'd need to ensure your methodology is correct. It's not trivial and great claims require great responsibility.
If you're looking for a company to blame for the endless Google results of "top-10 ways of doing X" or "the best new vacuum cleaners review", look no further than HubSpot. Their business model was based on helping small business gain traction by writing a a lot of verbose blog posts. So now when you're looking how to fix a leaking faucet, you first have to read about the history of faucets.
It's not that I don't believe you, but the "top-10 X" format is so easy to replicate that I highly suspect that it was pushed by one single company.
Proof / refs?
Their founders (Brian Halligan and Dharmesh Shah) are credited with defining "inbound marketing."
Here is the original version of their book: https://www.amazon.com/Inbound-Marketing-Found-Google-Social...
Here is the original version of their book: https://www.amazon.com/Inbound-Marketing-Found-Google-Social...
In a way only marketeers think hubspot does very good product marketing. (i’m a user of hubspot for 8 years or so)
Not proof, per se, but the term to look up is "inbound marketing".
HubSpot was very big on pushing companies to publish lots of content like blog posts and then having calls to action for people to submit their info in exchange for a whitepaper download or similar. Predictably if your main goal is to consistently publish blog posts and whitepapers to generate leads, and you don't have a strong culture of quality and good writing, it's going to lead to lots of slop (even before you could automate writing it with AI).
That being said, I'm not sure how much to blame HubSpot vs. this just generally having been a marketing approach/idea that was "in the air" while it sort of worked (for some definition of "worked"). I sort of remember a handful of companies at the time doing pretty good blog/content marketing by writing useful and thoughtful stuff, and then lots of companies going „got it, make blog and profit!“. But possible that the HubSpot push accelerated that a lot — I don‘t feel like I have a good intuition about that part.
See e.g. https://www.hubspot.com/inbound-marketing
HubSpot was very big on pushing companies to publish lots of content like blog posts and then having calls to action for people to submit their info in exchange for a whitepaper download or similar. Predictably if your main goal is to consistently publish blog posts and whitepapers to generate leads, and you don't have a strong culture of quality and good writing, it's going to lead to lots of slop (even before you could automate writing it with AI).
That being said, I'm not sure how much to blame HubSpot vs. this just generally having been a marketing approach/idea that was "in the air" while it sort of worked (for some definition of "worked"). I sort of remember a handful of companies at the time doing pretty good blog/content marketing by writing useful and thoughtful stuff, and then lots of companies going „got it, make blog and profit!“. But possible that the HubSpot push accelerated that a lot — I don‘t feel like I have a good intuition about that part.
See e.g. https://www.hubspot.com/inbound-marketing
Inbound marketing is pretty explicitly about creating high quality content that builds a business's credibility, making prospects more likely to want to engage and buy from said business. Creating slop 'top 10' lists is kind of the opposite, pushing customers away. It's a bit of a stretch, therefore to blame HubSpot for garbage content on the internet - since doing this is the opposite of what they advocated. Grifters are always going to be around looking to make a quick buck with the least amount of effort possible.
> Inbound marketing is pretty explicitly about creating high quality content that builds a business's credibility, making prospects more likely to want to engage and buy from said business
Well, yeah, I agree and would probably pursue it like this if I was running a business. However I get the impression that is not what happened at many places that adopted the approach, including one I've previously worked for.
Well, yeah, I agree and would probably pursue it like this if I was running a business. However I get the impression that is not what happened at many places that adopted the approach, including one I've previously worked for.
I'm a bit surprised to see so many negative comments targeted at HubSpot. I got to final round interviews with them a little while back and came away with a very positive impression. Where does all the hate come from? Just their association with marketing in general and the fight over the book about them?
I mean, it's a big 'just':
https://en.wikipedia.org/wiki/HubSpot#Controversy
https://en.wikipedia.org/wiki/HubSpot#Controversy
That said, in terms of "bizarre behaviour by tech company execs", eBay probably still wins: https://en.wikipedia.org/wiki/EBay_stalking_scandal
Because they started as a spam farm and now they're a slop farm?
A way I've been developing and is working really well is to identify high level "goals" of the day or week by analyzing their AI chats first.
Then, measure time taken, AI usage, and sentiment of AI usage.
With this, we find out how quickly was the task done, how much AI was used, and whether the individual was frustrated at any point and if the process went smoothly etc.
My system already hooks into top AI providers and measures these outcomes for engineers. Working on measuring other use cases. Email in profile if anyone wants to chat.
Now of course we can't do a blind comparison with the exact same task, but this at least gives insights into usage, outcomes, and ease-of-use.
Then, measure time taken, AI usage, and sentiment of AI usage.
With this, we find out how quickly was the task done, how much AI was used, and whether the individual was frustrated at any point and if the process went smoothly etc.
My system already hooks into top AI providers and measures these outcomes for engineers. Working on measuring other use cases. Email in profile if anyone wants to chat.
Now of course we can't do a blind comparison with the exact same task, but this at least gives insights into usage, outcomes, and ease-of-use.
[deleted]
Honestly its been pretty wild to see this company succeed over the years. They took on Salesforce and everyone predicted they were going to fail. Yet year over year they've continued to succeed.
If someone wants to pat themselves on the back with how great they think they are, thats cool, but I dont think its really worth talking about.
…unless they have something to show, specifically?
Demos? Code? Details?
Nothing?
…unless they have something to show, specifically?
Demos? Code? Details?
Nothing?
Admittedly, more detail would be better, but this high-level stuff is mostly the level that engineering leaders are discussing this topic currently (and it is by far the most discussed topic).
They actually revelead an interesting tidbit where they are with AI adoption and how they are positioning it now to new hires, e.g. "we made AI fluency a baseline expectation for engineers by adding it to job descriptions and hiring expectations".
It seems inevitable now that engineering teams will demand AI fluency when hiring, cuious though what they are doing with their existing staff who refuse to adopt AI into their workflow. Curious also if they mandated it or relied solely on incentives to adopt.
They actually revelead an interesting tidbit where they are with AI adoption and how they are positioning it now to new hires, e.g. "we made AI fluency a baseline expectation for engineers by adding it to job descriptions and hiring expectations".
It seems inevitable now that engineering teams will demand AI fluency when hiring, cuious though what they are doing with their existing staff who refuse to adopt AI into their workflow. Curious also if they mandated it or relied solely on incentives to adopt.
This was just our first post FWIW, and we definitely want to follow up with more concrete demos/details/etc here. I am working on another post specifically about how we leverage our internal RPC system to make adding AI tools super easy so expect more from us.
I really don’t understand why AI usage is mandatory for roles. Nobody’s doing anything like that for other productivity tools even when they’re proven to be helpful. Hell, a lot of employers can’t be bothered to provide basics like nice keyboards and monitors that exceed 1080p.
The current era of tech has way too many corporate losers.