ilyaa

6 years agodiscuss

yup, shame on me for that. thanks for headsup, this was addressed.

6 years agodiscuss

Hey John, how about you try it yourself? For the right mindset, I've been told that it is very fun and addictive :)

Thanks for the questions, let me elaborate:

   - I didn't have time to prepare privacy policy, but we'll indeed make one and post it. You're right, we should have it.
   - There is no tracking. Never will be. We'll blog about why we're for 100% anonymity of the candidates in more detail, but for now: We don't collect names. We don't require valid emails to enroll in the demo. We don't use tracking cookies. We don't send HTML emails with tracking pixels, and we don't use any 3rd party service. We use cookies to authenticate you, and that's it. Javascript, social media buttons, or trackers or analytics of any sort on our pages are considered a bug. What we don't have, is a proper explicit privacy policy explaining all that.
   - We're a European company.
   - That's how Namecheap sets it up -- curious, why do you think it is a bad thing?

ilyaa

6 years agodiscuss

Hi! Thanks for asking. We will blog about it in more detail, but I can give you some numbers right away. At the moment when we decided to start making it into a product, we've been using this approach for roughly 4 months. During this period, the data was: - we processed roughly 600 candidates - of which we hired about 50 - per-recruiter rate of processing people increased from 15/month to 100/month - average invite-to-offer time reduced from two months to two weeks - on top of that, the number of challenges to review by our engineers dropped by 60%. The explanation of why will be in our next blog post about the cost of engineering engagement.

At this point, we decided that this is too good to hold only for ourselves. Thus, AutoIterative was born, where we also significantly improved the parts that weren't polished well enough and added features we lacked.

ilyaa

6 years agodiscuss

Thanks! Full disclosure -- it is my first advertisement, ever, so I'm glad you liked it :)

I would challenge the assumption that this is purely advertisement, though. In my head, this is not "buy our shit™" type of post -- in fact, there's no mention of buying the subscription anywhere. All there is is a link to a demo, with a valid challenge (not a fizzbuzz/fibonacci, but one you can actually have a lot of fun with), which we provide for free to anyone, w/o even requiring valid email -- it is up to you if you don't wanna reset the password later. Instead, the post is more of a "use the right metric -- here's our experience and what has worked for us" type of post.

Would you agree that this is a more fair assessment?

ilyaa

6 years agodiscuss

Hey, thanks for insightful feedback (blog post author here)!

I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option.

Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :)

We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric.

Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step.

ilyaa

6 years agodiscuss

Hey! To be fair, we don't criticize it (blog post author here), nor do we discourage such training, of course. It is perfectly valid and could be a good ROI, as you mentioned. What we found out is that using algorithmic knowledge as a metric for hiring was suboptimal. As other people mentioned, there's plenty of people w/o formal CS background who can deliver, and there's plenty of people who know algorithms by heart but have trouble shipping things. There's some correlation, of course, but I would say those are different dimensions.

And then it looks like this: when hiring someone, the company measures them by one dimension and then expects them to perform on another one. I did get some backlash for comparing engineers to cooks, heh, but the main point is and always was about choosing the right metric.

ilyaa

6 years agodiscuss

Hey, post author here. Apologies if it sounded paternalistic, that was definitely not the intention.

I would argue that 40-50 senior people in one place would be rather an exception from the norm, and your company would be quite an outlier on the bell curve. If all of you are not just seniors, but also had first-hand experience participating (or even building) in hiring loops of the large companies, then you would have all the necessary knowledge of why those rituals were created, and why. The point of the blog post was that this is usually not the case, so the imitation is the only option.

Could you elaborate on your hiring loop? Which of the stages is the most effective and brings you the most value?

ilyaa

6 years agodiscuss

Hey juped, of course, please do reach out, I would love to hear your take on this problem because it is a very complex one indeed.

Some context: we're not claiming we have a silver bullet or fit-it-all solution as some people got from the post (and that's my fault because I'm responsible for that text, and blogging is hard, haha). And we're also not claiming we can replace _the whole_ of your hiring loop (that would be the silver bullet).

We claim that choosing the right metric can drastically improve the hiring loop, and we've built the tool that automates the assessment of this exact metric.

ilyaa

6 years agodiscuss

You're spot on. We use hugo with a custom theme, which is derived from etch (https://themes.gohugo.io/etch/).

ilyaa

6 years agodiscuss

Hey folks! co-founder here, I'm one of the people behind autoiterative.com. I did a ShowHN yesterday[1], but obviously, the blog post provokes much more engaged discussion, love it ;) I appreciate your feedback, please keep it coming and challenge our assumptions!

I will try to address the questions tomorrow, I already see some of them require a ton of thinking indeed.

[1]: https://news.ycombinator.com/item?id=23986002

ilyaa

6 years agodiscuss

Hey HN!

I want to share something we've built to improve the current state of hiring in our industry. We think algorithmic interviews are the wrong metric, so we're trying to have a better approach. We tested this idea successfully for more than a year in a ~2K people company, saw great results, and decided to make it into a product[1].

If I had to sum up the idea in one sentence, it would be this: "to hire great software engineers, test their ability to deliver something as close to real work as possible." Thus, we build a product that provides a challenge, a CI pipeline, and a "production environment" -- what the candidate needs to do is deliver something that works, making pipeline green and iterate. Sans peer review, this closely resembles the actual way it happens at their jobs. Behind the scenes, our system runs all sorts of tests against the submission, simulating the actual production. I can spend a lot of time talking about it, but we'll blog[2] about it in more detail in the following weeks.

We're currently in the super early stages, but we have a working demo that I want to share so that people can play with it, break it, criticize it and hopefully have as much fun with it as we had when building it. If you want to give us more direct feedback -- feel free to reach out over email to demo+hn at autoiterative.com.

[1]: Direct link to the demo: https://ais.autoiterative.com/demo [2]: Our blog: https://autoiterative.com/blog

Submissions

Can I Work with You?

Show HN: AutoIterative – an opinionated way to hire software engineers

comments