97 Beechcroft Road
London, SW17 7BP
Call Us on:
+44 (0) 7990522932
Email us on:
neilbatchelor@icloud.com

How Takeout Improved Our Candidate Experience: A HubSpot Experiment


Here at HubSpot, we take culture pretty seriously. After all,
we have an entire code[1] dedicated
to it, and it doesn’t just apply to our internal environment —
it also shows up when we’re recruiting new people to join our
team. We have an inbound recruiting mission of attracting top
talent through a world-class candidate experience.

That’s why, one January night that started like any other —
watching Netflix in my pajamas and eating chicken tikka masala
from my favorite Indian takeout joint — I decided
to respond to a mobile customer satisfaction survey from the
food delivery platform that I use, called Grubhub.


I had been thinking a lot about mobile, and how it could play a
role in our inbound
recruiting
[2] efforts at
HubSpot. So, when I got an automated text that night saying
“Grubhub here! Tell us about your order from India Quality
Restaurant,” I wanted to know: How could we recreate that kind
of seamless experience for HubSpot’s candidates?

So read on — and find out how takeout inspired our approach to
recruiting and interviewing.

How Takeout Improved Our Candidate Experience

The Hypothesis

We all know that texting for business is nothing new. You might
get a text message when you pay your wireless, make a hair
appointment, or confirm a time slot at the dentist. But texting
hasn’t played a role in HubSpot’s recruiting and hiring process
since its earliest days, when developer recruiting was
essentially managed on one VP’s smart phone.

Eleven years later, whether or not a candidate receives or
accepts an offer to work at HubSpot at the end of her interview
process, we want her to enjoy their time with us. That
experience has a big impact on whether or not candidates
advocate for HubSpot in the future, the Glassdoor reviews they
leave, and the likelihood that they refer friends or pursue
future opportunities with us. That’s why we ask for feedback —
so we can learn how we can improve. And until recently, we used
what we called a Net Promoter Score survey (NPS), that was
distributed via email.

But when you consider that, today, people spend
more time browsing on mobile than they do on
desktop
[3], we
couldn’t help but wonder if following up with candidates via
text, instead of emailing them for feedback, would make it that
much easier and engaging for them to actually respond.

So, we came up with a bit of a unique hypothesis: If
our candidate NPS survey was more like Grubhub’s, and the
survey was sent via mobile instead of email, response rates
would increase.
The objective, then, was to get a
higher volume and quality of feedback that could enable us to
even better improve the recruiting experience.

The Experiment

The Candidate NPS Survey Today: Email

When candidates globally have a face-to-face interview with
HubSpot, they receive an automated email at 7:00 PM that
evening with a link to take a short survey with three
questions:

  1. On a scale of 1-10, how likely are you to recommend HubSpot
    to a friend based on your experience?
  2. Which department did you interview with?
  3. Anything else you’d like to share with us?


From there, my colleague Danielle
McLellan
[4] analyzes
the results and synthesizes the feedback, so the recruiting
team has some tangible insights into what is (not) working
well.

For context, the response rate to date on the email survey
hovers around 55%.

The Introduction of Mobile

There was just one hiccup with implementing my mobile survey
distribution idea: I’m not mobile development savvy, so
executing an automated text survey was pretty foreign. But, I
could still perform research with the best of ’em, so I
gathered some information on mobile survey vendors, and chatted
with HubSpot’s developer team about the feasibility of two
options:

  1. Buying the tech to create this feedback system.
  2. Building that tech internally for the same purpose.

We ended up going with a scrappy, but reliable, solution that
combined two platforms: Textit.in + Twilio.

Textit.in[5] is a mobile
messaging platform with pretty intuitive usability. You build a
visual workflow of the desired text series, upload contacts’
phone numbers, and schedule the workflow to start at a certain
time and date. While it’s not highly sophisticated, for the
purposes of this experiment, it provided the solution we
needed.

Twilio[6], a
cloud communications platform, provided us with a virtual phone
number that could easily be connected to Textit.in. All in all,
this technology cost us about $60, showing how frugally
something like this experiment could be replicated by other
marketers.

Only candidates interviewing in our Cambridge office received
the mobile survey, as establishing an international virtual
phone number would have required additional steps that,
considering our deadline, time simply didn’t permit. That said,
we had a population of roughly 220 Cambridge candidates to work
with.

The Process

We ran the experiment from February 13 – March 31. Once the
technology was up and running, here’s how it worked:

  1. Collect Phone Numbers. Our recruiting
    coordinators added candidates’ mobile phone numbers to a
    shared Google spreadsheet by 4:00 PM daily, with information
    like the candidate’s full name, date of her face-to-face
    interview, and the department she interviewed with.
  2. Schedule Text Workflow. I then imported that
    information into Textit.in, and set up a workflow to have
    that group of contacts receive the mobile feedback survey via
    text at 7:00 PM on the evening of their respective
    interviews. To make sure we didn’t change any variables aside
    from the method of communication, we kept the following
    variables consistent across all messages:
    • Time of send.
    • The series of questions asked of candidates, with the
      exception of one trigger question: “Hi, HubSpot CareerBot
      here! Thanks for interviewing today, we’d love your
      feedback. Will you answer a few quick questions about your
      experience? Y/N”. (By the way — shout out to my colleague
      Noah
      Gilman
      [7] for
      coming up with the “CareerBot” name.)
  3. Collect results. I then created another
    spreadsheet to track the following results:
    • Who did (not) respond to the survey.
    • Each person’s NPS score.
    • Any open-ended comments.
  4. Analyze results. McLellan analyzed those
    results to look for patterns and other actionable outcomes.


For the sake of privacy, the above example isn’t one from an
actual candidate, but one that I made up to illustrate how the
system works — though, the part about Dave’s beard is true.

Results


As predicted, the volume of responses to the mobile surveys
outnumbered those from email considerably. Even better: We saw
an improvement to our percentage of promoters and the overall
candidate NPS score via this medium.

And thanks to the higher volume of responses, we also were able
to gain better insights from more open-ended comments. We
observed that, no matter what the medium, our recruiters were
receiving high praise from candidates, and — although, yes,
I’m biased — the best part of this experiment was getting to
read that feedback. Here are a few of our favorites:

Julia Blatt and Kelsey Freedman were awesome coordinators and
made my day great! Everything was timely and went smoothly.
Thanks to Becky, Gus and Amy!”

The entire interview experience was amazing. The recruiter
was flexible to accommodate my scheduling requests at every
stage of the interview. I was well informed on each stage of
the interview which certainly helps the preparation. The
follow ups have been prompt and timely.”

My recruiter, Noah, has done an absolutely amazing job
throughout the process. Very transparent and informative.
Send him my thanks.”

(P.S. We’re hiring[8].)

Next Steps and Takeaways

Across the board, the mobile results indicate an improved
follow-up experience for candidates — and that candidates are
more likely to give us feedback via mobile. And based on these
positive results from the experiment, we know there’s a place
for mobile in our recruiting process … and that there could
be in yours, too. Remember the statistic from above about how
many people are browsing on mobile than desktop? Keep that in
mind next time you’re looking to improve a user experience.

We’ll continue to carry out additional tests of this kind to
collect more directional benchmarks — things like language
modification, timing, and scalable technology. Also, we’d like
to expand the experiment to our global offices and will be
formulating a timeline and corresponding plan to do so.

If you’re curious about what else is going on in our world of
inbound recruiting, check out the Move On
Up
[9] blog, which
gives readers a peek inside culture and careers at HubSpot.

And in the meantime — I would highly recommend India Quality
Restaurant to a friend.

How has your team enhanced its recruiting experience? Let
us know about your best experiments in the comments –and hey,
we might even feature it on our blog.