Very few people know Alexa, Amazon’s artificial intelligence service, better than Emily Serviss, a 34-year-old data associate.
“We’re getting acquainted pretty well,” she jokes. And she’s only half kidding: In the two months since she joined Amazon, Emily has been reviewing Alexa’s responses to real user commands for accuracy, one command at a time.
“She has Easter eggs,” says Emily. “She can drop a beat, and sing country songs. And she knows nerdy things, like if you say, ‘May the force be with you…”
You get it.
Emily’s colleague Sean Elks, 25, signed on as a data associate in mid-March after a year and a half working at a doggy day care and is—not to be sappy—equally smitten.
“I’m always learning new things about her capabilities,” he says. “Users will ask something that I’ve never heard somebody ask before, and she has this clever, relevant response that I wouldn’t have guessed she had.”
In case you’re wondering where Alexa got her charm, the answer is simpler than you might think: She had help from humans.
Don’t get us wrong. Alexa’s capabilities are backed by sophisticated engineering and cutting-edge tech that’s way too complicated to explain in one article. But behind the scenes, a troupe of 100-plus Boston-based Amazon employees on the Alexa Data Services (ADS) team are busy gathering the insights that help engineers improve Alexa’s day-to-day functionality so when you ask her a question, she will actually understand you.
Here’s a look at who they are and how they’re doing it.
The Road to AI
When Emily graduated from Ithaca College in 2004 with a degree in cinema and photography, her plan was to work in film. So naturally she shipped off to LA and took odd jobs in pre- and post-production. A few years later, she switched to early childhood education and spent the better part of a decade working with toddlers. Today, Emily finds herself reaching for old skills she learned in those earlier careers: transcribing from the film days; deciphering the sometimes bewildering requests of children from the pre-school era. Proof that even if you didn’t think your current field lends itself to a tech transition, it definitely can.
Transcriptions are actually the first thing data associates learn. The ADS team needs to be able to compare Alexa’s interpretations with those of a native English speaker, and to do that they need to know what users ask her. That means they need a written record of the user’s command. The idea: Give Amazon’s AI engineers visibility into which commands Alexa is having trouble understanding and why.
Sometimes transcribing throws curveballs.
“A lot of times kids will just play with Alexa,” says Emily. “But I’m used to talking with kids, so I can understand them better when they’re talking to her.” Emily is a go-to for colleagues who are left scratching their heads. “A lot of people might not recognize Kidz Bop, or The Wiggles. But thankfully I have a wealth of knowledge about Kidz Bop.”
The Human Touch
Understanding Alexa’s intent and how she arrived at it may be the most critical part of a data associate’s job. But to get inside her thinking, they first have to “annotate” transcribed commands, or categorize them for the type of response they should trigger.
“We’ll see what Alexa hears and how she interprets each word,” explains Sean, who graduated from Northeastern University in 2014 with a degree in animation and game design, “whether that’s playing music, looking at your calendar, or setting up a timer.”
Annotation paves the way for GSR, or goal success rate, the final part of the data-gathering process that aims to pinpoint where Alexa messed up. With transcriptions and annotations in front of them, data associates can compare how humans heard and interpreted the command with what Alexa heard and interpreted and flag Alexa’s missteps.
“Did she do what she was supposed to do, and if she didn’t, why?,” says Emily.
Evaluating command after command can be tiring, and it requires a keen sense of concentration. Sean says that his artistic background in animation prepped him with a heightened sense of detail. “Animation is all computer work, or at least the animation I did, so all my experience with computer and technology carries over well. And during animation, half the work is put on 10 percent of the finished product because there’s so much attention to detail to make it look good.”
To be fair, there’s going to be a lot of computer work for anyone involved in AI. But it’s not time wasted. People working on AI today are at the forefront of a huge emerging technology that promises to transform the way people live.
“What we do is about not just technology. It’s [how it connects with] with people, and fostering that connection,” says Sean.
So the next time you ask Alexa if you need an umbrella, or your children ask her to play the song from Moana (again), remember, there’s a whole team of people that made that simple command happen. One that you could even be a part of.