Edwin.AI is your English as a Foreign Language (EFL) Tutor (Dmitry Alekseev and Dmitry Stavisky)

Part of the Bot Master Builders Series

The Edwin chatbot is the first step in building a cost-effective service for learning English as a foreign language (EFL). Edwin’s co-founders, Dmitry Alekseev and Dmitry Stavisky, aim for Edwin “to become a comprehensive EFL service delivered by both bots and humans that gets students to proficiency in one-third the time and for one-third the cost of existing solutions.” Mssr. Alekseev is Head of Product while Mssr. Stavisky is the CEO.

The two Dmitrys are building Edwin for one simple reason: Students and their families spend an estimated $40 billion a year on EFL education. Still, most students who receive English instruction never reach proficiency because public school courses in English are insufficient in many countries. Families often supplement schools with private language lessons that use outdated methodologies like group rote learning, and don’t personalize instruction. Students of these second-rate English schools are surrounded by peers at the same low fluency level and that impedes progress, while private tutoring is unaffordable.

Edwin uses AI-powered bots to do the bulk of English teaching and language practice. It frees teachers to use their time on more valuable tasks. The goal is for Edwin to become a comprehensive EFL service that gets students to proficiency in one-third the time and for one-third the cost of existing English schools.

we’re using natural language processing, machine learning, the newest speech technologies, and industry experience from the first wave of ed-tech products to bring personalized pedagogy and adaptive learning plans to students.

Based in San Francisco, Edwin supports Spanish, Japanese, Korean, Russian, Arabic, and Hindi speakers. To use the tool, a student signs into Facebook Messenger, where they are connected with an AI chatbot that gauges the learner’s English proficiency level and learning goals in order to make a personal learning plan. After that evaluation, students can take one of 3 courses: vocabulary, TOEFL or TOEIC. They interact with Edwin to practice different language skills and to prepare for the tests. For example, they can complete verbal exercises with Edwin by recording themselves and comparing their pronunciation with examples Edwin provides. This one of more than 50 different practice tools.

Edwin is backed by General Catalyst, Maxfield Capital, and the Google Assistant investment program. For Google Assistant, Edwin’s goal is to practice spoken English. Edwin would record a student’s words and use their NLP tools to check their pronunciation. Then learners can speak directly with their virtual assistant to learn how to pronounce English words.

What was the original design vision for your bot? Does it have one or two clear functions, or many?

Our goal is to teach quality English learning for a fraction of the cost of the fraction of the time — learning words and grammar, improving reading and listening comprehension, and practicing other language skills — to automate the most of it. Human teachers focus on writing and speaking, that are the hardest skills to learn. Also, they can handle new problem solving. We use our own human teachers “in the loop” if there is a problem and a user is stuck; they also grade speaking and writing assignments. The specific problem Edwin deals with is passing the TOEFL (the English as a foreign language written exam), and soon TOEIC (the English as a foreign language conversational exam) in beta. Both TOEFL and TOEIC have cover writing and speaking, as well as reading and listening. In fact, TOEFL has higher requirements for speaking. These tests are developed by ETS for different geographies.

Team — how did your team come together, and what are the roles?

Both Dmitrys have first hand bad experience with EFL services. Right now Edwin has 7 people; the core team is distributed all over the world, and the Dmitrys met casually through friends. The team has a background with Evernote and Lingualeo ( the largest EFL learning service in Eastern Europe, with over 15 million students ), plus Yandex, the main internet company in Russia.

How do you measure success; what are your metrics? MAUs vs typical session length and completions?

We are extremely data-driven. We look at engagement via who completes exercises and the “nailed and failed” rate. Too much nailing (high scores) or failing (low scores) is bad; it means the test’s level is too easy or too hard. Writing and pronunciation are also two important skills. Humans evaluators do this now — they do all the nuanced grading until we get a large data set built. Edwin runs with a combo of technology and humans — doing the “th” sound to practice. The goal is to get to automate 100% ofr repetitive tasks and let humans do the intricate learning and problem solving. That means scaling the best tutors effectively, not eliminating them.

The ultimate metric is how many students complete the course and take the TOEFL to pass with a high score (you can’t fail these tests — you just get a miserable score). We measure every step — every learning exercise is adaptable. Students will also do mock tests first — they won’t do the actual test or issue certificates. TOEFL has 4 sections and we want a test passed with the score passed.

Edwin have over 800k registered — most use the teaser practice product for learning vocabulary, though the Dmitrys are keeping the monthly active user (MAU) count private for now. The TOEFL course has 200 students.

What are successful interactions? What are failed interactions?

We work backwards: courses completed, topics mastered, exercises failed and nailed. The onboarding is standard but elaborate: only 40% of users complete it. We want to see single exercises passed, concepts mastered. To typically master a word, you need 11 repetitions and 8 minutes of exposure. We are finding “distorted normal distributions” when it comes to usage. The average session length is 5 min for the free service, closer to 1 hour for the TOEFL product.

Editorial and scripting — what have you learn from flows so far?

Professional content experts build out our content. We have 2 types: 1) what we call the “conversational glue content,” which is the onboarding, hellos, and social niceties, etc. 2) then we have the instructional content, the exercises and quizzes. These two areas have different metrics: conversational content may look at “conversions into learning content” — we don’t want the glue to get in the way, so this a thin layer. For the learning content: failed/nailed rates, topics mastered, etc — and we use algorithmic optimization to serve this content. We will also license content from Oxford University Press. One learning: First we had free-form input that we handled with NLP, but that was hard on users and so we moved to buttons — it helps them control and optimize the user journey. Edwin is not an entertainment tool. The dialogues have to be straight to the point and highly structured. It’s mostly core instructional content and gets updated over time — it’s not seasonal.

What are the most common things users ask outside of the main function?

It’s pretty straightforward — people ask Edwin to translate words and phrases — they will occasionally swear and do stuff like that. But we don’t want to be a translator bot so we discourage this.

User acquisition strategy — how do people hear about your bot and start using it?

Our main paid product is in beta — most people find us via Facebook through Messenger’s discover mechanism — many find through word of month. We have built in sharing functionality. Facebook has featured Edwin as the #1 chatbot for education section in March. After we release the product, we want to use other acquisition channels — we will experiment with paid ads, and distribution partnerships.

Re-triggering and re-use strategy — as many bot developers know, how do you get initial users to engage again?

Our onboarding is light-weight and sophisticated — we show how Edwin works. We also use well-timed notifications. We remind students to get back to lessons with a push. We will also re-trigger people who bailed from offering some help. We do not want to act as attendance police. For a new student, a message of “Hey, are you ready to continue?” is easy and not invasive. Easy text with funny GIFs works best — we remind them of the features. We try to link it to their own English learning goals.

What did you learn designing Edwin’s personality profile?

We spent a lot of time thinking through Edwin’s virtual personality. All Turtles (a new bot-focused startup studio) gave a lesson on it. Personality is tough — it’s hard to nail — there are many cultural restrictions. Korea and Japan have cultural restrictions unique to them. Edwin is a European-looking guy who is a little quirky, uses emoji, not an everyday guy. We have one personality for all the markets.

Monetization? Many bots are a great free service — how have you tested monetizing it?

We have a free practice course and a paid TOEFL course in beta. We will release a TOEIC course. The paid packages take from 1 month and 3 months to complete with an expanded curriculum — we will add other international tests later. Students also want help studying for national tests for English and that could be an intermediate course subscription. We are starting with standardized tests because it is the biggest segment — lots of interest in Korea and Japan with conversational English. Also young professionals scored high in tests, but can’t speak English. They have good vocabulary and can write decently, but listening and speaking are the challenges.

What can you tell us about your tech stack? Do you do NLP in-house, what external services do you like?

Edwin lives on FB Messenger. Our back end uses AWS, Python, and Postgres RDS. We use EC2 — we have not had many problems and have a 24x7 dev-ops team. AWS does a good job on autoscaling. For NLP, we use Dialogflow — it helps for speaking practice on Google Assistant. We use Neo4j to store and operate our knowledge graph — it’s a graph of concepts, what comes first, what comes next — dependencies and so on. We optimize for serving path of content. We use PostGres for logs. Most of our learning data is in Neo4j. We also use FB analytics and our internal database. This is not speed sensitive, so we use Google Data Studio for dashboards.

Thoughts on the different platforms? FB Messenger vs Amazon and Google or Kik? Voice platforms?

Facebook Messenger and Google Assistant are the major ones — we would like to support all the major ones, like giving speaking lessons for Google Assistant. We looked into Alexa, Cortana, and Siri. Google is the best and constantly researching. FB Messenger is the biggest and sufficiently mature. Most users are in other countries, don’t have smart speakers yet.

What other bots have you looked to for inspiration — what other bots made you say “WOW”? Are there other use cases you’ve thought were simply brilliant?

We personally love the Poncho Weathercat chatbot — we love his personality. Swell and Replica.AI for in-house AI are amazing.

Other smart people in the bot world you’ve met, whether on the tech stack, UX, scripting, or even financing sides?

Phil Libin is super bright and is the guy to talk about practical AI. Evgenya and Philip of Replica AI — they are experts at NLP and have done it twice. Google’s Dialogflow team are the best NLP teams — an amazing product.

Any lessons to share with other bot-builders on useful tools?

The main lesson is to NOT overcomplicate things. We use Chatfuel for prototyping and building live experiences in minutes. We use Google docs for scripting and collaboration — there are complicated tools to do this, but simple Docs works well. We use Coggle for mind-mapping tools to design complex tree-mapping flows.

Machine learning and where this goes — building data as an asset

Edwin is like a stateless machine, and a graph structure tracks the progress of all our students. Every step a student takes affect her learning plan, so everything in Edwin lessons is built dynamically.

Read the prior articles in this series:

Woebot — Your AI Cognitive Behavioral Therapist (Alison Darcy)

X.AI’s Amy and Andrew Ingram (Diane Kim)

Rose the Loebner Chatbot Winner (Bruce Wilcox)

Poncho the WeatherCat bot (Greg Leuch)

Howdy and Botkit (Eric Soelzer)

Statsbot for Business Metrics (Artyom Keydunov)

Earplay: What Chatbots can Learn from Interactive Voice Games (Jon Myers)