Smart Reply: Automated Response Suggestion for

of 11

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
PDF
11 pages
0 downs
4 views
Share
Description
Smart Reply: Automated Response Suggestion for Authors Anjuli Kannan* Karol Kurach* Sujith Ravi* Tobias Kaufmann* Andrew Tomkins Balint Miklos Greg Corrado László Lukács Marina Ganea Peter Young
Transcript
Smart Reply: Automated Response Suggestion for Authors Anjuli Kannan* Karol Kurach* Sujith Ravi* Tobias Kaufmann* Andrew Tomkins Balint Miklos Greg Corrado László Lukács Marina Ganea Peter Young Vivek Ramavajjala * Google Employees Presenter Andrew Janick Why Smart Reply? o continues to be a primary medium for communication for billions of users across the world o It is increasingly challenging for users to process and respond to incoming messages especially on mobile devices o Roughly 25% of replies have 20 or less tokens (words and punctuation) The Authors Question: Is it possible to assist users in composing short replies to messages when appropriate that they can use in one tap on a mobile device? 2 What Is Smart Reply? Abstract o Generates semantically diverse suggestions that can be used as complete responses with just one tap on mobile o Currently used in Inbox by Gmail o Responsible for assisting with 10% of all mobile responses o Designed to work at very high throughput and process hundreds of millions of messages daily o System exploits state-of-the-art, large-scale deep learning 3 The Smart Reply Flow 4 The Smart Reply Flow: Preprocess Language detection - The language of the message is identified and non-english messages are discarded. Tokenization - Subject and message body are broken into words and punctuation marks. Sentence segmentation - Sentences boundaries are identified in the message body. Normalization - Infrequent words and entities like personal names, URLs, addresses, phone numbers etc. are replaced by special tokens. Quotation removal - Quoted original messages and forwarded messages are removed. Salutation/close removal - Salutations like Hi John and closes such as Best regards, Mary are removed. 5 The Smart Reply Flow: Trigger Response? Two requirements: 1. Needs to be fast 100s of millions of message means decision needs to happen in milliseconds 2. Figure out cases where a response is not expected to save resources on the calls to the LSTM inference and to not provide useless responses Uses a feedforward network - the first and simplest type of artificial neural network devised. In this network, the information moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network. Probability score the feedforward network produces a probability score that is compared to a threshold. Above threshold = Yes/Below threshold = No Trigger response says Yes only 11% of s get Smart Reply suggestions 6 The Smart Reply Flow - LSTM Long short-term memory (LSTM) 1. A recurrent neural network (RNN) architecture (an artificial neural network) 2. Has loops and can reprocess errors and data to form cause and effect relationships 3. Has persistent data so is learning as time passes Response tokens to the original message get scores the most likely response tokens are fed back in and rescored until the best responses are found. Response set is generated a few million unique sentences 7 The Smart Reply Flow - LSTM Challenges: 1. need quality responses (no bad grammar, spelling, or mechanics) 2. Need responses with a large amount of utility specific to original message 3. Model needs to scale due to number of messages Responses are clustered broken down into clusters based on intent. Thank you versus sorry versus lol all seeds in different clusters. 100 clusters and on average 3 to 5 labeled seed examples per cluster. 8 The Smart Reply Flow Diversity selection Remove redundant responses Iterate over the clusters until the resulting list contains only the highest-scored representation of each intent in order Enforce negative and positive responses LSTM has a strong tendency to produce positive responses (probably due to the fact that most conversations are positive). A second LSTM pass is performed limited to negative responses. 9 The Smart Reply Flow Can you join tomorrow s meeting? 10 Conclusion Smart reply an end-to-end system that generates short, complete response using a deep LSTM model that accurately predicts valid responses. Challenges met: 1. Responses are of high quality 2. Enforcing diversity has given the responses high utility 3. Efficient searching has made the solution scalable Best metric of success: 10% of mobile replies in Inbox are now composed with assistance from the Smart Reply system 11
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks