Splitting text into words or subword pieces is called what?
AnswerTokenization
Tokenization breaks text into units a model can process.
AI · Facts & stats
7 fact-checked facts about natural language processing and how computers understand text, each with the reason behind it. 5 more are in today's round and join this page after it closes.
AnswerTokenization
Tokenization breaks text into units a model can process.
AnswerThe Turing Test
Alan Turing proposed the imitation game in 1950.
AnswerJoseph Weizenbaum
Joseph Weizenbaum built ELIZA at MIT in the mid-1960s.
AnswerReduces words to a root form
Stemming turns words like running and runs toward the stem run.
AnswerGoogle
Google researchers released BERT in 2018.
AnswerBidirectional Encoder Representations from Transformers
BERT reads context from both directions using Transformer encoders.
AnswerA Rogerian psychotherapist
The DOCTOR script reflected users' statements back as questions.