Understanding NLP

Professor Regina Barzilay introduces the field of natural language processing (NLP). Professor Barzilay discusses what it means for machines to understand something, and she delves into the history of conversational devices. She explains natural language processing tasks that are considered solved (such as spam detection), tasks where progress is being made (such as sentiment analysis), and tasks that are still difficult for machines (such as question-answering systems). 

  • Intro
    • what they can do and what they cant
    • intuition of what’s possible
  • What is NLP?
    • Natural language (e.g. English, Spanish, Russian..)
    • Natural language processing (NLP) is a field of computer science, artificial intelligence concerned with the interactions between computers and human (natural) languages, and, in particular, concerned with programming computers to fruitfully process large natural language data.
  • Language technology
    • Solved
      • SPAM DETECTION
      • PARTS OF SPEECH TAGGING
      • NAMED ENTITY
    • Making Progress
      • Information 
      • SENTIMENT ANALYSIS
      • CO-REFERENCE RESOLUTION
      • Parsing
      • Word sense disambiguation
      • Machine translation
    • Still Really Hard
      • Question Answering
      • Paraphrase
      • Dialogs
  • Syntactic parsing
    • Close to solved problem
  • Machine translation (MT)
    • Think “Google Translate”
    • You need to train the system to get the results you are looking for….
  • Question answering (Q&A)
    • Think Watson
    • machines have a better ability to search enourmouse amounts of inforkmation, quickly..
    • However, doing QA based on reading a paragraph still has challenges
  • Text summarization
    • Again, machines trained on creating summaries can do a good job of creating them.
  • What NLP cant do automatically
    • Being able to take domain specific content and pulling out what’s important for summarization is still outside the realm of possibility for NLP.

Professor Barzilay describes how NLP is generally applied. She illustrates how machines can perform well on some tasks that are considered difficult for humans and perform poorly on some tasks that are considered easy for humans. She then explores the challenges associated with NLP by describing different levels of language ambiguity.

  • NLP in industry
    • Search
    • Information extraction
    • Machine translation
    • text generation
    • Sentiment analysis
  • Challenges for NLP
    • Anne Hathaway vs Berkshire Hathaway….
    • Difficulties with ambiguitiesexamples:
  • Ambiguity for NLP
    • Need to dis-disambiguate to get understanding. this is difficult.
    • Anaphora, where say a pronouns can co-refers to some other discourse entity.
    • What is a word? is it just a space, this changes across languages.

Professor Barzilay explains how NLP has evolved by looking at the different approaches to making machines understand natural language. She describes a statistical approach versus a symbolic approach, as well as recent breakthroughs in the field. Professor Barzilay then talks about an approach that uses supervised classifiers and discusses modern techniques related to deep learning that are currently dominating the NLP field. 

  • Knowledge bottleneck in NLP
    • Needs to know grammar and world around it…
    • symbolic approach: all coded in…
    • Statistical approach: give language samples…
    • Noam Chomsky – MIT prof, pioneer
      • “Colorless green ideas sleep furiously.” vs.
      • “Furiously sleep ideas green colorless”.
        • he felt…statistical approach would not find the difference…
    • “Whenever I fire a linguist, our system performance improves.” Jelenick 1998
  • Symbolic Era
    • SHRDLU, tried to encode everything in to a machine.
    • however, even the smallest of domains have too much information to capture every circumstance.
  • Statistical Era
    • DARPA developed a tree bank… took sentence and developed syntactic mapping..
    • This developed robust parsings
  • Determiner placement
    • Where to put here and there… hard for non English speakers,
  • Supervised learning in NLP
    • What if you model with supervised classifications?
    • Need to represent data as a feature vectors with + or – decisions… Supervised
  • Deep learning
    • You can use to generate the feature vectors.
    • Deep learning has enabled a revolution in NLP, and that we will be able to achieve what Turing believed in a short period of time.

Professor Barzilay tells Professor Malone that having training data of a good quality is a key enabler that allows NLP systems to work effectively. They discuss NLP as a branch of machine learning and explore the use of human-powered annotation services, staffed by workers from Amazon’s Mechanical Turk, to augment the machine learning engine.

  • Important NLP considerations
    • What kind of tolerance does customers have to system mistakes? (filtering links on Search, Google translation being good enough), not meant for legal docs, medical on its own..
    • What kind of training data do you have? The training data needs to align on the problem you are solving for. Creating training examples can be very expensive, but is very important.
  • NLP as a branch of machine learning
    • takes in unique properties of sentence structure, etc.. unique models like sequence to sequence methods.
  • Understanding features
    • It’s important that humans flag what’s important, where there may be correlations…
    • need to give knowledge as inputs to the models.
  • Annotation services
    • You can use services like mechanical turk for annotation, but probably not for domain expertise. You use them to label the data.
    • ML expert will determine how to code the data to maximize learning…

Professor Barzilay mentions two ways that natural language can be generated and discusses with Professor Malone the challenges involved in trying to get machines to use deeper, more generalized kinds of reasoning.

  • Natural Language Generation, two methods
    • Completely from scratch, training with a large amount of data.
    • Template generation, many responses, with blanks, grab the right template, and plug in the data.
  • Current limits in reasoning
    • Systems are trying to learn from observations (from watching & listening) to draw out inferences.
    • The future is learning through human observation… more abstract learning.
    • You want to be able to just tell the machine what you want
    • Today you need a lot of examples….

Casebook: Business applications of NLP

An enormous amount of data in the form of unstructured human language is currently being generated every minute through a variety of channels, such as news articles and blog posts, tweets, and posts on platforms like WhatsApp and Facebook. Business communications include those between people working together, and online interactions between companies and their customers. This module focuses on how companies can deploy natural language processing (NLP) to derive value both from understanding some of this vast amount of unstructured language and from generating natural language responses to it

https://www.techemergence.com/natural-language-processing-business-applications/

Professor Thomas Malone talks about three things that artificial intelligence programs can do with natural languages. The three functions of natural language processing are then explored through the examples that follow the video.

  1. Understand Text/Speech
  2. Generate Text/Speech
  3. Converse in Text/Speech
  • Natural language understanding
    • Call centers
      • balancing the trade off between customser services with lower costs
      • A.I. can triage callers questions, and who may be best to handle the call..
      • Three components
        • Automatic speech recognition
        • natural language processing, meaning
        • information retrieval 
      • Is this a factoid or more involved response?
      • The things it can do well, are those things it has seen before.
      • System must be trained for the specific task/problem.
      • Har
      • der questions… Those things it hasn’t seen before. Multiple responses are generate, however it’s the one with the highest confidence, above a threshold that is typically used; otherwise it goes to a human.
      • Key points and strategy
        • NLP is being used in call centers in service of two potential strategies: cost or differentiation. Text- or voice-based chatbots can do some of the work previously performed by human customer service representatives, making it possible to reduce the number of people needed, thereby lowering costs. Most companies installing customer service chatbots, however, are doing so in service of a differentiation strategy. In this case, the chatbots take care of routine matters while more complex requests are transferred to human representatives, leading to a higher quality of service.
    • Understanding documents
      • Need the representation of the language first.
      • Repetitive processes are easy. Document classification in discovery before a trial.
        • Keywords not enough
        • Machine learning let lawyers classify sample, then use that to classify documents. confidence .9 take, .4 or less reject, between human review.
      • Contract management
        • in due diligence… Look for clauses, written different ways (cancel contract if firm is acquired)
      • Key points and strategy
        • As Professor Levy explained, NLP can be used in the legal discovery process to identify responsive documents that must be turned over to the opposing party (or that clearly do not need to be turned over). The NLP system gives its confidence rating on each document, and the documents that have a high confidence rating can be automatically turned over. If the NLP system is uncertain about a document, a human can step in to make the final decision. By automating routine tasks, NLP can support a lower-cost strategy. But more often, time is freed up for higher-value tasks, supporting a law firm’s differentiation strategy to provide a higher level of service to its clients.
  • IBM Watson
    • PDF text vs image and being able to extract out letters and words to meaning (concepts…) from either.
    • Tool for compare and comply by extract meaning  between two documents, then comparing then.
    • Associate words to higher categories or concepts, can link with Wikipedia entry to get more evidence, this starts top create layers.
    • knowledge graph starts to build out insights and understanding, you don’t have to learn from scratch.
    • Can  takes different modules (skills) access via APIs to build solutions.
    • More work needs to be done in learning and reasoning.
    • Reasoning is an important issue in AI… Lots of academic and industry work is happening here.
  • Natural language generation (NLG)
    • Narrative Science
      • Take a look at data, and humanize it. From data to a narrative (description), and a piece of advise (prescription)
      • Quill is the technology
      • This is a story of focus.
    • Key points and strategy
      • Companies can use NLG to support several of Porter’s generic strategies. NLG can support a cost-leadership strategy by producing text much more inexpensively and quickly than human writers. On the other hand, NLG can also support a differentiation strategy by freeing up writers to focus on higher-value tasks. 
  • Interacting in natural language
    • x.ai
      • intelligent agent, that can schedule agents
      • Key points and strategy
        • Virtual assistants like x.ai can be used in a low-cost strategy, to reduce costs of customer interaction, or in a differentiation strategy, by freeing up staff to do higher-value work, thereby improving quality. However, some of what x.ai is doing today is still done by people.
    • Alexa
      • Amazon’s Alexa is a voice-based assistant that shows what is possible with NLP. Users can speak out loud into Amazon’s Echo speaker to ask Alexa to perform any number of a growing number of tasks, such as to tell a joke, play a song, give the weather forecast, lead an exercise routine, or order products on Amazon. Google, Microsoft, Apple, and China’s Baidu also have voice-based assistants of this kind. Amazon can also mine the queries that Alexa cannot yet understand or satisfy. If enough people ask for sports scores, for example, it is likely that this kind of functionality would be popular, and Amazon may decide to build it. Amazon also lets other companies such as Uber, Fitbit, 1-800-Flowers, and Campbell Soup build apps on the Alexa platform. As of June 2017, Alexa had 15,000 such apps, although many of them remain quite simple.
  • Interacting in natural language (continued)
    • Key points and strategy
      • As this example shows, NLP can be used to support a strategy of differentiation, whereby companies that build apps on Alexa make it easier and more convenient for their customers, thus increasing the quality of the goods or services they provide.
    • It’s a method called wavenet, discovered by researchers at Google’s DeepMind and published as a research paper in September 2016. The method uses a particular type of neural-network architecture to create sound, and is said to represent a significant leap forward in artificial-voice technology. Voysis
  • As a technology, natural language processing is making daily life easier by automating tasks, sorting through data with a level of speed and accuracy that humans are not capable of, and making connections between data to enhance and personalize the online experience. It is being applied to a wide range of business problems to deliver tangible business value and will continue to transform how people live and work.

Module Artifacts:

MIT AI M3U1 Video 1 Transcript.pdfMIT AI M3U1 Video 2 Transcript.pdfMIT AI M3U1 Video 3 Transcript.pdfMIT AI M3U1 Video 4 Transcript.pdfMIT AI M3U1 Video 5 Transcript.pdfMIT AI M3U2 Casebook Video 1 Transcript.pdfMIT AI M3U2 Casebook Video 2 Transcript.pdfMIT AI M3U2 Casebook Video 3 Transcript.pdfMIT AI_M3 U2 Casebook.pdfMIT AI_M3 U3 Assignment.docx