Writing quality content and meta data at scale is a big problem for most enterprise sites. In this webinar we are going to explore what is possible given the latest advances in deep learning and natural language processing.Our main focus is going to be about generating metadata: titles, meta descriptions, h1s, etc that are critical for technical SEO performance. But, we will cover full article generation as well.
5. @hamletbatista
LET’S FIX THAT WITH AUTOMATION!
We want to address 4 scenarios common in enterprise websites.
For large ecommerce sites, we will focus on:
1. Pages with large images and no text.
2. Pages with large images and some text.
For large web publishers, we will focus on:
1. Pages with a lot of quality text and no metadata.
2. Pages with very little text.
6. @hamletbatista
AGENDA
We are going to explore different text generation strategies and
recommend the best one for each problem.
Specifically, we will cover:
1. Image captioning
2. Visual question and answering
3. Text summarization
4. Question and answering from text (short answers)
5. Long-form question and answering
6. Full article generation
7. @hamletbatista
AGENDA
We are going to build two models from scratch:
1. We will build an image captioning and visual question and answering
model
2. We will also build a state of the art text summarization model
At the end, I will share some resources to learn more about these topics.
11. @hamletbatista
IMAGE CAPTIONING AND VISUAL QUESTION ANSWERING
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g and
Visual
Question
Answering
22. @hamletbatista
YOU CAN ALSO ASK QUESTIONS ABOUT IMAGES
“what are
these
people
riding?”
“boat with
99.96
confidenc
e”
23. @hamletbatista
CAPTIONING AND VISUAL QUESTION ANSWERING PAPER
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g and
Visual
Question
Answerin
g
24. @hamletbatista
CAPTIONING AND VISUAL QUESTION ANSWERING
RESULTS
Bottom-
Up and
Top-
Down
Attention
for Image
Captionin
g and
Visual
Question
Answerin
g
36. @hamletbatista
BERTSUM MODEL OVERVIEW
"Meanwhile,
although BERT has
segmentation
embeddings for
indicating different
sentences, it only
has
two labels (sentence
A or sentence B),
instead of
multiple sentences
as in extractive
summarization.
Therefore, we
modify the input
sequence and
embeddings of BERT
to make it possible
46. @hamletbatista
BERTSUM TESTING RESULTS
BERTSUM Colab
notebook
Gold Summary: 'click on the brilliant
interactive graphic below for details on each
hole of the masters 2015 course',
Candidate Summary after 50,000 training
steps: 'click on the graphic below to get a
closer look at what the biggest names in the
game will face when they tee off on thursday
.',
58. @hamletbatista
GENERATING WIKIPEDIA BY SUMMARIZING LONG
SEQUENCES
CONCLUSION
“We have shown that generating Wikipedia can be approached as a
multi-document summarization
problem with a large, parallel dataset, and demonstrated a two-stage
extractive-abstractive framework for carrying it out. The coarse
extraction method used in the first stage appears to have a significant
effect on final performance, suggesting further research on improving it
would be fruitful.
We introduce a new, decoder-only sequence transduction model for the
abstractive stage, capable of
handling very long input-output examples. This model significantly
outperforms traditional encoder/decoder architectures on long
sequences, allowing us to condition on many reference documents and
to generate coherent and informative Wikipedia articles.”
Generatin
g
Wikipedia
by
Summarizi
ng Long
Sequences
61. @hamletbatista
RESOURCES TO LEARN MORE
Faster Data Science Education
https://www.kaggle.com/learn/overview
Data Scientist’s Guide to Summarization
https://towardsdatascience.com/data-scientists-guide-to-summarization-fc0db952e363
An open source neural machine translation system
http://opennmt.net/
Bottom-Up Abstractive Summarization
http://opennmt.net/OpenNMT-py/Summarization.html
Abstractive Text Summarization (tutorial 2) , Text Representation made very easy
https://hackernoon.com/abstractive-text-summarization-tutorial-2-text-representation-made-very-easy-ef4511a1a46
62. @hamletbatista
RESOURCES TO LEARN MORE
Build an Abstractive Text Summarizer in 94 Lines of Tensorflow !! (Tutorial 6)
https://hackernoon.com/build-an-abstractive-text-summarizer-in-94-lines-of-tensorflow-tutorial-6-f0e1b4d88b55
What Is ROUGE And How It Works For Evaluation Of Summarization Tasks?
https://rxnlp.com/how-rouge-works-for-evaluation-of-summarization-tasks/
Introducing Eli5: How Facebook is Tackling Long-Form Question-Answering Conversations
https://towardsdatascience.com/introducing-eli5-how-facebook-is-tackling-long-form-question-answering-
conversations-4f8e59374717
Pythia’s Documentation
https://learnpythia.readthedocs.io/en/latest/
Hinweis der Redaktion
Writing quality content and meta data at scale is a big problem for most enterprise sites. In this webinar we are going to explore what is possible given the latest advances in deep learning and natural language processing.
Our main focus is going to be about generating metadata: titles, meta descriptions, h1s, etc that are critical for technical SEO performance. But, we will cover full article generation as well.
I will also cover the concepts you need to understand to get practical value out of these advanced techniques.
I will also cover the concepts you need to understand to get practical value out of these advanced techniques.
I love the site Papers with Code. It has a clearly organized and frequently updated list of the latest deep learning papers that include code to reproduce their results.
Feel free to browse the SOTA (state of the art section) that has many of the best papers. That is where we found several of the examples we will be reviewing Today.
When it comes to generating quality metadata for web publisher sites, we are mostly talking about article pages. Let’s explore two examples: one with a lot of text, and another with very little text.
When it comes to generating metadata text and there is a lot of it, the most appropriate approach is text summarization. We have two types of automated text summarization techniques: extractive and abstractive.
Extractive copies the most relevant sentences in the text and abstractive generates new sentences.
The paper I used for this example is this one: Fine-tune BERT for Extractive Summarization by Yang Liu. Not only he shared the code needed to reproduce his paper results, he emailed me a trained model when I asked. I found his paper on the Papers with code website.
We will walk step by step on how to put this code to practical use for our problem.
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
You can copy my notebook and follow my steps
When it comes to generating quality metadata for web publisher sites, we are mostly talking about article pages. Let’s explore two examples: one with a lot of text, and another with very little text.
When it comes to generating metadata text and there is a lot of it, the most appropriate approach is text summarization. We have two types of automated text summarization techniques: extractive and abstractive.
Extractive copies the most relevant sentences in the text and abstractive generates new sentences.