↓Skip to main content
  1. Data Science Blog/

Topic Modeling with BERT

·225 words·2 mins· loading · ·
Natural Language Processing (NLP) Data Analysis & Visualization AI/ML Models Language Models (LLMs) NLP Applications Text Analysis Natural Language Processing (NLP) Machine Learning Text Mining

Topic Modeling with BERT

Topic Modeling with BERT
#

Key steps in BERTopic modelling are as following.

  • Use “Sentence Embedding” models to embed the sentences of the article
  • Reduce the dimensionality of embedding using UMAP
  • Cluster these documents (reduced dimensions) using HDBSAN
  • Use c-TF-IDF extract keywords, their frequency and IDF for each cluster.
  • MMR: Maximize Candidate Relevance. How many words in a topic can represent the topic?
  • Intertopic Distance Map
  • Use similarity matrix (heatmap), dandogram (hierarchical map), to visualize the topics and key_words.
  • Traction of topic over time period. Some may be irrelevant and for other traction may be increasing or decreasing.

Installation
#

# Installation, with sentence-transformers, can be done using pypi:

pip install bertopic

# If you want to install BERTopic with other embedding models, you can choose one of the following:

# Choose an embedding backend
pip install bertopic[flair, gensim, spacy, use]

# Topic modeling with images
pip install bertopic[vision]

Supported Topic Modelling Techniques
#

BERTopic supports all kinds of topic modeling techniques as below.

  • Guided
  • Supervised
  • Semi-supervised
  • Manual
  • Multi-topic distributions
  • Hierarchical
  • Class-based
  • Dynamic
  • Online/Incremental
  • Multimodal
  • Multi-aspect
  • Text Generation/LLM
  • Merge Models

Related Resources#

Tools in BERTopic
#

Tools-in-BERTopic

Best Topic Modeling Tool in BERTopic
#

BEST-Tools-in-BERTopic

BERTopic Model Building
#

BERTopic-Model-Building

Application
#

  • arXiv Dataset (1.7m+ STEP papers)
  • Images/photographs
  • Historical Documents
  • News articles

Related

Understanding Batteries: Sizes, Chemistry, Voltage, Capacity and Lifespan
·2194 words·11 mins· loading
Physics Technology Trends & Future Batteries Electrochemistry Energy Electrical Engineering Safety
Understanding Batteries: Sizes, Chemistry, Voltage, Capacity and Lifespan # A practical guide to …
From Geometry to Quantum Mechanics
·1029 words·5 mins· loading
Mathematics Research & Academia Geometry Quantum Mechanics Quantum Computing Linear Algebra Mathematics
From Geometry to Quantum Mechanics # Part 26 of the Learning Quantum Physics series. See the same …
Operators in Quantum Mechanics
·3249 words·16 mins· loading
Mathematics Research & Academia Quantum Mechanics Quantum Computing Linear Algebra Mathematics Pauli Matrices
Operators in Quantum Mechanics # Part 27 of the Learning Quantum Physics series. An operator takes …
How Many Numbers Does a Qubit Need?
·3788 words·18 mins· loading
Mathematics Research & Academia Quantum Computing Quantum Mechanics Mathematics Linear Algebra Machine Learning
How Many Numbers Does a Qubit Need? # You already know how to place a point. On a plane you need …
Opening a Power: Where the Coefficients Come From
·1591 words·8 mins· loading
Mathematics Research & Academia Mathematics Mathematics for Machine Learning Mathematics for AI Applied Mathematics Algebra
Opening a Power: Where the Coefficients Come From # A power such as $(x+y)^{55}$ is 55 factors of …