Week 8: Word Embeddings

Representing words as dense vectors in continuous space — capturing semantic relationships that bag-of-words approaches miss.

Learning Objectives

By the end of this week, you will be able to:

Explain the intuition behind word embeddings and how they differ from count-based representations
Train word embeddings from scratch on a domain-specific corpus using your Nexis Uni data
Load and query pre-trained GloVe embeddings to explore semantic similarity and analogy tasks
Compare corpus-trained embeddings to general-purpose pre-trained embeddings

Lecture Materials

Slides

🖥 Slide deck from this week’s lecture

View Slides

Code & Data

💻 R scripts used during class:

Server Version Local Version

No server access? Download the GloVe embeddings file and run the local version above.

Download GloVe Embeddings

📂 Assignment

Lab 6 — Word Embeddings Graded · 11 pts

Due: May 26 @ 11:59 pm

In this lab you will train word embeddings directly from your Nexis Uni corpus and compare them against pre-trained GloVe embeddings. You will query nearest neighbors, explore how domain-specific training shapes the embedding space, and visualize semantic relationships in your environmental topic area.

📚 Additional Resources

Type	Resource
Interactive	Word2Vec Galaxy — interactive 3D visualization of word embedding space

Word Embeddings in Environmental Science — Key Citations

Transformer-Based & BERT Approaches

Citation	Year	Topic	Keywords
Callaghan et al. — Nature Climate Change	2021	Evidence & Attribution Mapping	BERT, 100k+ studies, climate impacts, geospatial attribution, evidence synthesis
Bingler et al. — ClimateBERT	2022	Domain-Adapted Climate Language Model	BERT fine-tuning, climate narratives, ESG reports, sustainability disclosure, NLP

Word2Vec & Concept Mapping

Citation	Year	Topic	Keywords
Authors — Climate Knowledge or Climate Debate?	—	Climate Discourse Analysis	Word2Vec, semantic shift, expert vs. media framing, ideological variation, vector distance
Authors — Using Word Embeddings to Learn a Better Food Ontology	—	Environmental Lexicon Expansion	geotagged social media, food systems, land use, co-occurrence, ontology learning

Spatiotemporal Embeddings & Ecology

Citation	Year	Topic	Keywords
Jeawak et al. — Ecological Informatics	—	Predicting Environmental Features	spatiotemporal embeddings, social media, species distributions, localized climate features, geotext

Environmental Policy & Compliance

Citation	Year	Topic	Keywords
Authors — Using word embedding for environmental violation analysis	—	Oil & Gas Compliance	word embeddings, violation text, enforcement trends, shale gas, semantic distance, regulatory NLP

← Week 7: Text Classification