Overview

This project analyzes New York City Airbnb listings to understand the factors that influence listing prices, predict property type, and identify natural market segments within the short-term rental market. Using regression, classification, and clustering techniques, the project connects statistical modeling with practical business insights for pricing, customer targeting, and market positioning.

Business Questions

1. Which listing characteristics are strongly associated with Airbnb prices?
2. Can listing type be predicted using price and availability?
3. Are there distinct market segments within the NYC Airbnb market?

Dataset

The analysis used NYC Airbnb listing data containing information on price, room type, neighborhood group, availability, reviews, geographic location, and listing names.

Methods

I cleaned the data by removing invalid prices, filtering extreme outliers, replacing missing review values, and log-transforming price to reduce skewness. I also used Exploratory analysis to examine price distributions, missing values, outliers, and common words in listing titles.

Three main approaches were applied:

• Regression modeling to predict listing price
• Classification modeling to predict whether a listing was an entire home/apartment
• Clustering to identify market segments based on price and location

Key Findings

Room type and neighborhood were strong predictors of Airbnb price. Entire homes and apartments were generally more expensive than private or shared rooms, while Manhattan listings showed a clear price premium compared with other boroughs.

The multiple regression model improved price prediction compared with the simple model, increasing adjusted R² from about 0.41 to 0.48 after adding neighborhood group and review activity.

For classification, logistic regression and Linear Discriminant Analysis both performed well. Logistic regression achieved approximately 82% accuracy, while LDA achieved approximately 81% accuracy, showing that price and availability can help distinguish entire homes/apartments from other listing types.

The clustering analysis identified four market segments within the NYC Airbnb market. These ranged from higher-priced listings concentrated around Manhattan and North Brooklyn to more affordable listings in areas farther from the city center.

Tools Used

R, tidyverse, ggplot2, tidytext, MASS, cluster, factoextra, stargazer

Skills Demonstrated

Data cleaning, exploratory data analysis, feature engineering, regression modeling, classification, clustering, text analysis, data visualization, and business interpretation.

Conclusion

This project shows how data science techniques can be used to understand pricing behavior and market structure in the short-term rental industry. The analysis highlights the importance of room type and location in Airbnb pricing and demonstrates how predictive modeling and clustering can support more informed pricing and market segmentation strategies.

Technical Implementation

# Load packages
library(tidyverse)
library(stargazer)
library(MASS)
library(cluster)
library(factoextra)
library(gridExtra)
library(tidytext)

# Load data
nyc_data <- read.csv("AB_NYC.csv")

# Inspect price outliers
boxplot(nyc_data$price,
        main = "Boxplot of NYC Airbnb Prices",
        ylab = "Price ($)")

outliers <- boxplot.stats(nyc_data$price)$out
length(outliers)
max(nyc_data$price)

# Clean and prepare data
nyc_clean <- nyc_data %>%
  mutate(reviews_per_month = replace_na(reviews_per_month, 0)) %>%
  filter(price > 0, price < 1000) %>%
  mutate(
    log_price = log(price),
    neighbourhood_group = as.factor(neighbourhood_group),
    room_type = as.factor(room_type)
  )

# Text analysis of listing names
nyc_clean %>%
  unnest_tokens(word, name) %>%
  anti_join(stop_words, by = "word") %>%
  filter(!word %in% c("nyc", "apartment", "room", "bedroom")) %>%
  count(word, sort = TRUE) %>%
  slice_head(n = 10) %>%
  ggplot(aes(x = reorder(word, n), y = n)) +
  geom_col() +
  coord_flip() +
  labs(
    title = "Top 10 Words in Airbnb Listing Titles",
    x = "Word",
    y = "Frequency"
  ) +
  theme_minimal()

# Regression models
model_1 <- lm(log_price ~ room_type, data = nyc_clean)

model_2 <- lm(
  log_price ~ room_type + neighbourhood_group + reviews_per_month,
  data = nyc_clean
)

stargazer(model_1, model_2, type = "text")

# Classification: entire home/apartment vs other listing types
nyc_clean <- nyc_clean %>%
  mutate(entire_home = ifelse(room_type == "Entire home/apt", 1, 0))

logit_fit <- glm(
  entire_home ~ log_price + availability_365,
  data = nyc_clean,
  family = "binomial"
)

lda_fit <- lda(
  entire_home ~ log_price + availability_365,
  data = nyc_clean
)

# Model accuracy
logit_pred <- ifelse(predict(logit_fit, type = "response") > 0.5, 1, 0)
logit_accuracy <- mean(logit_pred == nyc_clean$entire_home)

lda_pred <- predict(lda_fit)$class
lda_accuracy <- mean(lda_pred == nyc_clean$entire_home)

logit_accuracy
lda_accuracy

# Confusion matrices
conf_logit <- table(
  Predicted = logit_pred,
  Actual = nyc_clean$entire_home
)

conf_lda <- table(
  Predicted = lda_pred,
  Actual = nyc_clean$entire_home
)

conf_logit
conf_lda

# Clustering
cluster_data <- nyc_clean %>%
  select(latitude, longitude, price) %>%
  scale()

# Elbow method using a random sample
set.seed(123)

cluster_sample <- cluster_data[
  sample(nrow(cluster_data), 5000),
]

elbow_plot <- fviz_nbclust(
  cluster_sample,
  kmeans,
  method = "wss"
) +
  labs(title = "Elbow Method for Selecting Number of Clusters") +
  geom_vline(xintercept = 4, linetype = 2)

# K-means clustering
set.seed(123)

kmeans_four <- kmeans(
  cluster_data,
  centers = 4,
  nstart = 25
)

# Visualize clusters
cluster_plot <- fviz_cluster(
  kmeans_four,
  data = cluster_data,
  geom = "point",
  ellipse.type = "convex",
  choose.vars = c("latitude", "longitude"),
  stand = FALSE,
  show.clust.cent = TRUE,
  sample = 2000
) +
  labs(title = "NYC Airbnb Market Segments, K = 4")

grid.arrange(elbow_plot, cluster_plot, ncol = 2)