API Catalog Maintenance Patterns

How APIs.guru Processes Its Growing Database of API Specifications

TidyTuesday
2025
A TidyTuesday analysis examining catalog maintenance patterns on APIs.guru, revealing that 40% of APIs receive same-day processing when added to the database. The visualization explores how different providers’ APIs are managed in the catalog system and highlights the efficiency of APIs.guru’s internal processing workflows.
Author

Steven Ponce

Published

June 17, 2025

Modified

June 19, 2025

Figure 1: Two-panel chart analyzing APIs.guru catalog patterns from 2016-2023. Top: scatter plot with log scale y-axis showing days between addition and update, 1,011 red dots at 0.5 days (same-day processing), 1,518 gray dots scattered above (later updates). Bottom: horizontal bars ranking 15 providers by same-day processing percentage.

Steps to Create this Graphic

1. Load Packages & Setup

Show code
```{r}
#| label: load
#| warning: false
#| message: false
#| results: "hide"

## 1. LOAD PACKAGES & SETUP ----
suppressPackageStartupMessages({
if (!require("pacman")) install.packages("pacman")
pacman::p_load(
  tidyverse,  # Easily Install and Load the 'Tidyverse'
  ggtext,     # Improved Text Rendering Support for 'ggplot2'
  showtext,   # Using Fonts More Easily in R Graphs
  janitor,    # Simple Tools for Examining and Cleaning Dirty Data
  scales,     # Scale Functions for Visualization
  glue,       # Interpreted String Literals
  patchwork   # The Composer of Plots
  )})

### |- figure size ----
camcorder::gg_record(
  dir    = here::here("temp_plots"),
  device = "png",
  width  =  8,
  height =  10,
  units  = "in",
  dpi    = 320
)

# Source utility functions
suppressMessages(source(here::here("R/utils/fonts.R")))
source(here::here("R/utils/social_icons.R"))
source(here::here("R/utils/image_utils.R"))
source(here::here("R/themes/base_theme.R"))
```

2. Read in the Data

Show code
```{r}
#| label: read
#| include: true
#| eval: true
#| warning: false

tt <- tidytuesdayR::tt_load(2025, week = 24)

info_raw <- tt$api_info |> clean_names()
apis_raw <- tt$apisguru_apis |> clean_names()

tidytuesdayR::readme(tt)
rm(tt)
```

3. Examine the Data

Show code
```{r}
#| label: examine
#| include: true
#| eval: true
#| results: 'hide'
#| warning: false

glimpse(info_raw)
glimpse(apis_raw)
```

4. Tidy Data

Show code
```{r}
#| label: tidy
#| warning: false

### |-  tidy data ----
apis_minimal <- apis_raw |>
  left_join(info_raw, by = "name") |>
  select(name, added, updated, provider_name) |>
  mutate(
    added_date = as_date(added),
    updated_date = as_date(updated)
  ) |>
  filter(!is.na(added_date), !is.na(updated_date))

catalog_analysis <- apis_minimal |>
  mutate(
    days_between_add_update = as.numeric(updated_date - added_date),
    catalog_update_type = case_when(
      days_between_add_update == 0 ~ "Same-day catalog processing",
      days_between_add_update <= 7 ~ "Updated within week",
      days_between_add_update <= 30 ~ "Updated within month",
      days_between_add_update <= 365 ~ "Updated within year",
      days_between_add_update > 365 ~ "Long gap before catalog update",
      TRUE ~ "Other"
    ),
    same_day_processing = days_between_add_update == 0,
    days_for_viz = ifelse(days_between_add_update == 0, 0.5, days_between_add_update)
  ) |>
  filter(days_between_add_update >= 0)

# Calculate key statistics
total_apis <- nrow(catalog_analysis)
same_day_count <- sum(catalog_analysis$same_day_processing)
same_day_pct <- round(same_day_count / total_apis * 100, 1)
later_updated_count <- total_apis - same_day_count
later_updated_pct <- round(later_updated_count / total_apis * 100, 1)

# P2. Provider Analysis ----
provider_catalog_analysis <- apis_minimal |>
  mutate(
    days_between_add_update = as.numeric(updated_date - added_date),
    same_day_processing = days_between_add_update == 0
  ) |>
  filter(days_between_add_update >= 0) |>
  group_by(provider_name) |>
  filter(n() >= 3) |>
  summarise(
    total_apis = n(),
    same_day_apis = sum(same_day_processing),
    same_day_rate = round((same_day_apis / total_apis) * 100, 1),
    avg_days_to_update = round(mean(days_between_add_update), 1),
    .groups = "drop"
  ) |>
  filter(same_day_rate > 0) |>
  arrange(desc(same_day_rate)) |>
  head(15) |> # Top 15 by same-day processing rate
  mutate(
    provider_clean = str_trunc(str_to_title(provider_name), 35),
    provider_clean = fct_reorder(provider_clean, same_day_rate)
  )
```

5. Visualization Parameters

Show code
```{r}
#| label: params
#| include: true
#| warning: false

### |-  plot aesthetics ----
colors <- get_theme_colors(
  palette = c("#95A5A6", "#E74C3C", "#7F8C8D")
)

### |-  titles and caption ----
caption_text <- create_social_caption(
  tt_year = 2025,
  tt_week = 24,
  source_text =  "APIs.guru"
)

### |-  fonts ----
setup_fonts()
fonts <- get_font_families()

### |-  plot theme ----

# Start with base theme
base_theme <- create_base_theme(colors)

# Add weekly-specific theme elements
weekly_theme <- extend_weekly_theme(
  base_theme,
  theme(
    # Axis elements
    axis.text = element_text(color = colors$text, size = rel(0.7)),
    axis.title.x = element_text(color = colors$text, face = "bold", size = rel(0.8), margin = margin(t = 15)),
    axis.title.y = element_text(color = colors$text, face = "bold", size = rel(0.8), margin = margin(r = 10)),

    # Grid elements
    panel.grid.major.y = element_line(color = "gray50", linewidth = 0.05),
    panel.grid.minor = element_blank(),
    panel.grid.major.x = element_blank(),

    # Legend elements
    legend.position = "plot",
    legend.title = element_text(family = fonts$tsubtitle, color = colors$text, size = rel(0.8), face = "bold"),
    legend.text = element_text(family = fonts$tsubtitle, color = colors$text, size = rel(0.7)),

    # Plot margins
    plot.margin = margin(t = 15, r = 15, b = 15, l = 15),
  )
)

# Set theme
theme_set(weekly_theme)
```

6. Plot

Show code
```{r}
#| label: plot
#| warning: false

# P1. The API Graveyard Plot ----
p1 <- catalog_analysis |>
  ggplot(aes(x = added_date, y = days_for_viz)) +
  # Geoms
  geom_point(
    data = catalog_analysis |> filter(!same_day_processing),
    color = colors$palette[1], size = 1.5, alpha = 0.6
  ) +
  geom_point(
    data = catalog_analysis |> filter(same_day_processing),
    color = colors$palette[2], size = 2.5, alpha = 0.9
  ) +
  # Scales
  scale_y_log10(
    labels = comma_format(),
    limits = c(0.1, NA),
    breaks = c(0.5, 1, 10, 100, 1000)
  ) +
  # Annotations
  annotate("text",
    x = as.Date("2016-06-01"), y = 12,
    label = paste0("Gray dots: Updated later in catalog\n(", later_updated_count, " APIs • ", later_updated_pct, "%)"),
    color = colors$palette[1], size = 3.5, fontface = "bold",
    hjust = 0, vjust = 0
  ) +
  annotate("text",
    x = as.Date("2016-06-01"), y = 0.9,
    label = paste0("Red dots: Same-day catalog processing\n(", same_day_count, " APIs • ", same_day_pct, "%)"),
    color = colors$palette[2], size = 3.5, fontface = "bold",
    hjust = 0, vjust = 0
  ) +
  # Labs
  labs(
    title = "**API Catalog Maintenance Patterns**",
    subtitle = paste0(
      "**", same_day_pct, "%** of APIs have <span style='color:#E74C3C'>**same-day catalog processing**</span> when added to APIs.guru<br><br>",
      "<span style='color:#555555; font-size:10px'>**Days Between Addition and Catalog Update (log scale)**</span>"
    ),
    x = "Date Added to APIs.guru",
    y = NULL,
  ) +
  # Theme
  theme(
    plot.title = element_markdown(
      size = rel(1.6),
      family = fonts$title,
      face = "bold",
      color = colors$title,
      lineheight = 1.1,
      margin = margin(t = 0, b = 5)
    ),
    plot.subtitle = element_markdown(
      size = rel(0.95),
      family = fonts$subtitle,
      color = alpha(colors$subtitle, 0.9),
      lineheight = 1.2,
      margin = margin(t = 5, b = 10)
    ),
    panel.grid.minor = element_blank(),
    legend.position = "none"
  )

# P2. Provider Responsibility Breakdown Plot ----
p2 <- provider_catalog_analysis |>
  ggplot(aes(x = provider_clean, y = same_day_rate)) +
  # Geoms
  geom_col(fill = colors$palette[2], alpha = 0.7, width = 0.7) +
  geom_text(aes(label = paste0(same_day_rate, "% • (", same_day_apis, "/", total_apis, ")")),
    hjust = -0.1, size = 3, color = "gray50", fontface = "bold"
  ) +
  # Scales
  scale_y_continuous(
    limits = c(0, 124),
    expand = c(0, 0)
  ) +
  coord_flip() +
  # Labs
  labs(
    title = "**Catalog Processing Patterns by Provider**",
    subtitle = "Providers with highest rates of same-day catalog processing on APIs.guru",
    x = NULL,
    y = "% of APIs with Same-Day Catalog Processing",
  ) +
  # Theme
  theme(
    plot.title = element_markdown(
      size = rel(1.6),
      family = fonts$title,
      face = "bold",
      color = colors$title,
      lineheight = 1.1,
      margin = margin(t = 5, b = 5)
    ),
    plot.subtitle = element_markdown(
      size = rel(0.95),
      family = fonts$subtitle,
      color = alpha(colors$subtitle, 0.9),
      lineheight = 1.2,
      margin = margin(t = 5, b = 10)
    ),
    panel.grid.major.y = element_blank(),
    panel.grid.minor = element_blank(),
    axis.text.y = element_text(size = 9),
    plot.caption = element_text(size = 8, color = colors$palette[3])
  )

# Final plot -----
combined_plot <- p1 / p2 +
  plot_layout(heights = c(1.2, 1))

combined_plot <- combined_plot +
  plot_annotation(
    caption = caption_text,
    theme = theme(
      plot.caption = element_markdown(
        size = rel(0.6),
        family = fonts$caption,
        color = colors$caption,
        hjust = 0.5,
        margin = margin(t = 10)
      )
    )
  )
```

7. Save

Show code
```{r}
#| label: save
#| warning: false

### |-  plot image ----  
save_plot_patchwork(
  plot = combined_plot, 
  type = "tidytuesday", 
  year = 2025, 
  week = 24, 
  width  =  8,
  height =  10
)
```

8. Session Info

R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22631)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] here_1.0.1      patchwork_1.3.0 glue_1.8.0      scales_1.3.0   
 [5] janitor_2.2.0   showtext_0.9-7  showtextdb_3.0  sysfonts_0.8.9 
 [9] ggtext_0.1.2    lubridate_1.9.3 forcats_1.0.0   stringr_1.5.1  
[13] dplyr_1.1.4     purrr_1.0.2     readr_2.1.5     tidyr_1.3.1    
[17] tibble_3.2.1    ggplot2_3.5.1   tidyverse_2.0.0 pacman_0.5.1   

loaded via a namespace (and not attached):
 [1] gtable_0.3.6       httr2_1.0.6        xfun_0.49          htmlwidgets_1.6.4 
 [5] gh_1.4.1           tzdb_0.5.0         yulab.utils_0.1.8  vctrs_0.6.5       
 [9] tools_4.4.0        generics_0.1.3     parallel_4.4.0     curl_6.0.0        
[13] gifski_1.32.0-1    fansi_1.0.6        pkgconfig_2.0.3    ggplotify_0.1.2   
[17] lifecycle_1.0.4    compiler_4.4.0     farver_2.1.2       munsell_0.5.1     
[21] codetools_0.2-20   snakecase_0.11.1   htmltools_0.5.8.1  yaml_2.3.10       
[25] crayon_1.5.3       pillar_1.9.0       camcorder_0.1.0    magick_2.8.5      
[29] commonmark_1.9.2   tidyselect_1.2.1   digest_0.6.37      stringi_1.8.4     
[33] labeling_0.4.3     rsvg_2.6.1         rprojroot_2.0.4    fastmap_1.2.0     
[37] grid_4.4.0         colorspace_2.1-1   cli_3.6.4          magrittr_2.0.3    
[41] utf8_1.2.4         withr_3.0.2        rappdirs_0.3.3     bit64_4.5.2       
[45] timechange_0.3.0   rmarkdown_2.29     tidytuesdayR_1.1.2 gitcreds_0.1.2    
[49] bit_4.5.0          hms_1.1.3          evaluate_1.0.1     knitr_1.49        
[53] markdown_1.13      gridGraphics_0.5-1 rlang_1.1.6        gridtext_0.1.5    
[57] Rcpp_1.0.13-1      xml2_1.3.6         renv_1.0.3         vroom_1.6.5       
[61] svglite_2.1.3      rstudioapi_0.17.1  jsonlite_1.8.9     R6_2.5.1          
[65] fs_1.6.5           systemfonts_1.1.0

9. GitHub Repository

The complete code for this analysis is available in tt_2025_24.qmd.

For the full repository, click here.

10. References

  1. Data Sources:
  • TidyTuesday 2025 Week 24: [API Specs](https://github.com/rfordatascience/tidytuesday/blob/main/data/2025/2025-06-17
Back to top