Code
shhh <- function(expr) suppressPackageStartupMessages(suppressWarnings(suppressMessages(expr)))
shhh({
library(tidyverse);
library(lubridate);
library(scales);
library(magrittr);
library(dplyr);
})Task: T411443
Introduction
After establishing the first retention baseline under FY25–26 KR WE3.6.1, we launched the second round in November to establish a second baseline and evaluate changes over time.(FY25–26 KR WE3.6.7) Similar to the first round, the second round used a 28-day A/A test, in which logged-out user traffic was bucketed based on Edge Unique ID, and their page-load events were tracked throughout the experiment period.
Methodology
We analyzed data from 2025-11-22 to 2025-12-20, calculated retention rates using the definitions adopted in the first-round analysis, computed the average and standard deviation across cohorts, and compared the results with the first baseline to evaluate changes.
Retention rates were calculated for the top 50 Wikipedias both overall and at the per-wiki level. The analysis was performed separately for desktop and mobile web. The final overall estimate was derived by combining the platform-specific results, weighted by the relative share of each platform.
Takeaways
- The overall retention rate changed only slightly (within −0.4% to +1.2%) between August and November across four definitions and two platforms, indicating that the baseline provides a reliable picture of user retention and can serve for estimation purposes.
- Trends vary by wiki. Some show increases while others show decreases, suggesting that trend may be influenced by wiki-specific factors.
- Short-term and long-term retention may exhibit divergent trends. Across the two baselines, short-term retention shows a slight decline, while longer-term retention trends upward.
Setup
Code
library(broom) # for tidy()Code
options(repr.plot.width = 15, repr.plot.height = 8)Code
#Count how many wikis increased and how many decreased.
count_wiki_changes <- function(df) {
df %>%
mutate(change = case_when(
diff > 0 ~ "increase",
diff < 0 ~ "decrease",
TRUE ~ "zero"
)) %>%
count(change)
}Code
# For summary tables
library(gt)
library(gtsummary)
library(IRdisplay)
# For html report
library(htmltools)Metric definitions
The metric definitions follow the terminology and framework documented on the Data Glossary page and in the first retention baseline report.
2nd day retention rate
The percentage of users who return on the second day after their first visit within a given cohort week. This metric requires a minimum of 9 days of data.
Parameters: cohort window = 7 days, retention length=1 day, return window = 1 dayTwo continued returns (2nd Day & 3rd Day) retention rate
The percentage of users who return on both the 2nd day and 3rd day after their first visit within a given cohort week. This metric requires a minimum of 10 days of data. Parameters: cohort window=7 days, Retention length=1 day, Return window= 1 day, Retention length2=2 days, Return window2 = 1 day2nd week retention rate
The percentage of users who return in the second week after their first visit within a given cohort week. This metric requires a minimum of 21 days of data.
Parameters: cohort window = 7 days, retention length = 7 days, return window = 7 days21-day cumulative retention rate
The percentage of users who return at least once by the end of day 21 after their first visit within a given cohort week.
Parameters: cohort window = 7 days, return window: by a cutoff date - day 21, retention length = 1
Analysis
Desktop
Overall retention rate
Compare the second baseline (November) with the first baseline (August) using data collected from the top 50 Wikipedias:
- Short-term retention slightly decreased
- 2nd-day retention: -0.39 pp
- 2-day consecutive retention (Day 2–3): -0.32 pp
- Longer-term retention increased
- 2nd-week retention: +0.98 pp
- 21-day cumulative retention: +1.19 pp
Code
df_c1rxr1_d_Aug <- read.csv('../2025_retention_rate/Data_out/desktop_c7rxr1.csv')
df_c1r1r1_d_Aug <- df_c1rxr1_d_Aug %>%
filter(return_interval_l==1) %>%
mutate(
baseline='Baseline1_Aug',
metric='2nd-day retention'
)
df <- df_c1r1r1_d_Aug %>%
select(metric, retention_rate, baseline)
df_c7r7r7_d_Aug <- read.csv('../2025_retention_rate/Data_final/desktop_c7r7r7.csv')
df_c7r7r7_d_Aug <- df_c7r7r7_d_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='2nd-week retention'
)
df <- df_c7r7r7_d_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_2returns_2nd3rd_day_d_Aug <- read.csv('../2025_retention_rate/Data_final/desktop_2returns_c7r1r1_c7r2r1.csv')
df_21day_cumulative_d_Aug <- read.csv('../2025_retention_rate/Data_final/desktop_cumulative_c7r1r21.csv')
df_2returns_2nd3rd_day_d_Aug <- df_2returns_2nd3rd_day_d_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='2-day consecutive retention (Day 2-3)'
)
df <- df_2returns_2nd3rd_day_d_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_21day_cumulative_d_Aug <- df_21day_cumulative_d_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='21-day cumulative retention'
)
df <- df_21day_cumulative_d_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_c1r1r1_d_Nov <- read.csv('Data_final/desktop_c7r1r1.csv')
df_c7r7r7_d_Nov <- read.csv('Data_final/desktop_c7r7r7.csv')
df_2returns_2nd3rd_day_d_Nov <- read.csv('Data_final/desktop_2returns_c7r1r1_c7r2r1.csv')
df_21day_cumulative_d_Nov <- read.csv('Data_final/desktop_cumulative_c7r1r21.csv')
df_c1r1r1_d_Nov <- df_c1r1r1_d_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2nd-day retention'
)
df <- df_c1r1r1_d_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_c7r7r7_d_Nov <- df_c7r7r7_d_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2nd-week retention'
)
df <- df_c7r7r7_d_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_2returns_2nd3rd_day_d_Nov <- df_2returns_2nd3rd_day_d_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2-day consecutive retention (Day 2-3)'
)
df_21day_cumulative_d_Nov <- df_21day_cumulative_d_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='21-day cumulative retention'
)
df <- df_2returns_2nd3rd_day_d_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df <- df_21day_cumulative_d_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df$metric <- factor(df$metric,
levels=c("2nd-day retention","2-day consecutive retention (Day 2-3)","2nd-week retention","21-day cumulative retention"))
Code
df_diff_d <- df %>%
group_by(metric, baseline) %>%
summarize(
mean_retention = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
.groups = "drop"
) %>%
pivot_wider(
names_from = baseline,
values_from = c(mean_retention, sd)
) %>%
mutate(diff = mean_retention_Baseline2_Nov - mean_retention_Baseline1_Aug)Code
display_html(
as_raw_html(
df_diff_d %>%
gt()%>%
tab_header(
title = md("Retention Rate – Baseline 2 vs. Baseline 1 (Desktop)")
) %>%
tab_spanner(
label = "Mean",
columns = c(mean_retention_Baseline1_Aug, mean_retention_Baseline2_Nov)
) %>%
tab_spanner(
label = "Standard Deviation",
columns = c(sd_Baseline1_Aug, sd_Baseline2_Nov)
) %>%
tab_spanner(
label = "Mean Delta",
columns = c(diff)
) %>%
cols_label(
mean_retention_Baseline1_Aug = "Baseline1",
mean_retention_Baseline2_Nov = "Baseline2",
sd_Baseline1_Aug = "Baseline1",
sd_Baseline2_Nov = "Baseline2",
diff = "Baseline2-Baseline1",
metric = "Metrics"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_spanners(c("Mean", "Standard Deviation","Mean Delta"))
) %>%
fmt_percent(
columns = everything(),
decimals = 2
) %>%
# opt_stylize(6) %>%
cols_width(everything() ~ px(150)) %>%
tab_source_note(
source_note = md(
"Timeframe: <br>
Baseline2: 2025-11-22 ~ 2025-12-20 <br>
Baseline1: 2025-08-03 ~ 2025-08-30 <br>
Logged-out users on desktop")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)
| Retention Rate – Baseline 2 vs. Baseline 1 (Desktop) | |||||
| Baseline1 | Baseline2 | Baseline1 | Baseline2 | Baseline2-Baseline1 | |
|---|---|---|---|---|---|
| Timeframe: Baseline2: 2025-11-22 ~ 2025-12-20 Baseline1: 2025-08-03 ~ 2025-08-30 Logged-out users on desktop |
|||||
Code
df_desktop <- dfCode
g_d <- df_desktop %>%
ggplot(aes(x = baseline, y = retention_rate)) +
geom_boxplot(alpha = 0.7) +
facet_wrap(~metric, scales = "free_y") +
stat_summary(fun = mean, geom = "point", shape = 20, size = 5, color = "blue") +
# Add difference labels above Baseline2
geom_label(
data = df_diff,
aes(x = "Baseline2_Nov", y = Baseline2_Nov, label = paste0("Diff:", scales::percent(diff, accuracy = 0.1))),
color = "black",
fill = "white", # white background
vjust = 0.5,
hjust = 1.5,
inherit.aes = FALSE
) +
theme(legend.position = "none") +
scale_fill_brewer(palette = "Set1") +
scale_y_continuous(labels = percent_format(accuracy = 0.1)) +
labs(
title = 'Retention Rate Change: Baseline 2 vs Baseline 1',
x = 'Baseline version',
y = 'Retention rate',
caption = paste(
"Baseline1: data collected from 2025-08-03 to 2025-08-30\n",
"Baseline2: data collected from 2025-11-22 to 2025-12-20\n",
"Logged-out users on desktop"
)
) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic"),
strip.text = element_text(
color = "white", # facet title text color
face = "bold"
),
strip.background = element_rect(
fill = "#000099", # facet title background
color = "black", # border color
linewidth = 0.5 # border thickness (instead of size)
)
)
g_d
Per wikis
Overall, short-term retention tends to decline, whereas longer-term retention demonstrates broader improvement on desktop. This pattern is evident across most wikis.
Retention rates for each wiki are available in the sheet.
Second-day Retention Rate
Among the 50 wikis analyzed, 21 (42%) show an increase and 29 (58%) show a decrease.
Code
df_wiki_d_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/desktop_c7r1r1_wiki_stats.csv')
df_wiki_d_Nov <- read.csv('Data_out_wiki/desktop_c7r1r1_wiki.csv')
df_wiki_d_Nov_stats <- df_wiki_d_Nov %>%
# Exclude wikis outside the top 50
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_d_Nov_stats,
df_wiki_d_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "Second-day Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on desktop web"
)
)
g
Code
df_joined_d_2ndDay <- df_joined
count_wiki_changes(df_joined_d_2ndDay)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_d_2ndDay) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: Second-day Retention Rate<br>
User group: logged-out users on desktop ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: Second-day Retention Rate User group: logged-out users on desktop |
2-day consecutive retention (Day 2-3)
Among the 50 wikis analyzed, 15 (30%) show an increase and 35 (70%) show a decrease.
Code
# Wiki size (in thousands)
df_wiki_d_size <- df_wiki_d_Nov_stats %>%
select(wiki_db, wiki_size) Code
df_wiki_d_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/desktop_c7r1r1_c7r2r1_wiki_stats.csv')
df_wiki_d_Nov_stats <- read.csv('Data_out_wiki/desktop_c7r1r1_c7r2r1_wiki_stats.csv') %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki"))
df_joined <- df_wiki_d_size %>%
inner_join(df_wiki_d_Nov_stats, by = "wiki_db") %>%
inner_join(df_wiki_d_Aug_stats, by = "wiki_db", suffix = c("_Nov", "_Aug"))Code
df_joined <- df_joined %>%
mutate(
diff= mean_Nov-mean_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "2-day consecutive retention (Day 2-3) - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on desktop web"
)
)
g
Code
df_joined_d_2nd3rdDay <- df_joined
count_wiki_changes(df_joined_d_2nd3rdDay)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_d_2nd3rdDay) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: 2-day consecutive retention (Day 2-3)<br>
User group: logged-out users on desktop ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: 2-day consecutive retention (Day 2-3) User group: logged-out users on desktop |
Second-week Retention Rate
Among the 50 wikis analyzed, 33 (66%) show an increase and 17 (34%) show a decrease.
Code
df_wiki_d_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/desktop_c7r7r7_wiki_stats.csv')
df_wiki_d_Nov <- read.csv('Data_out_wiki/desktop_c7r7r7_wiki.csv')
df_wiki_d_Nov_stats <- df_wiki_d_Nov %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_d_Nov_stats,
df_wiki_d_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "Second-week Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on desktop web"
)
)
g
Code
df_joined_d_2ndWeek <- df_joined
count_wiki_changes(df_joined_d_2ndWeek)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_d_2ndWeek) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: Second-week Retention Rate <br>
User group: logged-out users on desktop ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: Second-week Retention Rate User group: logged-out users on desktop |
21-day Cumulative Retention Rate
Among the 50 wikis analyzed, 34 (68%) show an increase and 16 (32%) show a decrease
Code
df_wiki_d_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/desktop_c7_cumul21_wiki_stats.csv')
df_wiki_d_Nov <- read.csv('Data_out_wiki/desktop_c7_cumul21_wiki.csv')
df_wiki_d_Nov_stats <- df_wiki_d_Nov %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_d_Nov_stats,
df_wiki_d_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "21-day Cumulative Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on desktop web"
)
)
g
Code
df_joined_d_21Day <- df_joined
count_wiki_changes(df_joined_d_21Day)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_d_21Day) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: 21-day Cumulative Retention Rate <br>
User group: logged-out users on desktop ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: 21-day Cumulative Retention Rate User group: logged-out users on desktop |
Mobile Web
Overall retention rate
Compare the second baseline (November) with the first baseline (August) using data collected from the top 50 Wikipedias:
- Short-term retention slightly decreased
- 2nd-day retention: -0.32 pp
- 2-day consecutive retention (Day 2–3): -0.21 pp
- Longer-term retention increased modestly
- 2nd-week retention: +0.84 pp
- 21-day cumulative retention: +0.19 pp
- The trends are consistent across both desktop and mobile web platforms.
Code
df_c1rxr1_m_Aug <- read.csv('../2025_retention_rate/Data_out/mobile_c7rxr1_d2.csv')
df_c1r1r1_m_Aug <- df_c1rxr1_m_Aug %>%
filter(return_interval_l==1) %>%
mutate(
baseline='Baseline1_Aug',
metric='2nd-day retention'
)
df <- df_c1r1r1_m_Aug %>%
select(metric, retention_rate, baseline)
df_c7r7r7_m_Aug <- read.csv('../2025_retention_rate/Data_final/mobile_c7r7r7.csv')
df_2returns_2nd3rd_day_m_Aug <- read.csv('../2025_retention_rate/Data_final/mobile_2returns_c7r1r1_c7r2r1.csv')
df_21day_cumulative_m_Aug <- read.csv('../2025_retention_rate/Data_final/mobile_cumulative_c7r1r21.csv')
df_c7r7r7_m_Aug <- df_c7r7r7_m_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='2nd-week retention'
)
df <- df_c7r7r7_m_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_2returns_2nd3rd_day_m_Aug <- df_2returns_2nd3rd_day_m_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='2-day consecutive retention (Day 2-3)'
)
df <- df_2returns_2nd3rd_day_m_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_21day_cumulative_m_Aug <- df_21day_cumulative_m_Aug %>%
mutate(
baseline='Baseline1_Aug',
metric='21-day cumulative retention'
)
df <- df_21day_cumulative_m_Aug %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_c1r1r1_m_Nov <- read.csv('Data_final/mobile_c7r1r1.csv')
df_c7r7r7_m_Nov <- read.csv('Data_final/mobile_c7r7r7.csv')
df_2returns_2nd3rd_day_m_Nov <- read.csv('Data_final/mobile_2returns_c7r1r1_c7r2r1.csv')
df_21day_cumulative_m_Nov <- read.csv('Data_final/mobile_cumulative_c7r1r21.csv')
df_c1r1r1_m_Nov <- df_c1r1r1_m_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2nd-day retention'
)
df <- df_c1r1r1_m_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_c7r7r7_m_Nov <- df_c7r7r7_m_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2nd-week retention'
)
df <- df_c7r7r7_m_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df_2returns_2nd3rd_day_m_Nov <- df_2returns_2nd3rd_day_m_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='2-day consecutive retention (Day 2-3)'
)
df_21day_cumulative_m_Nov <- df_21day_cumulative_m_Nov %>%
mutate(
baseline='Baseline2_Nov',
metric='21-day cumulative retention'
)
df <- df_2returns_2nd3rd_day_m_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df <- df_21day_cumulative_m_Nov %>%
select(metric, retention_rate, baseline) %>%
bind_rows(df)
df$metric <- factor(df$metric,
levels=c("2nd-day retention","2-day consecutive retention (Day 2-3)","2nd-week retention","21-day cumulative retention"))
Code
df_diff_m <- df %>%
group_by(metric, baseline) %>%
summarize(
mean_retention = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
.groups = "drop"
) %>%
pivot_wider(
names_from = baseline,
values_from = c(mean_retention, sd)
) %>%
mutate(diff = mean_retention_Baseline2_Nov - mean_retention_Baseline1_Aug)Code
display_html(
as_raw_html(
df_diff_m %>%
gt()%>%
tab_header(
title = md("Retention Rate – Baseline 2 vs. Baseline 1 (Mobile Web)")
) %>%
tab_spanner(
label = "Mean",
columns = c(mean_retention_Baseline1_Aug, mean_retention_Baseline2_Nov)
) %>%
tab_spanner(
label = "Standard Deviation",
columns = c(sd_Baseline1_Aug, sd_Baseline2_Nov)
) %>%
tab_spanner(
label = "Mean Delta",
columns = c(diff)
) %>%
cols_label(
mean_retention_Baseline1_Aug = "Baseline1",
mean_retention_Baseline2_Nov = "Baseline2",
sd_Baseline1_Aug = "Baseline1",
sd_Baseline2_Nov = "Baseline2",
diff = "Baseline2-Baseline1",
metric = "Metrics"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_spanners(c("Mean", "Standard Deviation","Mean Delta"))
) %>%
fmt_percent(
columns = everything(),
decimals = 2
) %>%
# opt_stylize(6) %>%
cols_width(everything() ~ px(150)) %>%
tab_source_note(
source_note = md(
"Timeframe: <br>
Baseline2: 2025-11-22 ~ 2025-12-20 <br>
Baseline1: 2025-08-03 ~ 2025-08-30 <br>
Logged-out users on mobile web")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)
| Retention Rate – Baseline 2 vs. Baseline 1 (Mobile Web) | |||||
| Baseline1 | Baseline2 | Baseline1 | Baseline2 | Baseline2-Baseline1 | |
|---|---|---|---|---|---|
| Timeframe: Baseline2: 2025-11-22 ~ 2025-12-20 Baseline1: 2025-08-03 ~ 2025-08-30 Logged-out users on mobile web |
|||||
Code
df_mobile <- dfCode
g_m <- df_mobile %>%
ggplot(aes(x = baseline, y = retention_rate)) +
geom_boxplot(alpha = 0.7) +
facet_wrap(~metric, scales = "free_y") +
stat_summary(fun = mean, geom = "point", shape = 20, size = 5, color = "blue") +
# Add difference labels above Baseline2
geom_label(
data = df_diff2,
aes(x = "Baseline2_Nov", y = Baseline2_Nov, label = paste0("Diff:", scales::percent(diff, accuracy = 0.1))),
color = "black",
fill = "white", # white background
vjust = 0.5,
hjust = 1.5,
inherit.aes = FALSE
) +
theme(legend.position = "none") +
scale_fill_brewer(palette = "Set1") +
scale_y_continuous(labels = percent_format(accuracy = 0.1)) +
labs(
title = 'Retention Rate: Baseline 2 vs. Baseline 1',
x = 'Baseline version',
y = 'Retention rate',
caption = paste(
"Baseline1: data collected from 2025-08-03 to 2025-08-30\n",
"Baseline2: data collected from 2025-11-22 to 2025-12-20\n",
"Logged-out users on mobile web"
)
) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic"),
strip.text = element_text(
color = "white", # facet title text color
face = "bold"
),
strip.background = element_rect(
fill = "#000099", # facet title background
color = "black", # border color
linewidth = 0.5 # border thickness (instead of size)
)
)
g_m
Per wikis
Short-term retention shows mixed results. For both second-day retention and 2-day consecutive retention (Day 2–3), the results are evenly split among the 50 wikis analyzed, with 25 showing increases and 25 showing decreases. Longer-term retention metrics show more consistent improvement across wikis.
Retention rates for each wiki are available in the sheet.
Second-day Retention Rate
Among the 50 wikis analyzed, the results are evenly split: 25 increases and 25 decreases.
Code
df_wiki_m_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/mobile_c7r1r1_wiki_stats.csv')
df_wiki_m_Nov <- read.csv('Data_out_wiki/mobile_c7r1r1_wiki.csv')
df_wiki_m_Nov_stats <- df_wiki_m_Nov %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_m_Nov_stats,
df_wiki_m_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "Second-day Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on mobile web"
)
)
g
Code
df_joined_m_2ndDay <- df_joined
count_wiki_changes(df_joined_m_2ndDay)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_m_2ndDay) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: Second-day Retention Rate<br>
User group: logged-out users on mobile web ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: Second-day Retention Rate User group: logged-out users on mobile web |
2-day consecutive retention (Day 2-3)
Among the 50 wikis analyzed, the results are evenly split: 25 increases and 25 decreases.
Code
# Wiki size (in thousands)
df_wiki_m_size <- df_wiki_m_Nov_stats %>%
select(wiki_db, wiki_size) Code
df_wiki_m_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/mobile_c7r1r1_c7r2r1_wiki_stats.csv')
df_wiki_m_Nov_stats <- read.csv('Data_out_wiki/mobile_c7r1r1_c7r2r1_wiki_stats.csv') %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki"))
df_joined <- df_wiki_m_size %>%
inner_join(df_wiki_m_Nov_stats, by = "wiki_db") %>%
inner_join(df_wiki_m_Aug_stats, by = "wiki_db", suffix = c("_Nov", "_Aug"))Code
df_joined <- df_joined %>%
mutate(
diff= mean_Nov-mean_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "2-day consecutive retention (Day 2-3) - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on mobile web"
)
)
g
Code
df_joined_m_2nd3rdDay <- df_joined
count_wiki_changes(df_joined_m_2nd3rdDay)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_m_2nd3rdDay) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: 2-day consecutive retention (Day 2-3)<br>
User group: logged-out users on mobile web ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: 2-day consecutive retention (Day 2-3) User group: logged-out users on mobile web |
Second-week Retention Rate
Among the 50 wikis analyzed, 35 (70%) show an increase and 15 (30%) show a decrease.
Code
df_wiki_m_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/mobile_c7r7r7_wiki_stats.csv')
df_wiki_m_Nov <- read.csv('Data_out_wiki/mobile_c7r7r7_wiki.csv')
df_wiki_m_Nov_stats <- df_wiki_m_Nov %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_m_Nov_stats,
df_wiki_m_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "Second-week Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on mobile web"
)
)
g
Code
df_joined_m_2ndWeek <- df_joined
count_wiki_changes(df_joined_m_2ndWeek)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_m_2ndWeek) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: Second-week Retention Rate<br>
User group: logged-out users on mobile web ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: Second-week Retention Rate User group: logged-out users on mobile web |
21-day Cumulative Retention Rate
Among the 50 wikis analyzed, 32 (64%) show an increase and 18 (36%) show a decrease.
Code
df_wiki_m_Aug_stats <- read.csv('../2025_retention_rate/Data_out_wiki/mobile_c7_cumul21_wiki_stats.csv')
df_wiki_m_Nov <- read.csv('Data_out_wiki/mobile_c7_cumul21_wiki.csv')
df_wiki_m_Nov_stats <- df_wiki_m_Nov %>%
filter(!wiki_db %in% c("bjnwiki", "arzwiki")) %>%
mutate(
# Adjusted to users (in thousands)
wiki_size=case_when(
wiki_db == "enwiki" ~ cohort_users ,
TRUE ~ cohort_users*0.1
),
.groups='drop'
) %>%
group_by(wiki_db) %>%
summarize(
retention_avg = mean(retention_rate, na.rm = TRUE),
sd = sd(retention_rate, na.rm = TRUE),
wiki_size = round(mean(wiki_size, na.rm = TRUE), 0)
)
df_joined <- inner_join(df_wiki_m_Nov_stats,
df_wiki_m_Aug_stats,
by = "wiki_db",
suffix = c("_Nov", "_Aug")) %>%
rename(retention_avg_Nov = retention_avg,
retention_avg_Aug = mean)
df_joined <- df_joined %>%
mutate(
diff=retention_avg_Nov-retention_avg_Aug
)
max_abs <- max(abs(df_joined$diff), na.rm = TRUE)
g <- df_joined %>%
ggplot(aes(x=diff, y=sd_Nov, size=wiki_size, color = wiki_size)) +
geom_point(alpha=0.5) +
scale_size(range = c(1, 24)) +
scale_color_gradient(low = "darkblue" , high = "lightblue") +
scale_x_continuous( limits = c(-max_abs, max_abs), labels = label_percent(accuracy = 0.1)) +
scale_y_continuous(labels = label_percent(accuracy = 0.1)) +
theme_light(base_size = 18) +
theme(
plot.caption = element_text(hjust = 0, face = "italic")
) +
labs(
title = "21-day Cumulative Retention Rate - Change vs. Standard Deviation",
x = "Retention rate change",
y = "Retention rate standard deviation",
size = "Wiki population (K)",
color = "",
caption = paste(
"Logged-out users on mobile web"
)
)
g
Code
df_joined_m_21Day <- df_joined
count_wiki_changes(df_joined_m_21Day)Code
display_html(
as_raw_html(
count_wiki_changes(df_joined_m_21Day) %>%
gt()%>%
tab_header(
title = md("Wikis Showing Increase vs. Decrease")
) %>%
cols_label(
change="Change",
n="Number of wikis"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(change,n ))
) %>%
fmt_number(
columns = c("change","n"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(200)) %>%
tab_source_note(
source_note = md(
"Metric: 21-day Cumulative Retention Rate<br>
User group: logged-out users on mobile web ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Wikis Showing Increase vs. Decrease | |
| Metric: 21-day Cumulative Retention Rate User group: logged-out users on mobile web |
Retention rate estimate for desktop + mobile web
Code
result <- tibble(
# Wiki size (in thousands)
desktop_in_K = sum(df_wiki_d_size$wiki_size, na.rm = TRUE),
mobile_in_K = sum(df_wiki_m_size$wiki_size, na.rm = TRUE)
) %>%
mutate(
total_in_K = desktop_in_K + mobile_in_K,
desktop_pct = desktop_in_K / total_in_K,
mobile_pct = mobile_in_K / total_in_K
)Code
display_html(
as_raw_html(
result %>%
gt()%>%
tab_header(
title = md("Weekly Edge Uniques by Platform <br>(in Thousands)")
) %>%
cols_label(
desktop_in_K = "Desktop (K)",
mobile_in_K = "Mobile Web (K)",
total_in_K = "Total (K)",
desktop_pct = "Desktop (%)",
mobile_pct = "Mobile Web (%)"
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(desktop_in_K,mobile_in_K,total_in_K,desktop_pct,mobile_pct ))
) %>%
fmt_percent(
columns = c("desktop_pct", "mobile_pct"),
decimals = 2
) %>%
fmt_number(
columns = c("desktop_in_K","mobile_in_K","total_in_K"),
decimals = 0
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(150)) %>%
tab_source_note(
source_note = md(
"User group: logged-out users on desktop and mobile web <br>
Data timeframe: 2025-11-22 ~ 2025-12-20 <br>
Method: rolling average")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Weekly Edge Uniques by Platform (in Thousands) |
||||
| User group: logged-out users on desktop and mobile web Data timeframe: 2025-11-22 ~ 2025-12-20 Method: rolling average |
Code
overall_retention <- df_diff_d %>%
select(metric,
ret2_desktop = mean_retention_Baseline2_Nov,
ret1_desktop = mean_retention_Baseline1_Aug) %>%
left_join(
df_diff_m %>%
select(metric,
ret2_mobile = mean_retention_Baseline2_Nov,
ret1_mobile = mean_retention_Baseline1_Aug),
by = "metric"
) %>%
mutate(
overall_retention2 =
ret2_desktop * result$desktop_pct + ret2_mobile * result$mobile_pct,
overall_retention1 =
ret1_desktop * result$desktop_pct + ret1_mobile * result$mobile_pct,
overall_diff= overall_retention2 - overall_retention1
)Code
display_html(
as_raw_html(
overall_retention %>%
select(metric, overall_retention2,overall_retention1,overall_diff) %>%
gt()%>%
tab_header(
title = md("Estimated Overall Retention Rate <br>(Desktop + Mobile Web)")
) %>%
cols_label(
metric = "Metrics",
overall_retention2 = "Overall Retention Baseline2",
overall_retention1 = "Overall Retention Baseline1",
overall_diff = md("Delta <br>(Baseline2-Baseline1)")
) %>%
tab_style(
style = cell_text(weight = "bold"),
locations = cells_column_labels(columns = c(metric, overall_retention2,overall_retention1,overall_diff ))
) %>%
fmt_percent(
columns = c("overall_retention2", "overall_retention1", "overall_diff"),
decimals = 2
) %>%
opt_stylize(6) %>%
cols_width(everything() ~ px(150)) %>%
tab_source_note(
source_note = md(
"User group: logged-out users on desktop and mobile web ")
)%>%
tab_style(
style = cell_text(align = "left"),
locations = cells_source_notes()
)
)
)| Estimated Overall Retention Rate (Desktop + Mobile Web) |
|||
| User group: logged-out users on desktop and mobile web |
Retention is measured by Edge Unique IDs, so activity on multiple platforms or devices is counted separately and does not reflect a user-based cross-platform retention rate.