scholtzan · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026 · Oct 24, 2026
diff --git a/src/concepts/pipeline/event_pipeline.md b/src/concepts/pipeline/event_pipeline.md
@@ -125,6 +125,7 @@ On the pipeline side, the event data is made available in different datasets:
   its event payload for Firefox versions before 62.
 - [`events`](../../datasets/batch_view/events/reference.md) contains a row for each event received from main pings and event pings. See [`STMO#52582`](https://sql.telemetry.mozilla.org/queries/52582/source).
 - For applications that collect events through Glean, each application has a separate `events` dataset.
+- [`events_daily`](../../datasets/bigquery/events_daily/reference.md) encodes each client-day's events as a compressed string of Unicode characters, for use with the `mozfun.event_analysis` functions for funnel and event-count analysis.
 
 # Data tooling
 

diff --git a/src/concepts/segments.md b/src/concepts/segments.md
@@ -25,6 +25,8 @@ so that your communication is forwards compatible.
 
 ## Current user states/segments
 
+Note: a newer `clients_last_seen_v2` view now exists alongside `clients_last_seen_v1`, adding columns not yet described below: `new_profile_7_day_activated_v1`, `new_profile_14_day_activated_v1`, and `new_profile_21_day_activated_v1` (new-profile activation flags), plus a `segment` column that generalizes `activity_segments_v1` using a `days_active_bits` column. See the view source for exact definitions.
+
 ### Regular users v3
 
 `clients_last_seen.is_regular_user_v3`

diff --git a/src/concepts/terminology.md b/src/concepts/terminology.md
@@ -158,7 +158,7 @@ In Firefox, [Sessions](#session) are split into subsessions after every 24-hour
 
 ## STMO (sql.telemetry.mozilla.org)
 
-A service for creating queries and dashboards. See [STMO under analysis tools](../introduction/tools.md#sqltelemetrymozillaorg-stmo).
+A service for creating queries and dashboards. While not deprecated, Looker is now the preferred solution for producing data visualizations and dashboards at Mozilla, where possible. See [STMO under analysis tools](../introduction/tools.md#sqltelemetrymozillaorg-stmo).
 
 ## Telemetry
 

diff --git a/src/contributing/structure.md b/src/contributing/structure.md
@@ -9,6 +9,11 @@ The directory structure should feel comfortable to anyone who is familiar with t
    |--tools - contains tool level documentation
    |--concepts - contains tutorials meant to introduce a new concept to the reader
    |--cookbooks - focused code examples for reference
+   |--introduction - contains introductory material for newcomers to the data platform
+   |--reference - contains in-depth reference material on the data platform
+   |--contributing - contains documentation about contributing to this documentation
+   |--historical - contains archived, historical documentation
+   |--metrics - contains archived, historical metrics documentation
 ```
 
 This documentation is meant to take the reader from beginner to expert.

diff --git a/src/cookbooks/clients_last_seen_bits.md b/src/cookbooks/clients_last_seen_bits.md
@@ -79,6 +79,9 @@ ORDER BY
   submission_date
 ```
 
+Other bit pattern fields available in the desktop table include `days_active_bits`
+and `days_visited_1_uri_bits`.
+
 Adding a new usage criterion is possible, but requires some work especially
 if a historical backfill is necessary, so
 [file a bug](../concepts/reporting_a_problem.md) to begin discussions on

diff --git a/src/cookbooks/data_monitoring/intro.md b/src/cookbooks/data_monitoring/intro.md
@@ -16,6 +16,8 @@ Please refer to additional helpful videos available on the Bigeye channel.
 
 In addition to configuring monitors through the Bigeye UI, monitoring can be defined as code in [mozilla/bigquery-etl](https://github.com/mozilla/bigquery-etl). Tables can include a `bigconfig.yml` alongside their `metadata.yaml`, with monitoring enabled via a `monitoring.enabled` field. The `bqetl monitoring` CLI (`deploy`, `update`, `validate`, `deploy_custom_rules`) pushes these configs to Bigeye.
 
+**Caution:** Avoid running `bigconfig apply` or `bqetl monitoring deploy` ad hoc on your local machine outside of the normal deployment automation. Doing so with an incomplete set of config files can delete metrics that aren't included in the plan.
+
 ## Stay Updated on What's Happening
 
 [Jira Epic Link](https://mozilla-hub.atlassian.net/browse/DENG-4563)

diff --git a/src/cookbooks/live_data.md b/src/cookbooks/live_data.md
@@ -117,7 +117,7 @@ Google Cloud Functions can process incoming messages in Pub/Sub and stream the d
 
 ## Tools for Visualizing Live Data
 
-Currently, live data in BigQuery could be considered "near real time" or nearline (as opposed to online), with data typically loaded within 10 minutes to each live table in our ingestion-sink code base, though a delay of up to 30 minutes is normal.
+Currently, live data in BigQuery could be considered "near real time" or nearline (as opposed to online), with latency usually below 1 hour to each live table in our ingestion-sink code base.
 
 For near real time product monitoring use cases, it is recommended to use Looker for hosting dashboards that refresh on a regular interval such as every 10 minutes.
 

diff --git a/src/cookbooks/looker/event_counts_explore.md b/src/cookbooks/looker/event_counts_explore.md
@@ -9,4 +9,4 @@ You can see a quick demo of how to use this explore to answer a simple Firefox f
 </center>
 
 This explore also exists for Mozilla VPN and other products (and you can use the same Glean Dictionary-based workflow to fill it out).
-For Firefox Desktop, this explore is currently using legacy (non-Glean data) and you will need to use the [probe dictionary](../analysis/probe_dictionary.md) instead.
+For Firefox Desktop, the `event_counts` explore is currently using legacy (non-Glean) data and you will need to use the [probe dictionary](../analysis/probe_dictionary.md) instead. A separate `glean_event_counts` explore is also available for Firefox Desktop, which follows the same Glean Dictionary-based workflow as other Glean products.
diff --git a/src/cookbooks/looker/performance_caching.md b/src/cookbooks/looker/performance_caching.md
@@ -27,6 +27,7 @@ infrequently updated, shared views such as `countries.view.lkml` can still use t
 
 After finding the source table, in your `explore.lkml` file, include the auto-generated datagroup and add a
 `persist_with` parameter that matches the datagroup. By convention, the datagroup name matches source tables.
+Generated datagroups also set a default `max_cache_age` of 24 hours as a fallback alongside their SQL trigger.
 See the following example:
 
 ```

diff --git a/src/datasets/batch_view/addons/reference.md b/src/datasets/batch_view/addons/reference.md
@@ -15,10 +15,16 @@ there will be a row with nulls for all the addon fields
 
 This dataset is updated daily via the [telemetry-airflow](https://github.com/mozilla/telemetry-airflow) infrastructure.
 The job DAG runs every day after the Main Summary data has been generated.
-The DAG defined in [`dags/bqetl_addons.py`](https://github.com/mozilla/bigquery-etl/blob/master/dags/bqetl_addons.py)
+The DAG, named `bqetl_addons`, is declaratively defined in [`dags.yaml`](https://github.com/mozilla/bigquery-etl/blob/main/dags.yaml)
 
 ## Code Reference
 
 This dataset is generated by [BigQuery ETL](https://github.com/mozilla/bigquery-etl/). The query that generates the dataset is [`sql/moz-fx-data-shared-prod/telemetry_derived/addons_v2/query.sql`](https://github.com/mozilla/bigquery-etl/blob/master/sql/moz-fx-data-shared-prod/telemetry_derived/addons_v2/query.sql).
 
 You may find the [environment reference](https://firefox-source-docs.mozilla.org/toolkit/components/telemetry/telemetry/data/environment.html#addons) in The Firefox Source Documentation helpful for understanding the source data.
+
+## Storage
+
+The underlying table is partitioned by day on `submission_date`, requires a
+partition filter, and expires partitions after 775 days. It is clustered on
+`sample_id`.
diff --git a/src/datasets/batch_view/clients_daily/reference.md b/src/datasets/batch_view/clients_daily/reference.md
@@ -61,6 +61,8 @@ The data is partitioned by `submission_date`.
 
 As of 2019-11-28, the current version of the `clients_daily` dataset is `v6`.
 
+As of Q1 2021, the `telemetry.clients_daily` view points to `clients_daily_joined`, which merges in event-ping, crash, and first/second-seen data on top of the `v6` main-ping table.
+
 ## Code Reference
 
 This dataset is generated by [BigQuery ETL](https://github.com/mozilla/bigquery-etl/). The query that generates the dataset is [`sql/moz-fx-data-shared-prod/telemetry_derived/clients_daily_v6/query.sql`](https://github.com/mozilla/bigquery-etl/blob/ad84a15d580333b41d36cfe8331e51238f3bafa1/sql/moz-fx-data-shared-prod/telemetry_derived/clients_daily_v6/query.sql).
diff --git a/src/datasets/search/client_ltv/reference.md b/src/datasets/search/client_ltv/reference.md
@@ -123,3 +123,5 @@ ORDER BY
 ```
 
 This produces a histogram for the observed user frequencies and the model's predicted frequencies, allowing a chart similar to the one shown in the ["assessing model fit" example](https://lifetimes.readthedocs.io/en/latest/Quickstart.html#assessing-model-fit) in the `lifetimes` documentation. This table only checks performance for clients the model expects have, for example, clicked an ad in 0 to 28 days in the past year, since most of the distribution is contained in that interval.
+
+_Note: this `lifetimes`-based survival model and the `analysis.ltv_daily_model_perf` table are part of the retired v1 design. The current implementation instead derives client states via a Markov-chain model (`mozfun.ltv.\*\_states_\*`UDFs), fed by per-product`ltv*states_v1` tables.*
diff --git a/src/datasets/search/search_clients_last_seen/reference.md b/src/datasets/search/search_clients_last_seen/reference.md
@@ -87,6 +87,17 @@ root
     |- days_created_profile_bytes: bytes
 ```
 
+## Mobile Search Clients Last Seen
+
+A separate dataset, `search.mobile_search_clients_last_seen` (backed by
+`search_derived.mobile_search_clients_last_seen_v1`), captures search activity
+for mobile clients over the past 365 days and is used as the basis for mobile
+LTV calculations. It is scheduled under the `bqetl_mobile_search` DAG and
+handles app-migration logic (e.g. legacy Fennec/Focus/Klar transitioning to
+their Glean-based successors). See the
+[`mobile_search_clients_last_seen_v1` metadata](https://github.com/mozilla/bigquery-etl/blob/main/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_last_seen_v1/metadata.yaml)
+for details.
+
 # Code Reference
 
 The `search_clients_last_seen` job is

diff --git a/src/metrics/index.md b/src/metrics/index.md
@@ -1,14 +1,14 @@
 # Standard Metrics
 
-> **⚠** Some of the information in this section is obsolete. For an up to date explanation of accepted metrics used at Mozilla, [contact the Data team](../concepts/getting_help.md).
+> **⚠** Some of the information in this section is obsolete. For an up to date explanation of accepted metrics used at Mozilla, see the [Metrics](https://docs.telemetry.mozilla.org/metrics/metrics) page and the [Growth and Usage Dashboard (GUD)](https://mozilla.github.io/gud/), or [contact the Data team](../concepts/getting_help.md).
 
 This section provides an overview of standard metrics used at Mozilla.
 Here you'll find the definitions and descriptions for each.
 
 The [Telemetry Behavior Reference](../concepts/index.md) section also provides
 information related to the definitions below.
 
-We are now in the process of setting up the metrics top-level section here. This information will be moved into the appropriate subsection and this page will be replaced with an overview.
+Canonical metric definitions are now maintained as code in [Metric Hub](https://github.com/mozilla/metric-hub) (`definitions/*.toml`, `nimbus/definitions/*.toml`) and in BigQuery tables such as `telemetry.active_users_aggregates` and `telemetry.mobile_active_users_aggregates`, rather than in this static page.
 
 ## Activity
 
@@ -43,6 +43,10 @@ active during the week beginning one week after the day specified.
 
 Among profiles that were active at least once in the week starting on the
 specified day, what proportion (out of 1) are active during the following week.
+Firefox Desktop now defines retention primarily via the "Retained (DAU)" metric
+(based on DAU-qualifying days); this legacy "submitted any ping" definition has
+been deprecated as of June 2026. See [Metric Hub](https://mozilla.github.io/metric-hub/default_configs/firefox_desktop/)
+for details.
 
 ## Frequently Asked Questions
 

diff --git a/src/metrics/metrics.md b/src/metrics/metrics.md
@@ -52,7 +52,7 @@ The day 2-7 activation metric is calculated as:
 
 \\[ \frac{\text{Activated New Profiles (day 2-7)}}{\text{New Profiles (day 1-7)}} \\]
 
-Activated New Profiles (day 2-7): Unique count of client ids who use the product at any point starting the day after they created a profile up to 6 days after.
+Activated New Profiles (day 2-7): Unique count of client ids who are seen on more than one day during the day 2-7 window after profile creation and have performed at least one search in that period.
 
 New Profiles: Unique count of client ids with a given profile creation date. As not all initial pings are received exactly on the day of profile creation, we wait for 7 days after the profile creation date before establishing the New Profile cohort to ensure the data is complete.
 

diff --git a/src/tools/spark.md b/src/tools/spark.md
@@ -45,6 +45,11 @@ gcloud beta dataproc clusters create cluster-name \
     --project project-id
 ```
 
+> **Note:** Dataproc image versions 2.1 and above (including the `2.3` image used above) ship with a
+> built-in Spark BigQuery connector. Setting `spark.jars` via `--properties` will not override it, since
+> the built-in connector takes precedence; to use a different connector version, pass
+> `--metadata SPARK_BQ_CONNECTOR_VERSION=<version>` instead.
+
 You can retrieve the Jupyter URL with the following command:
 
 ```bash
Original file line number	Diff line number	Diff line change
Expand Up		@@ -123,3 +123,5 @@ ORDER BY
		```

		This produces a histogram for the observed user frequencies and the model's predicted frequencies, allowing a chart similar to the one shown in the ["assessing model fit" example](https://lifetimes.readthedocs.io/en/latest/Quickstart.html#assessing-model-fit) in the `lifetimes` documentation. This table only checks performance for clients the model expects have, for example, clicked an ad in 0 to 28 days in the past year, since most of the distribution is contained in that interval.

		_Note: this `lifetimes`-based survival model and the `analysis.ltv_daily_model_perf` table are part of the retired v1 design. The current implementation instead derives client states via a Markov-chain model (`mozfun.ltv.\\_states_\`UDFs), fed by per-product`ltvstates_v1` tables.