Point aggregation

Use point aggregation methods within your line plots for improved data visualization accuracy and performance. There are two types of point aggregation modes: full fidelity and random sampling. W&B uses full fidelity mode by default.

Full fidelity

When you use full fidelity mode, W&B breaks the x-axis into dynamic buckets based on the number of data points. It then calculates the minimum, maximum, and average values within each bucket while rendering a point aggregation for the line plot.

There are three main advantages to using full fidelity mode for point aggregation:

• Preserve extreme values and spikes: retain extreme values and spikes in your data
• Configure how minimum and maximum points render: use the W&B App to interactively decide whether you want to show extreme (min/max) values as a shaded area.
• Explore your data without losing data fidelity: W&B recalculates x-axis bucket sizes when you zoom into specific data points. This helps ensure that you can explore your data without losing accuracy. Caching is used to store previously computed aggregations to help reduce loading times which is particularly useful if you are navigating through large datasets.

Turn on full fidelity

W&B uses full fidelity mode by default. To configure it manually, follow these steps:

All charts in a workspace

1. Navigate to your workspace.
2. Select the gear icon on the top right corner of the screen next to the left of the Add panels button.
3. From the UI slider that appears, select Line plots
4. Choose Full fidelity from the Point aggregation section.
5. Configure the Smoothing algorithm and settings.
6. Set Aggregation to Mean, Min, or Max.
7. Click Apply.

Individual chart in a workspace

1. Navigate to your workspace.
2. Select on the Workspace icon on the left tab
3. Hover over the line plot panel you want to configure, then click the gear icon.
4. Within the modal that appears, set Point aggregation method to Full fidelity.
5. Configure the Smoothing algorithm and settings.
6. Click Apply.

Configure shading

The shaded areas of a full-fidelity line plot can show:

• Min/Max: For each X-axis point, shade the area between the minimum and maximum values. The shaded area shows all points from the lowest to the highest value in each bucket:

  $$\text{Min/Max Range} = [\min(x_1, x_2, \ldots, x_n),\ \max(x_1, x_2, \ldots, x_n)]$$

  where $x_1, x_2, \ldots, x_n$ are the values in a given bucket.

• Standard deviation: For each X-axis point, calculate the variability of the values using standard deviation, and shade the resulting area.

  $$SD = \sqrt{\frac{1}{n}\sum_{i=1}^{n}(x_i - \overline{x})^2}$$

• Standard error: For each X-axis point, calculate the likelihood of a sampling error by dividing the value by the square root of the sample size:

  $$SE = \frac{SD}{\sqrt{n}}$$

• None: No shading (the default).

The proceeding image shows a blue line plot. The light blue shaded area represents the minimum and maximum values for each bucket.

To configure shading:

1. Navigate to your workspace.
2. Hover over a line plot, then click the gear icon.
3. In the Data tab, set Point aggregation to Full fidelity if necessary, then configure the smoothing algorithm.
4. In the Grouping tab, turn on Group runs. Optionally, set Group by to a run attribute.
5. Set Agg to Mean (default), Min, or Max.
6. Set Range to Min/Max, Std Dev, Std Err, or None.
7. Click Apply.

Explore your data without losing data fidelity

Analyze specific regions of the dataset without missing critical points like extreme values or spikes. When you zoom in on a line plot, W&B adjusts the buckets sizes used to calculate the minimum, maximum, and average values within each bucket.

W&B divides the x-axis is dynamically into 1000 buckets by default. For each bucket, W&B calculates the following values:

• Minimum: The lowest value in that bucket.
• Maximum: The highest value in that bucket.
• Average: The mean value of all points in that bucket.

W&B plots values in buckets in a way that preserves full data representation and includes extreme values in every plot. When zoomed in to 1,000 points or fewer, full fidelity mode renders every data point without additional aggregation.

To zoom in on a line plot, follow these steps:

1. Navigate to your W&B project
2. Select on the Workspace icon on the left tab
3. Optionally add a line plot panel to your workspace or navigate to an existing line plot panel.
4. Click and drag to select a specific region to zoom in on.

Line plot grouping and expressions

When you use Line Plot Grouping, W&B applies the following based on the mode selected:

• Non-windowed sampling (grouping): Aligns points across runs on the x-axis. The average is taken if multiple points share the same x-value; otherwise, they appear as discrete points.
• Windowed sampling (grouping and expressions): Divides the x-axis either into 250 buckets or the number of points in the longest line (whichever is smaller). W&B takes an average of points within each bucket.
• Full fidelity (grouping and expressions): Similar to non-windowed sampling, but fetches up to 500 points per run to balance performance and detail.

Random sampling

Random sampling uses 1500 randomly sampled points to render line plots. Random sampling is useful for performance reasons when you have a large number of data points.

Random sampling samples non-deterministically. This means that random sampling sometimes excludes important outliers or spikes in the data and therefore reduces data accuracy.

Enable random sampling

By default, W&B uses full fidelity mode. To enable random sampling, follow these steps:

All charts in a workspace

1. Navigate to your W&B project
2. Select on the Workspace icon on the left tab
3. Select the gear icon on the top right corner of the screen next to the left of the Add panels button.
4. From the UI slider that appears, select Line plots
5. Choose Random sampling from the Point aggregation section

Individual chart in a workspace

1. Navigate to your W&B project
2. Select on the Workspace icon on the left tab
3. Select the line plot panel you want to enable random sampling for
4. Within the modal that appears, select Random sampling from the Point aggregation method section

Access non sampled data

You can access the complete history of metrics logged during a run using the W&B Run API. The following example demonstrates how to retrieve and process the loss values from a specific run:

# Initialize the W&B API
run = api.run("l2k2/examples-numpy-boston/i0wt6xua")

# Retrieve the history of the 'Loss' metric
history = run.scan_history(keys=["Loss"])

# Extract the loss values from the history
losses = [row["Loss"] for row in history]