V

Vuer-Viz

vuer-vizv1.0.2

Publication-Quality Visualizations

Example Data & Glob Patterns

Documentation example datasets and glob pattern matching for querying experiments.

Table of Contents

Example Data Fixtures

The documentation includes three example datasets demonstrating different visualization patterns. These datasets are used throughout the examples to show how data hooks work.

Learning Curves

Realistic training curves comparing FFN vs MLP architectures.

Purpose: Demonstrate time series visualization with aggregation across multiple seeds.

Data Structure:

export interface ExperimentRun {
  prefix: string;              // e.g., "ffn/no-tgt/seed-0"
  metrics: {
    step: number[];           // Training steps [0, 1000, 2000, ...]
    reward: number[];         // Training reward per step
    loss: number[];           // Training loss per step
    "eval.reward": number[];  // Evaluation reward per step
    "eval.loss": number[];    // Evaluation loss per step
  };
}

export const learningCurves: Record<string, ExperimentRun[]> = {
  "ffn/no-tgt": [...],    // Fast learning, plateaus ~850
  "mlp/3_layer": [...],   // Slower learning, better final ~900
};

Characteristics:

  • 200 steps (0-199k)
  • 3 seeds per method
  • FFN learns faster initially but plateaus lower
  • MLP has slower initial learning but achieves better final performance

Usage Example:

const { series } = useSeries({
  glob: "ffn/*",           // Match all seeds
  xKey: "step",
  yKeys: ["reward", "eval.reward"],
  agg: "mean"              // Average across seeds
});

Benchmark Results

Tabular benchmark data across DM Control domains and methods.

Purpose: Demonstrate tabular data queries with filtering.

Data Structure:

export interface BenchmarkResult {
  from: string;              // "lff/no-tgt/cheetah-run/seed-0"
  domain: string;            // "cheetah-run"
  model: string;             // "lff"
  "no-tgt": boolean;         // Configuration flag
  "eval.reward": number;     // Final evaluation reward
  "eval.success_rate"?: number;
  steps: number;             // Total training steps
}

export const benchmarkResults: BenchmarkResult[] = [...];

Contents:

  • 6 domains: cheetah-run, walker-walk, hopper-stand, reacher-easy, finger-spin, cartpole-swingup
  • 5 methods: mlp, ffn, ffn/no-tgt, lff, lff/no-tgt
  • LFF/no-tgt performs best across all domains

Usage Example:

const { data } = useTabular({
  glob: "lff/no-tgt/*",
  filters: { domain: "cheetah-run" }
});

Hyperparameter Sweep

Hyperparameter configurations for parallel coordinates visualization.

Purpose: Demonstrate multi-dimensional data visualization.

Data Structure:

export interface HyperparameterRun {
  from: string;              // "sweep/config-0"
  lr: number;                // Learning rate
  batch_size: number;        // Batch size
  hidden_dim: number;        // Hidden layer dimension
  num_layers: number;        // Number of layers
  gamma: number;             // Discount factor
  tau: number;               // Target network update rate
  entropy_coef: number;      // Entropy coefficient
  "eval.reward": number;     // Final performance
  "eval.success_rate": number;
  converged_step: number;    // Step where training converged
}

export const hyperparameterRuns: HyperparameterRun[] = [...];

Contents:

  • 15 configurations exploring parameter space
  • lr: 0.0001 to 0.01
  • batch_size: 32, 64, 128, 256
  • hidden_dim: 128 to 512
  • Performance metrics for each configuration

Usage Example:

const { data } = useSweep({
  prefix: "sweep",
  filters: { num_layers: 3 }
});

Glob Patterns

Glob patterns provide flexible matching for experiment directories in your data queries.

Basic Patterns

Match All Seeds

glob: "ffn/no-tgt/*"
// Matches: ffn/no-tgt/seed-0, ffn/no-tgt/seed-1, ffn/no-tgt/seed-2, ...

Match Specific Domain

glob: "*/cheetah-run/*"
// Matches: lff/cheetah-run/seed-0, mlp/cheetah-run/seed-1, ...

Match Method Variants

glob: "ffn/*"
// Matches: ffn/no-tgt/..., ffn/with-tgt/..., ffn/baseline/...

Common Use Cases

Average Across Seeds

const { series } = useSeries({
  glob: "mlp/3_layer/*",
  xKey: "step",
  yKey: "reward",
  agg: "mean"  // Average all matched experiments
});

Specific Seed Across Methods

const { series } = useSeries({
  glob: "*/seed-0",
  xKey: "step",
  yKey: "reward",
  agg: null  // Show each method separately
});

All Experiments in a Domain

const { data } = useTabular({
  glob: "*/walker-walk/*",
  filters: { "no-tgt": true }
});

Compare Methods

const methods = ["ffn", "mlp", "lff"];
const allSeries = methods.map(method =>
  useSeries({
    glob: `${method}/*/seed-*`,
    xKey: "step",
    yKey: "reward",
    agg: "mean"
  })
);

Advanced Patterns

Multiple Wildcards

glob: "*/*/*"
// Matches: any/path/with/three/segments

Combining with Prefix

const { series } = useSeries({
  prefix: "experiments/2024",  // Base directory
  glob: "ffn/*/seed-*",        // Pattern within prefix
  xKey: "step",
  yKey: "reward"
});

Glob vs Prefix

  • prefix - Sets the current working directory for the query
  • glob - Pattern to match within the prefix directory
// These are equivalent:
{ prefix: "experiments", glob: "ffn/*" }
{ glob: "experiments/ffn/*" }

// But prefix is cleaner when all queries share a base:
<SeriesDataProvider
  fetchSeries={async (query) => {
    const basePath = "/experiments/2024";
    const fullPath = query.prefix
      ? `${basePath}/${query.prefix}`
      : basePath;
    // Use fullPath + query.glob
  }}
/>

Best Practices

Use Consistent Naming

Organize experiments with predictable structure:

experiments/
  ffn/
    baseline/
      seed-0/
      seed-1/
      seed-2/
    no-tgt/
      seed-0/
      seed-1/
  mlp/
    3_layer/
      seed-0/
      seed-1/

This allows patterns like:

  • {method}/{variant}/seed-* - All seeds for a variant
  • */baseline/* - All baseline experiments
  • ffn/* - All FFN variants

Avoid Over-Matching

Be specific with patterns to avoid unintended matches:

// Too broad - matches everything
glob: "*"

// Better - matches specific method
glob: "ffn/*"

// Best - matches exact variant
glob: "ffn/baseline/seed-*"

Test Your Patterns

Always verify glob patterns return expected results:

const { series, loading } = useSeries({
  glob: "ffn/*/seed-*",
  xKey: "step",
  yKey: "reward"
});

// Check in dev mode
console.log(`Matched ${series.length} series`);

Implementation Notes

Simple Glob to RegExp

The reference implementations convert glob patterns to regular expressions:

function globToRegex(pattern: string): RegExp {
  const escaped = pattern.replace(/\*/g, '.*');
  return new RegExp(`^${escaped}$`);
}

// Usage:
const pattern = globToRegex("ffn/*/seed-*");
const matches = experiments.filter(exp => pattern.test(exp.path));

Production Glob Libraries

For production use, consider using a proper glob library:

import { minimatch } from 'minimatch';

function matchesGlob(path: string, pattern: string): boolean {
  return minimatch(path, pattern);
}

Or with micromatch for better performance:

import micromatch from 'micromatch';

function filterByGlob(paths: string[], pattern: string): string[] {
  return micromatch(paths, pattern);
}

Related Documentation

Previous
useGlob
Next
Data Query Examples