BrainLayer Data Locations

LLM enrichment is retired. Its feature and run/resume instructions below are historical. Existing metadata, local pipelines, and legacy writer-stop checks remain available. History: enrichment retirement.

Single source of truth for where all data lives, where it moved from, and the archive strategy.

Active Data

What	Path	Size	Notes
Main database	`~/.local/share/brainlayer/brainlayer.db`	~8 GB	297K+ chunks, sqlite-vec + FTS5
Current sessions	`~/.claude/projects/{encoded-path}/*.jsonl`	~805 files	Claude Code session transcripts
Archived sessions	`~/.claude-archive/{project-id}/archive-{timestamp}/`	1.2 GB	Moved by session-archiver

Path Resolution

BrainLayer resolves the database path in this order (see src/brainlayer/paths.py):

BRAINLAYER_DB env var — explicit override
~/.local/share/brainlayer/brainlayer.db — canonical path

Session Archiver

Service: com.brainlayer.session-archiver (launchd, runs daily at 4am)

How it works:

Scans ~/.claude/projects/ for all session JSONL files
Keeps last 7 days of active sessions per project
Moves older sessions to ~/.claude-archive/{project-id}/archive-{timestamp}/
Writes manifest.json per batch (UUIDs, timestamps, sizes)
After BrainLayer indexes the archived sessions, the archiver cleans up verified copies

Archive structure:

~/.claude-archive/
  my-project/
    archive-2026-02-09T02-00-05/
      {uuid}.jsonl           # Archived session transcript
      {uuid}/                # Optional: subagent files
      manifest.json          # Batch metadata
    archive-2026-02-10T02-00-05/
      ...
  domica/
    ...
  songscript/
    ...

Manifest format:

{
  "archivedAt": "2026-02-09T02:00:05.123Z",
  "projectId": "my-project",
  "originalPath": "/Users/username/Gits/my-project",
  "sessions": [
    {
      "uuid": "abc123...",
      "originalMtime": "2026-02-07T15:30:00.000Z",
      "size": 524288,
      "hasSubdir": true,
      "firstMessageTimestamp": "2026-02-07T15:28:42.123Z",
      "gitBranch": "feature/some-branch"
    }
  ],
  "metadata": {
    "archiver_version": "1.1.0",
    "sessions_kept": 7,
    "total_archived": 12,
    "total_size_bytes": 6291456
  }
}

Natural-state transcript backup contract

Each claude-jsonl-YYYY-MM-DD.tar.gz bundle stores every selected transcript as a regular raw file under source-{root-index}/<source-relative-path> outside the BrainLayer database, and verification must stream-compare every tar member byte-for-byte with the source before the bundle counts as a natural-state copy; an exact match or a strict prefix of a source that only grew after bundling is valid, while any divergence within the archived bytes fails the entire bundle before upload.

Backups (Manual)

Before any bulk operation, back up the database:

# WAL-safe copy
sqlite3 ~/.local/share/brainlayer/brainlayer.db "VACUUM INTO '/path/to/backup/brainlayer-$(date +%Y%m%d).db'"

Store backups in your preferred location (iCloud, external drive, etc.).

Historical: Data Migrations

Repo path change (Jan-Feb 2026)

Repos moved from ~/Desktop/Gits/ to ~/Gits/. This means: - Old chunks reference ~/.claude/projects/-Users-username-Desktop-Gits-{repo}/ - New chunks reference ~/.claude/projects/-Users-username-Gits-{repo}/ - The old JSONL session files at the Desktop paths no longer exist

Session archiver setup (Feb 9, 2026)

Before the archiver was set up, old sessions were manually deleted. ~160K chunks reference sessions that no longer exist anywhere. These chunks are still searchable — they just don't have created_at timestamps.

BrainLayer extraction (Feb 19, 2026)

Extracted to standalone repository. Code moved, data consolidated at ~/.local/share/brainlayer/brainlayer.db (March 2026).

Vertex AI Batch Enrichment (Feb 17-18, 2026)

153,825 chunks submitted to Vertex AI batch prediction
Results imported Feb 18 at 08:00 (135,865 chunks enriched)
Job tracking: scripts/backfill_data/vertex_jobs.json
Predictions stored in: scripts/backfill_data/predictions/

Coverage Stats (as of Feb 19, 2026)

Metric	Count	Percentage
Total chunks	268,864	100%
Have `created_at`	107,935	40.1%
Missing `created_at`	160,929	59.9%
Enriched	144,146	53.6%
Enrichable but not enriched	22,974	8.5%
Too small to enrich (<50 chars)	101,744	37.8%

The 160K chunks without created_at are from pre-archiver sessions whose JSONL files were deleted. The chunks themselves are fully indexed and searchable — date filtering just won't apply to them (they'll always be included in unfiltered searches).