Edge Function Upload Architecture (V2)

As of January 2026, Sable uses Edge Functions as the single source of truth for file uploads. This replaces the complex Bronze→Silver→Gold dbt pipeline with direct database inserts.

Why Edge Functions?

The original dbt pipeline had too many moving parts:

• 6-step flow (upload → detection → bronze → job → dbt → verification)
• Multiple polling services (file_watcher, worker)
• Latency: minutes from upload to visible data
• Debugging difficulty: failures could occur at any step

New architecture benefits:

• Single endpoint handles everything
• Sub-second upload to database
• Both CLI and browser use same code path
• Easy to debug (one place to look)

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     UPLOAD PATH (V2)                              │
│                                                                   │
│   CLI (sable.js)  ──┐                                            │
│                     ├──→ Edge Function ──→ sable.* tables        │
│   Browser Upload  ──┘                      + file_upload record  │
│                                                                   │
└─────────────────────────────────────────────────────────────────┘

Edge Functions

sable_btig_upload_v2

Endpoint: https://hrezmztjmvzzibfhvpyo.supabase.co/functions/v1/sable_btig_upload_v2

Handles:

• PnL files → sable.pnl_daily_v2
• Perf files → sable.perf_daily_v2
• Evidence PDFs → Supabase Storage

Features:

• CSV parsing with type conversion
• Account code → entity_id lookup via sable.account
• SHA-256 deduplication
• PDF reconciliation (CSV totals vs PDF totals)
• Soft delete before insert (fn_upload_pnl_v2, fn_upload_perf_v2)
• file_upload record for audit trail

sable_trans_upload_v2

Endpoint: https://hrezmztjmvzzibfhvpyo.supabase.co/functions/v1/sable_trans_upload_v2

Handles:

• Transaction files → sable.trans_daily_v2

Features:

• CSV parsing (handles MM/DD/YYYY dates, accounting numbers)
• Account code extraction from filename
• SHA-256 deduplication
• Empty file tracking (tracked for audit, not rejected)
• Soft delete before insert (fn_upload_trans_v2)
• Batch insert (500 rows per batch)

Upload Flow

1. Client sends multipart/form-data
   - file: CSV content
   - org_id: Organization UUID
   - force: "true" to bypass deduplication

2. Edge Function processes:
   a. Parse CSV content
   b. Compute SHA-256 hash
   c. Check for duplicate (unless force)
   d. Soft delete existing records for date range
   e. Insert new records in batches
   f. Create file_upload record

3. Response includes:
   - rows_inserted
   - rows_soft_deleted
   - account_code
   - date_range
   - file_hash
   - file_upload_id

CLI Usage

# Full pipeline (download + upload)
sable uploads pipeline --dates 2026-01-14

# Skip download, force re-upload
sable uploads pipeline --skip-download --force

# Single file upload
sable upload pnl /path/to/pnl_file.csv

Authentication

Edge Functions use service role key for database access:

// CLI (sable.js)
headers: {
  'Authorization': `Bearer ${process.env.SUPABASE_SERVICE_ROLE}`
}

Soft Delete Pattern

Before inserting new data, existing records for the same date range are soft-deleted:

-- fn_upload_trans_v2(p_account_code, p_start_date, p_end_date)
UPDATE sable.trans_daily_v2
SET deleted_at = NOW()
WHERE account_code = p_account_code
  AND trade_date BETWEEN p_start_date AND p_end_date
  AND deleted_at IS NULL;

This allows:

• Re-uploading corrected data
• Auditing what was replaced
• Recovery if needed

Deduplication

Files are tracked by SHA-256 hash:

SELECT id, filename, created_at
FROM sable.file_upload
WHERE file_hash = $1
  AND org_id = $2
  AND deleted_at IS NULL;

Use force=true to bypass and re-upload anyway.

Empty File Handling

Empty CSV files (header only, no data rows) are tracked but not rejected:

{
  "success": true,
  "skipped": true,
  "reason": "empty_file",
  "message": "File contains no data rows - tracked for audit"
}

This maintains audit trail of what was downloaded from BTIG.

Tables

Table	Purpose
sable.pnl_daily_v2	Daily P&L by position
sable.perf_daily_v2	Daily returns (Modified Dietz)
sable.trans_daily_v2	Transaction history (MTD)
sable.file_upload	Upload audit trail

Comparison: V1 dbt vs V2 Edge Functions

Aspect	V1 (dbt)	V2 (Edge Functions)
Latency	Minutes	Sub-second
Code paths	6+ services	1 endpoint
Debugging	Multiple logs	Single function
Complexity	High	Low
Browser support	Via Vercel	Direct to Edge Function

Version History

• 3.0.0-edge (2026-01-14): Trans uploads via Edge Function
• 2.0.0 (2026-01): Initial Edge Function architecture