Feed Configuration Guide

This guide clarifies the three distinct feed management areas in GP3MixMedia TrailMix and when to use each.

Quick Reference

1. Admin Sources (/admin/sources)

Purpose: Define the library of available data sources that campaigns can use.

Who should use this: System administrators only.

What you configure here:

Pre-Configured Sources Tab

• Global Political Sources: National-level news sources available to all campaigns
• State-Specific Sources: Regional news sources filtered by campaign state
• Source types: newspapers, X/Twitter lists
• Ad-buy keywords for content matching

Newsletter Sources Tab

• Newsletter ingestion endpoints
• Email addresses and webhook configurations
• Parser settings for different newsletter formats

Example use cases:

• Adding a new national news outlet to the source library
• Setting up state-specific news sources for a new market
• Configuring a newsletter subscription for the system

Key point: Sources configured here become available for campaigns to enable. They don't automatically run for any campaign until enabled in campaign settings.

2. Ingestion Feeds (/ingestion/feeds)

Purpose: Monitor and control system-wide data ingestion pipelines.

Who should use this: Campaign analysts and system administrators.

What you see here:

• All registered feed types with status indicators
• Last run time and duration for each feed
• Registration status and scheduled cadence
• Error details for failed runs
• Lock status (prevents duplicate runs)

Available feed types:

Actions available:

• Trigger manual feed runs ("Run Now" button)
• View run history and error details
• Monitor feed health metrics

Key point: This page shows the system-wide execution status of all feeds. Individual campaigns must still enable and configure feeds in their settings for data to be routed to them.

3. Campaign Feeds (/campaigns/[id]/settings → Feeds Tab)

Purpose: Configure which data sources are enabled for a specific campaign and set campaign-specific parameters.

Who should use this: Campaign analysts assigned to the campaign.

What you configure here:

FCC Political Files

• DMAs (Markets): Select which markets to monitor
• Advertiser Keywords: Names as they appear in FCC filings
• Regex Patterns: Custom matching rules for advertiser names
• Station Roster: Override automatic station detection

AdImpact

• Endpoint URL: AIDE workflow endpoint
• API Token: Authentication credentials
• Campaign ID Filters: Limit to specific AdImpact campaigns
• Market ID Filters: Limit to specific markets

AIDE Pull/Post Integration

The AIDE/AdImpact integration uses a two-sided webhook model:

1. Outbound "pull" request: GP3MixMedia sends a request to AIDE telling it what data to fetch
2. Inbound "post" response: AIDE delivers results back to our endpoint

Configuration Fields:

Office Types:

• governor
• lt_governor
• attorney_general
• us_senate
• us_house (requires district_number)
• state_senate_district (requires district_number)
• state_house_district (requires district_number)

Race String Format: The outbound webhook body includes a race field with one of these formats:

• [ST] Governor [YYYY]
• [ST] Lt. Governor [YYYY]
• [ST] Attorney General [YYYY]
• [ST] Senate [YYYY]
• [ST] CD-[##] [YYYY] (Congressional District, e.g., "MA CD-07 2024")
• [ST] SD-[##] [YYYY] (State Senate District)
• [ST] HD-[##] [YYYY] (State House District)

District numbers are left-padded to 2 digits (e.g., 7 becomes "07").

Outbound Webhook Body:

{
  "race": "MA Governor 2024",
  "date_of_last_run": "2024-01-15T10:30:00.000Z",
  "look_back_date": "2024-01-14"
}

The date_of_last_run is derived from the last successful ingest. If no prior success, defaults to now minus 7 days. The look_back_date is always date_of_last_run minus 1 day.

Inbound POST Endpoint:

• URL: /api/v1/aide/inbound
• Authentication: Bearer token or token query parameter
• Required: campaignId query param OR correlationId OR matching race/date fields

Token Placement: By default, tokens are sent in both:

• Query parameter: ?token=...
• Authorization header: Authorization: Bearer <token>

This can be configured via aide_token_placement: query_param, bearer_header, or both.

OpenFEC

• FEC Committee ID: Required (format: C00XXXXXX)
• FEC Candidate ID: Optional
• Filing Types: Schedule E, Electioneering Communications, 24-Hour IE Reports

Meta Ads Library

The Meta Ads Library feed pulls political and issue ads from Facebook and Instagram via the Meta Ad Library API.

Prerequisites (System-Level)

Before the Meta feed can work, an admin must configure these environment variables (or Fly.io secrets):

How to get a Meta access token:

1. Go to Meta for Developers and create a developer account
2. Create a new app (type: Business) or use an existing one
3. Add the "Marketing API" product to your app
4. Apply for Ad Library API access (requires identity verification)
5. Once approved, generate a long-lived access token:
   • Go to the Graph API Explorer
   • Select your app
   • Add ads_read permission
   • Generate a User Access Token
   • Exchange it for a long-lived token (valid 60 days):

     GET https://graph.facebook.com/v20.0/oauth/access_token?
       grant_type=fb_exchange_token&
       client_id={app-id}&
       client_secret={app-secret}&
       fb_exchange_token={short-lived-token}
     

6. Set the token: fly secrets set META_ADS_API_TOKEN=<token> --app gp3mixmedia-web

Token Renewal: Long-lived tokens expire after 60 days. Set a calendar reminder to regenerate and rotate the token before expiry. If the token expires, Meta feed runs will fail with auth errors visible in /ingestion/feeds.

Campaign-Level Configuration

Once the system-level token is configured, enable Meta Ads for individual campaigns:

1. Go to Campaign Settings → Feeds tab
2. Find the Meta Ads Library section
3. Configure:

Tips for search terms:

• Use candidate names, PAC names, and committee names
• The system will also auto-derive terms from campaign metadata if search terms are empty
• If META_ADS_ALLOW_KEYWORD_FALLBACK is true (default), the system falls back to keyword search when page-based search yields few results

How the Feed Works

1. Runs on the META_ADS_CRON schedule (default: every 8 hours)
2. For each campaign with Meta Ads enabled:
   • Builds search queries from configured or auto-derived terms
   • Queries graph.facebook.com/v20.0/ads_archive for political/issue ads
   • Paginates through results (up to META_ADS_MAX_PAGES pages)
   • Applies inter-request delay (META_ADS_INTER_REQUEST_DELAY_MS) to avoid rate limiting
   • Falls back to keyword search if primary search returns few results
3. Matching ads are converted to buy line candidates with fields: advertiser, spend range, impressions, platform, creative URL
4. Buy line candidates go through deduplication before appearing in the campaign Matrix
5. Circuit breaker protects against API failures; partial results are saved on timeouts

Troubleshooting Meta Feed

Key Files

• Feed worker: src/workers/feeds/meta-ads-library.ts
• API client: lib/services/social-creative/meta-ads-library.client.ts

Google Political Ads

• Advertiser Discovery: Auto-discovers advertisers from campaign keywords
• Geographic Filtering: Enhanced geo matching by state/region
• Date Range: Historical data lookback period
• Data Source: Google BigQuery public dataset
• Relevance Validation: Automatic detection of unrelated advertisers (wrong-campaign guardrail)

Advertiser Relevance Guardrail: The system validates that ingested advertisers are relevant to the campaign's state. If more than 30% of advertisers appear unrelated to the campaign state/office:

• An anomaly is logged in the feed run metadata
• A recommendation is provided (proceed, fallback_to_keywords, or abort)
• This helps detect and prevent wrong-campaign ingestion issues

Date Acceptance Rules for Buy Lines

When creating buy lines from ingested data, the system accepts records with various date fields in priority order:

1. start_date / end_date - Primary date fields
2. spend_period - Combined date range (e.g., "9/28/25 - 10/4/25")
3. flight_start_date / flight_end_date - Flight date fields
4. air_date - When the ad actually aired
5. spend_date - When spend was recorded
6. booking_date - When the ad was booked
7. update_date - Last update timestamp

Key rules:

• Records are NOT rejected if start_date/end_date are missing, as long as at least one alternative date field exists
• When only a single date is available (e.g., just air_date), it's used for both start and end date
• The minimum required fields for a valid buy line are:
  • Advertiser (org_name, advertiser_name, or similar)
  • Amount/Spend (one of: gross_cents, spend_cents, amount, etc.)
  • At least one date field from the priority list above

Mailbox Routing

• Define email pattern matching rules
• Route emails from specific senders to this campaign
• Toggle rules on/off

Key point: This is where you connect global data sources to your specific campaign. Without configuration here, data won't be routed to your campaign even if the feed is running.

Data Flow Overview

┌─────────────────────────────────────────────────────────────────┐
│  1. ADMIN: Define Sources (/admin/sources)                      │
│     → Creates library of available data sources                 │
└──────────────────────────────┬──────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│  2. MONITOR: Ingestion Feeds (/ingestion/feeds)                 │
│     → View all feed execution status                            │
│     → Trigger manual runs                                       │
│     → Monitor health and errors                                 │
└──────────────────────────────┬──────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│  3. CONFIGURE: Campaign Settings (/campaigns/[id]/settings)    │
│     → Enable feeds for specific campaign                        │
│     → Set campaign-specific parameters (DMAs, keywords, etc.)   │
│     → Create routing rules                                      │
└──────────────────────────────┬──────────────────────────────────┘
                               │
                               ▼
┌─────────────────────────────────────────────────────────────────┐
│  4. RESULT: Data appears in campaign                            │
│     → Raw Inputs, Matrix, Dedupe, etc.                         │
└─────────────────────────────────────────────────────────────────┘

Common Questions

"I added sources in Admin but don't see any data"

Sources in /admin/sources define available sources. You must:

1. Go to the campaign's Settings → Feeds tab
2. Enable the specific feeds you want
3. Configure campaign-specific parameters (keywords, DMAs, etc.)

"My feed shows 'Error' in the Ingestion Feeds page"

1. Click on the feed to see error details
2. Check if it's a temporary network issue (will auto-retry)
3. For persistent errors, check configuration in /admin/sources
4. Contact system admin if credentials or endpoints need updating

"What's the difference between global and state news sources?"

• Global: Available to all campaigns (national political news)
• State: Filtered by campaign state (local/regional news) Both are configured in /admin/sources and enabled per-campaign in Settings.

"How often do feeds run?"

Each feed type has a default schedule (see table above). You can:

• View last run time in /ingestion/feeds
• Trigger immediate runs with "Run Now" button
• Note: Global schedule cannot be changed per-campaign

Permissions by Role

Related Documentation

• User Guide: Setting Up Automated Data Collection
• Site Architecture: Feed Management Locations

Last Updated: 2026-03-26 (Meta Ads Library setup guide)