Automated Keyword Cannibalization Audits with Python and GSC API

Why Manual Spreadsheet Audits Fail at Scale

Connecting directly to the Google Search Console API allows technical SEOs to automate detection of internal URL conflicts without relying on manual spreadsheet exports. Do not export everything to CSV and call it an audit. When managing large sites or monitoring the rapid output of AI-assisted content pipelines, manual inspection of Search Console UI reports is insufficient. A site with tens of thousands of landing pages generates hundreds of thousands of query-URL pairs every month, making manual spot-checks ineffective for detecting keyword cannibalization.

Executing a systematic keyword cannibalization audit requires granular performance metrics across time. When two or more URLs continuously compete for identical search queries, impression share gets diluted, click-through rates plummet, and Google struggles to determine the authoritative canonical destination. Before writing diagnostic scripts, it is essential to focus on real conflict rather than expected SERP variations by differentiating between destructive cannibalization and semantic overlap.

Setting Up Google Search Console API Authentication

The practical route is simple: authenticate via a GCP Service Account to bypass repeated OAuth browser prompts when executing automated background scripts. Follow this google search console api tutorial setup to configure programmatic access.

First, enable the Google Search Console API in the Google Cloud Console, create a Service Account, and download the JSON key file. Next, add the Service Account email address as a user with Full or Read permissions inside Google Search Console for your target property.

Install the required Python client packages:

pip install google-api-python-client google-auth pandas

Use the following Python snippet to establish an authenticated connection to the Search Console API:

from google.oauth2 import service_account
from googleapiclient.discovery import build

SCOPES = ['https://www.googleapis.com/auth/webmasters.readonly']
SERVICE_ACCOUNT_FILE = 'credentials.json'
PROPERTY_URI = 'sc-domain:example.com'

def get_gsc_service():
    creds = service_account.Credentials.from_service_account_file(
        SERVICE_ACCOUNT_FILE, scopes=SCOPES
    )
    return build('searchconsole', 'v1', credentials=creds)

service = get_gsc_service()

With authentication in place, you can query Search Console data programmatically without UI extraction limits.

Extracting and Processing Query-URL Performance Data

To detect content cannibalization, you must query performance data grouped by both query and page. Querying by single dimensions will not reveal which URLs share rankings for identical search terms.

The Search Console API returns maximums of 25,000 rows per request. For large properties, you must paginate using startRow iterations over a standard 30-day or 90-day window. The script below pulls query and page metrics into a Pandas DataFrame:

import pandas as pd

def fetch_gsc_data(service, property_uri, start_date, end_date, row_limit=25000):
    all_rows = []
    start_row = 0

    while True:
        request = {
            'startDate': start_date,
            'endDate': end_date,
            'dimensions': ['query', 'page'],
            'rowLimit': row_limit,
            'startRow': start_row
        }
        response = service.searchanalytics().query(siteUrl=property_uri, body=request).execute()
        rows = response.get('rows', [])

        if not rows:
            break

        for row in rows:
            all_rows.append({
                'query': row['keys'][0],
                'page': row['keys'][1],
                'clicks': row['clicks'],
                'impressions': row['impressions'],
                'ctr': row['ctr'],
                'position': row['position']
            })

        start_row += row_limit
        if len(rows) < row_limit:
            break

    return pd.DataFrame(all_rows)

df = fetch_gsc_data(service, PROPERTY_URI, '2026-07-01', '2026-07-31')

This structure gives us the exact dataset required to compute cannibalization logic programmatically.

Building the Python Cannibalization Logic

This is where the problem usually appears: SEO teams treat every multi-URL query as cannibalization. True cannibalization occurs when traffic and impressions are split between multiple pages, causing organic volatility.

To build a meaningful automated seo audit module, filter the DataFrame to identify queries with 2 or more active URLs meeting minimum performance thresholds (e.g., at least 50 total impressions):

def analyze_cannibalization(df, min_impressions=50):
    # Filter low-traffic noise
    filtered = df[df['impressions'] >= 10].copy()

    # Group by query and count distinct competing pages
    query_stats = filtered.groupby('query').agg(
        url_count=('page', 'nunique'),
        total_clicks=('clicks', 'sum'),
        total_impressions=('impressions', 'sum')
    ).reset_index()

    # Isolate queries with multiple URLs competing
    conflicting_queries = query_stats[
        (query_stats['url_count'] > 1) & 
        (query_stats['total_impressions'] >= min_impressions)
    ]

    # Merge back to get detailed URL rows
    cannibalized_df = df[df['query'].isin(conflicting_queries['query'])].sort_values(
        by=['query', 'clicks'], ascending=[True, False]
    )

    return cannibalized_df

cannibalization_report = analyze_cannibalization(df)

The table below contrasts traditional manual workflows with an automated Python pipeline:

Validating Data and Remediating URL Conflicts

Do not execute redirects or canonical updates based solely on API output. A crawl is evidence, not the whole truth. API insights must be validated against real-time site architecture and indexation state.

Before taking technical action, cross-reference API findings by using crawl data to supplement API insights. This helps confirm whether internal linking anomalies, duplicate H1 tags, or conflicting canonical tags are causing Google to serve multiple pages.

When reviewing conflicting URLs in your output, evaluate the root technical issue before implementing remedies:

• Consolidation via 301 Redirects: If two articles cover identical intent, merge the weaker page's content into the primary URL and issue a 301 redirect.
• Canonical Tag Alignment: Ensure subordinate URL variants explicitly canonicalise to the master target URL.
• Internal Link Re-architecting: Remove anchor text pointing to competing pages for target terms to strengthen clear entity signals.

When evaluating indexation anomalies, focus on distinguishing between indexing signals and actual errors to ensure you are fixing genuine structural issues rather than temporary SERP testing. Addressing root conflicts is critical to establishing a robust technical SEO foundation.

Automating Reports and Continuous Monitoring

Integrating seo automation into your operational workflow prevents cannibalization issues from compounding silently over time.

Export the finalized report to automated alerts (Slack/Email) or upload the DataFrame directly into Google BigQuery or PostgreSQL. Executing this script on a weekly schedule using GitHub Actions or Google Cloud Run turns a reactive audit into an automated monitoring system.

Prioritise fixes by commercial impact and total impression share lost. Addressing critical keyword conflict early preserves search equity, protects organic revenue, and reduces technical debt across your publishing systems.