NB Defense: Security Scanning for Jupyter Notebooks Before Your Secrets Hit GitHub

Hook

Every data scientist who's ever committed a Jupyter notebook to GitHub has probably leaked at least one API key—and most don't know it. The .ipynb format stores not just your code, but execution outputs, metadata, and embedded visualizations that traditional security scanners completely miss.

Context

Jupyter notebooks have become the de facto development environment for machine learning and data science, but they exist in a security blind spot. Traditional Static Application Security Testing (SAST) tools are built for .py files and struggle with the JSON-based .ipynb format where code is embedded within cell arrays. Even worse, notebooks capture execution state—meaning that API keys printed in output cells, PII displayed in DataFrames, or sensitive paths in error tracebacks all get serialized into the file. Security teams have tools like detect-secrets and Bandit for production code, but the experimentation phase where notebooks dominate has largely been a free-for-all.

This gap is particularly dangerous because notebooks are highly portable and frequently shared. Data scientists commit them to repositories, upload them to collaboration platforms, attach them to support tickets, and share them in Slack channels. Each sharing action multiplies the exposure surface. Protect AI built NB Defense specifically to address this problem, creating a scanner that understands notebook structure and can inspect not just code cells, but outputs, metadata, and dependency declarations. It's designed to integrate into both CI/CD pipelines and the interactive development workflow through a companion JupyterLab extension.

Technical Insight

NB Defense operates as both a CLI tool and an importable SDK, scanning the JSON structure of .ipynb files with purpose-built parsers. When you point it at a notebook, it deserializes the JSON, extracts code from input cells, inspects output cells for leaked data, and analyzes any requirements.txt or dependency manifests it finds. The architecture is modular, with separate scanning plugins for secrets detection, PII identification, CVE scanning, and license compliance.

Here's a basic CLI scan:

# Install nbdefense
pip install nbdefense

# Scan a single notebook
nbdefense scan notebook.ipynb

# Scan an entire directory
nbdefense scan ./notebooks/

# Output in JSON format for CI/CD integration
nbdefense scan --json --output-file results.json ./notebooks/

The real architectural insight is how NB Defense handles the dual nature of notebooks as both source code and execution artifacts. Consider this notebook cell:

import os
api_key = os.getenv('OPENAI_API_KEY', 'sk-proj-default-key-12345')
response = openai.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Analyze this data"}]
)
print(f"Request completed with key: {api_key}")

A traditional scanner might catch the hardcoded fallback key in the source, but when this cell executes, the print statement reveals the actual API key in the output. NB Defense scans both the input cell's source code AND the output cell's text, catching secrets that only appear during execution. This is crucial because data scientists often debug by printing variables, and those outputs get permanently serialized into the notebook file.

The SDK mode enables programmatic integration and powers the JupyterLab extension. You can build custom security workflows:

from nbdefense import scan_notebook
from pathlib import Path

# Scan a notebook programmatically
results = scan_notebook(Path('experiment.ipynb'))

for issue in results.issues:
    if issue.severity == 'HIGH':
        print(f"[{issue.code}] {issue.description}")
        print(f"Location: Cell {issue.cell_index}, Line {issue.line}")
        print(f"Content: {issue.matched_string}\n")

# Integration example: block commits with HIGH severity issues
if any(i.severity == 'HIGH' for i in results.issues):
    raise SystemExit(1)

This SDK approach makes NB Defense extensible. You can wrap it in pre-commit hooks, integrate it into MLOps platforms, or build custom reporting dashboards. The JupyterLab extension uses this same SDK to provide real-time scanning—it watches for notebook saves and displays security warnings directly in the interface, shifting security feedback left into the experimentation phase.

The dependency scanning component deserves special attention. NB Defense doesn't just look for import statements in code cells; it searches for requirements.txt, conda.yml, Pipfile, and poetry.lock files in the notebook's directory, then cross-references dependencies against CVE databases. This catches the common pattern where data scientists install packages directly in notebooks with !pip install commands but maintain separate dependency manifests for environment reproduction. The tool correlates both sources to give a complete vulnerability picture.

One clever architectural decision is the plugin system for detection rules. Rather than reinventing secret detection, NB Defense integrates with established pattern libraries but adds notebook-specific context. For example, it knows to inspect image metadata in output cells (since notebooks can embed PNG/SVG visualizations), check for credentials in Plotly or Bokeh chart configurations, and scan markdown cells for accidentally pasted tokens. This context-awareness is what differentiates it from generic scanners.

Gotcha

The biggest limitation is that NB Defense is early-stage tooling from a focused vendor, which means the detection rule library isn't as comprehensive as mature general-purpose scanners. With only 88 GitHub stars, community contributions are limited, so you may encounter false negatives with less common secret formats or proprietary internal patterns. You'll likely need to supplement it with custom regex patterns or maintain an allowlist for false positives specific to your environment.

Performance can also be a concern with large notebooks or directory scans. Since NB Defense deserializes the entire JSON structure and scans every cell's inputs and outputs, notebooks with extensive execution history or embedded large outputs (like base64-encoded images) can slow scanning considerably. There's no incremental scanning mode, so CI/CD pipelines need to scan the entire notebook set on each run. For repositories with hundreds of notebooks, this becomes noticeable. The tool also lacks sophisticated deduplication—if the same secret appears in multiple cells or notebooks, you'll get multiple findings rather than a single aggregated issue.

Verdict

Use NB Defense if you're on an ML or data science team that lives in Jupyter notebooks and needs to establish security guardrails before notebooks get committed, shared, or promoted to production. It's especially valuable if you want real-time security feedback during experimentation via the JupyterLab extension, or if you need a single tool that handles secrets, PII, and dependency CVEs without stitching together multiple scanners. The SDK mode makes it perfect for building custom security workflows in MLOps platforms. Skip it if you've already transitioned to production Python codebases where traditional SAST tools work fine, or if you need enterprise-grade detection rules with extensive customization and vendor support that mature tools provide. Also skip if you're looking for runtime security or model-specific threats—NB Defense is purely a static analysis tool for the notebook artifact itself.