# Real-ESRGAN video upscaler (local, free)

Upscales 480p AI-generated takes (Seedance etc.) 2x–4x before editing. Runs on the Apple GPU
(Metal) or CPU. No cloud, no account, no per-second cost. Model: Real-ESRGAN (BSD-3 licence,
Xintao Wang et al.), weights from the official GitHub releases.

Tested: macOS 26 on Apple Silicon, Python 3.11, PyTorch 2.14. About 1 s per 480x854 frame on
an M-series GPU (a 6 s clip ≈ 2.5 min). Also works on Linux/Windows with CUDA or CPU (slower).

## 1. Install once

Requirements: `ffmpeg` on PATH (`brew install ffmpeg`) and `uv` (`curl -LsSf https://astral.sh/uv/install.sh | sh`).

```bash
cd scripts/upscale
uv venv --python 3.11 venv
uv pip install --python venv/bin/python -r requirements.txt
```

Download the weights into `scripts/upscale/weights/` (about 72 MB total):

```bash
mkdir -p weights && cd weights
curl -LO https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth
curl -LO https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth
cd ..
```

Expected SHA-256:

```
4fa0d38905f75ac06eb49a7951b426670021be3018265fd191d2125df9d682f1  RealESRGAN_x4plus.pth
8dc7edb9ac80ccdc30c3a5dca6616509367f05fbc184ad95b731f05bece96292  realesr-general-x4v3.pth
```

Check with `shasum -a 256 weights/*.pth`. To keep the weights elsewhere, set `ESRGAN_WEIGHTS=/path/to/weights`.

Sanity check (prints the torch version and `True` on an Apple GPU):

```bash
venv/bin/python -c "import torch; print(torch.__version__, torch.backends.mps.is_available())"
```

## 2. Use

```bash
venv/bin/python upscale_esrgan.py <in.mp4> <out.mp4> [model] [scale]
```

- `model`: `x4plus` (default quality pick: natural skin and hair) or `general` (3x faster, slightly smoother/plastic).
- `scale`: output size relative to the input, 2 (default) to 4. The net always runs at 4x and is
  downsampled with area filtering, which is cleaner than a native 2x.
- Output is H.264, CRF 12, same frame rate, no audio (the raw takes carry none; mux audio back with ffmpeg if needed).
- Quick test of one frame to a PNG: `FRAME=60 venv/bin/python upscale_esrgan.py in.mp4 test.png x4plus 2`

Batch a folder:

```bash
mkdir -p raw_up
for f in raw/*.mp4; do venv/bin/python upscale_esrgan.py "$f" "raw_up/$(basename "$f")" x4plus 2; done
```

## 3. Where it fits in the pipeline

Generate at 480p (cheapest) → upscale 2x here → composite phone screens / edit in HyperFrames at
1080x1920 → render with `--video-frame-format png --crf 13`. The upscaled clips are the edit
sources; keep the original 480p raws as the delivered "raw take" next to the final export.

## Limits

- It sharpens what the generator drew; it cannot add real detail, and very soft areas can turn
  slightly waxy. Compare one frame before committing to a full run.
- No temporal model: very fine textures may shimmer slightly between frames.
- Memory: a 480x854 frame at 4x needs about 2 GB of GPU/unified memory. For 720p sources on
  small machines, run `general` or reduce to CPU (slower, no memory limit).

## Files

- `upscale_esrgan.py` — self-contained (network definitions included; no `basicsr` dependency).
- `requirements.txt` — Python packages.
- `weights/` — not committed; download as above.