================================================================================
  PROJECT TREE
================================================================================
📁 pipeline_project/
│   📄 .env  (1.0 KB)
│   📄 README.md  (1.0 KB)
│   📄 RealESRGAN_x4plus.pth  (65469.7 KB)
│   📄 hotfix3.py  (2.4 KB)
│   📄 hotfix_encoder.py  (2.2 KB)
│   📄 pipeline_state.db  (28.0 KB)
│   📄 pipeline_state.db-shm  (32.0 KB)
│   📄 pipeline_state.db-wal  (28.2 KB)
│   📄 pythondumb.py  (2.4 KB)
│   📄 requirements.txt  (0.2 KB)
│   📄 setup.py  (3.9 KB)
│   📄 upgrade_v103b.py  (29.4 KB)
│   📄 yolov8n.onnx  (12549.9 KB)
│   📄 yolov8n.pt  (6396.3 KB)
│   📁 1_input_videos/
│   │   📄 1mw_H9Alvlg.mp4  (18234.8 KB)
│   │   📄 9b5QuemMJRg.mp4  (12131.8 KB)
│   │   📄 9nq0id5ZSqQ.mp4  (428430.2 KB)
│   │   📄 DOAlIn85m2o.mp4  (48120.2 KB)
│   │   📄 F0sqfPkCcyM.mp4  (15572.0 KB)
│   │   📄 Mw4rNP3oQto.mp4  (438839.2 KB)
│   │   📄 N5DvzdbukD8.mp4  (598266.4 KB)
│   │   📄 Q1FucR5gaPU.mp4  (8056.7 KB)
│   │   📄 Wc51nE-wqvA.mp4  (219265.0 KB)
│   │   📄 c1rBk7XAlj0.mp4  (111229.3 KB)
│   │   📄 cdbPVz890ZQ.mp4  (23138.3 KB)
│   │   📄 test.mp4  (21474.0 KB)
│   📁 2_input_music/
│   📁 logs/
│   │   📄 pipeline.log  (73.9 KB)
│   📁 output_ready/
│   📁 pipeline/
│   │   📄 __init__.py  (0.0 KB)
│   │   📄 acquisition.py  (1.8 KB)
│   │   📄 audio.py  (1.1 KB)
│   │   📄 brain.py  (6.6 KB)
│   │   📄 core.py  (13.1 KB)
│   │   📄 decoder.py  (2.5 KB)
│   │   📄 detector.py  (2.1 KB)
│   │   📄 dml_guard.py  (0.6 KB)
│   │   📄 encoder.py  (1.9 KB)
│   │   📄 engine.py  (9.8 KB)
│   │   📄 interpolator.py  (3.7 KB)
│   │   📄 main.py  (5.6 KB)
│   │   📄 meme_fx.py  (2.4 KB)
│   │   📄 menu.py  (0.9 KB)
│   │   📄 parallel.py  (1.4 KB)
│   │   📄 scene.py  (0.8 KB)
│   │   📄 tts.py  (2.4 KB)
│   │   📄 upscaler.py  (3.5 KB)
│   │   📄 utilities.py  (2.2 KB)
│   │   📄 voice_extractor.py  (8.7 KB)
│   📁 temp_processing/
│   │   📄 concat.txt  (1.0 KB)
│   │   📄 narration.wav  (4577.6 KB)
│   │   📄 seg_0.mp4  (0.3 KB)
│   │   📄 seg_1.mp4  (0.3 KB)
│   │   📄 seg_10.mp4  (0.3 KB)
│   │   📄 seg_11.mp4  (0.3 KB)
│   │   📄 seg_12.mp4  (0.3 KB)
│   │   📄 seg_13.mp4  (0.3 KB)
│   │   📄 seg_14.mp4  (0.3 KB)
│   │   📄 seg_2.mp4  (0.3 KB)
│   │   📄 seg_3.mp4  (0.3 KB)
│   │   📄 seg_4.mp4  (0.3 KB)
│   │   📄 seg_5.mp4  (0.3 KB)
│   │   📄 seg_6.mp4  (0.3 KB)
│   │   📄 seg_7.mp4  (0.3 KB)
│   │   📄 seg_8.mp4  (0.3 KB)
│   │   📄 seg_9.mp4  (0.3 KB)
│   📁 voices/

================================================================================
  FILE CONTENTS
================================================================================

────────────────────────────────────────────────────────────
FILE: README.md
SIZE: 1.0 KB
────────────────────────────────────────────────────────────
# Pipeline v10.2 — Anime Meme Content Factory

## Features
- **Supertonic 3 TTS**: 31 dil, Turkce dahil, ONNX on-device
- **RIFE Frame Interpolation**: rife-ncnn-vulkan ile FPS artirma
- **Gemma 4 E2B Brain**: Ollama ile sahne analizi + narration
- **Real-ESRGAN**: GPU destekli goruntu buyutme
- **YOLOv8**: DirectML ile nesne tespiti
- **Meme FX**: zoom_cut, speed_lines, camera_shake, subtitle_bounce
- **3-Thread Pipeline**: Decode -> Process -> Encode paralel
- **FFmpeg AMF**: AMD GPU h264 encode + d3d11va decode

## Quick Start
```
cd pipeline_project
python -m pipeline.main
```

## Menu
1. Full Auto Pipeline (arama + indirme + isleme + TTS + birlestirme)
2. Manuel Video Secimi
3. TTS Only (Supertonic 3)
4. Upscale Only (Real-ESRGAN)
5. Brain Analysis (Gemma 4 E2B)
6. Meme FX Only
7. Scene Analysis
8. Audio Analysis
9. Settings / Health Check
10. RIFE Interpolation

## Requirements
- Python 3.12+
- Windows 11
- RX 9070 XT 16GB (DirectML)
- FFmpeg (PATH or tools/)


────────────────────────────────────────────────────────────
FILE: hotfix3.py
SIZE: 2.4 KB
────────────────────────────────────────────────────────────
#!/usr/bin/env python3
"""hotfix_critical.py — Fix 3 critical bugs: empty segments, bad narration, false JOB FAILED."""
import os

def patch(path, replacements):
    with open(path, "r", encoding="utf-8") as f:
        c = f.read()
    for old, new, desc in replacements:
        if old in c:
            c = c.replace(old, new)
            print("  [OK] " + desc)
        else:
            print("  [--] " + desc + " (not found)")
    with open(path, "w", encoding="utf-8") as f:
        f.write(c)
    compile(c, path, "exec")

def main():
    if not os.path.isdir("pipeline"):
        print("[!!] pipeline/ bulunamadi")
        return

    print("=== engine.py ===")
    patch("pipeline/engine.py", [
        (
            'dur = sc.get("duration", None)',
            'es = sc.get("end_sec", ss + 6)\n            dur = es - ss',
            "Bug 1: segment duration hesaplama"
        ),
        (
            "if final and os.path.isfile(final):",
            "if final and os.path.isfile(final) and os.path.getsize(final) > 1000:",
            "Bug 3: bos dosya kontrolu"
        ),
    ])

    print("\n=== brain.py ===")
    patch("pipeline/brain.py", [
        (
            "dramatik ve komik bir Turkce narration yaz. 2-3 cumle.",
            "dramatik ve komik bir Turkce narration yaz. "
            "KURALLAR: Sadece duz metin yaz. Markdown kullanma. "
            "Secenekler listeleme. Tirnak isareti kullanma. "
            "Baslik yazma. Tek bir versiyon yaz. Maksimum 2-3 cumle.",
            "Bug 2: narration prompt sıkılaştırma (karakter)"
        ),
        (
            "dramatik ve komik bir Turkce narration yaz. Kisa ve etkileyici olsun.",
            "dramatik ve komik bir Turkce narration yaz. "
            "KURALLAR: Sadece duz metin yaz. Markdown kullanma. "
            "Secenekler listeleme. Tirnak isareti kullanma. "
            "Baslik yazma. Tek bir versiyon yaz. 2-3 cumle, kisa ve etkileyici.",
            "Bug 2: narration prompt sıkılaştırma (genel)"
        ),
    ])

    print()
    print("=" * 55)
    print("  3 kritik bug duzeltildi:")
    print("    1. Segmentler artik gercekten kesilecek")
    print("    2. Narration duz metin olacak")
    print("    3. Bos dosya JOB DONE demeyecek")
    print("  python -m pipeline.main")
    print("=" * 55)

if __name__ == "__main__":
    main()


────────────────────────────────────────────────────────────
FILE: hotfix_encoder.py
SIZE: 2.2 KB
────────────────────────────────────────────────────────────
#!/usr/bin/env python3
"""hotfix_encoder.py — Timeout fix + H.265 hevc_amf support."""
import os

def patch_file(path, replacements):
    if not os.path.isfile(path):
        print("  [!!] Dosya bulunamadi: " + path)
        return False
    with open(path, "r", encoding="utf-8") as f:
        content = f.read()
    changed = False
    for old, new in replacements:
        if old in content:
            content = content.replace(old, new)
            changed = True
            print("  [OK] " + repr(old[:50]) + " -> patched")
        else:
            print("  [--] " + repr(old[:50]) + " not found (already patched?)")
    if changed:
        with open(path, "w", encoding="utf-8") as f:
            f.write(content)
        print("  [OK] " + path + " saved")
    return changed

def main():
    if not os.path.isdir("pipeline"):
        print("[!!] pipeline/ klasoru bulunamadi!")
        return

    print("\n=== 1/2: encoder.py — timeout fix + hevc_amf ===")
    patch_file("pipeline/encoder.py", [
        # Timeout: 60 -> 1800
        ("timeout=60", "timeout=1800"),
        # H.265 encoder: hw_h264 -> hw_hevc fallback
        ("gpu_backend.hw_h264", "getattr(gpu_backend, 'hw_hevc', gpu_backend.hw_h264)"),
    ])

    print("\n=== 2/2: core.py — hw_hevc attribute ===")
    # Read core.py and add hw_hevc if missing
    core_path = "pipeline/core.py"
    if os.path.isfile(core_path):
        with open(core_path, "r", encoding="utf-8") as f:
            content = f.read()
        if "hw_hevc" not in content:
            content = content.replace(
                'self.hw_h264',
                'self.hw_hevc="hevc_amf"; self.hw_h264'
            )
            with open(core_path, "w", encoding="utf-8") as f:
                f.write(content)
            print("  [OK] hw_hevc eklendi")
        else:
            print("  [--] hw_hevc zaten mevcut")
    else:
        print("  [!!] " + core_path + " bulunamadi")

    print()
    print("=" * 50)
    print("  Hotfix tamamlandi!")
    print("  - Encoder timeout: 60s -> 1800s")
    print("  - Codec: hevc_amf (H.265 GPU)")
    print("  python -m pipeline.main")
    print("=" * 50)

if __name__ == "__main__":
    main()


────────────────────────────────────────────────────────────
FILE: pythondumb.py
SIZE: 2.4 KB
────────────────────────────────────────────────────────────
# recovery_dump.py
# E:\DENEYSEL\pipeline_project\ içinde çalıştır

import os
from pathlib import Path

PROJECT_ROOT = Path(r"E:\DENEYSEL\pipeline_project")
OUTPUT_FILE  = PROJECT_ROOT / "FULL_DUMP.txt"

EXTENSIONS   = {".py", ".json", ".yaml", ".yml", ".toml", ".cfg", ".ini", ".txt", ".md", ".bat", ".ps1", ".sh"}
SKIP_DIRS    = {"__pycache__", ".git", "node_modules", "venv", ".venv", "env", "models", "tools"}
MAX_FILE_KB  = 200  # 200 KB üstü dosyaları atla (binary olabilir)

lines = []

# ── 1. TREE ──
lines.append("=" * 80)
lines.append("  PROJECT TREE")
lines.append("=" * 80)

for root, dirs, files in os.walk(PROJECT_ROOT):
    dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
    
    level  = len(Path(root).relative_to(PROJECT_ROOT).parts)
    indent = "│   " * level
    
    lines.append(f"{indent}📁 {Path(root).name}/")
    
    sub_indent = "│   " * (level + 1)
    for f in sorted(files):
        size_kb = (Path(root) / f).stat().st_size / 1024
        lines.append(f"{sub_indent}📄 {f}  ({size_kb:.1f} KB)")

# ── 2. FILE CONTENTS ──
lines.append("")
lines.append("=" * 80)
lines.append("  FILE CONTENTS")
lines.append("=" * 80)

for root, dirs, files in os.walk(PROJECT_ROOT):
    dirs[:] = [d for d in dirs if d not in SKIP_DIRS]
    
    for f in sorted(files):
        fp = Path(root) / f
        
        if fp.suffix.lower() not in EXTENSIONS:
            continue
        if fp.stat().st_size / 1024 > MAX_FILE_KB:
            lines.append(f"\n{'─'*60}")
            lines.append(f"FILE: {fp.relative_to(PROJECT_ROOT)}")
            lines.append(f"SKIPPED: {fp.stat().st_size/1024:.0f} KB (too large)")
            continue
        
        try:
            content = fp.read_text(encoding="utf-8", errors="replace")
        except Exception as e:
            content = f"[READ ERROR: {e}]"
        
        lines.append(f"\n{'─'*60}")
        lines.append(f"FILE: {fp.relative_to(PROJECT_ROOT)}")
        lines.append(f"SIZE: {fp.stat().st_size/1024:.1f} KB")
        lines.append(f"{'─'*60}")
        lines.append(content)

output = "\n".join(lines)

with open(OUTPUT_FILE, "w", encoding="utf-8") as fh:
    fh.write(output)

print(f"DUMP tamamlandi: {OUTPUT_FILE}")
print(f"Toplam satir: {len(lines)}")
print(f"Dosya boyutu: {OUTPUT_FILE.stat().st_size/1024:.0f} KB")

────────────────────────────────────────────────────────────
FILE: requirements.txt
SIZE: 0.2 KB
────────────────────────────────────────────────────────────
yt-dlp
scenedetect[opencv]
librosa>=0.10.2
pandas>=2.2.0
opencv-python>=4.9.0
numpy>=1.26,<2.0
requests>=2.31.0
python-dotenv>=1.0.0
basicsr>=1.4.2
realesrgan>=0.3.0
colorama
supertonic


────────────────────────────────────────────────────────────
FILE: setup.py
SIZE: 3.9 KB
────────────────────────────────────────────────────────────
#!/usr/bin/env python3
"""setup.py — Pipeline v10.2 otomatik kurulum."""
import os, sys, subprocess, shutil, urllib.request, zipfile

print()
print("  Pipeline v10.2 — Otomatik Kurulum")
print("  Windows 11 + RX 9070 XT 16GB")
print()

def run(cmd, label):
    print(f"  [..] {label}...")
    r = subprocess.run(cmd, capture_output=True, text=True)
    if r.returncode == 0:
        print(f"  [OK] {label}")
    else:
        print(f"  [!!] {label}: {r.stderr[:200]}")
    return r.returncode == 0

# 1. pip install
print("=" * 60)
print("  1/7 — requirements.txt & pip install")
print("=" * 60)
if os.path.exists("requirements.txt"):
    print("  [OK] requirements.txt mevcut")
else:
    print("  [!!] requirements.txt bulunamadi")
run([sys.executable, "-m", "pip", "install", "-r", "requirements.txt", "--quiet"], "pip install")

# 2. FFmpeg
print("=" * 60)
print("  2/7 — FFmpeg")
print("=" * 60)
if shutil.which("ffmpeg"):
    print("  [OK] FFmpeg PATH'te:", shutil.which("ffmpeg"))
else:
    print("  [!!] FFmpeg bulunamadi! Kurun: winget install Gyan.FFmpeg")

# 3. YOLO
print("=" * 60)
print("  3/7 — YOLOv8n ONNX")
print("=" * 60)
yolo = "yolov8n.onnx"
if os.path.exists(yolo):
    print(f"  [OK] {yolo} ({os.path.getsize(yolo)/1e6:.1f} MB)")
else:
    print("  [..] Indiriliyor...")
    try:
        url = "https://github.com/ultralytics/assets/releases/download/v8.3.0/yolov8n.onnx"
        urllib.request.urlretrieve(url, yolo)
        print(f"  [OK] {yolo} indirildi")
    except Exception as e:
        print(f"  [!!] YOLO indirilemedi: {e}")

# 4. ESRGAN
print("=" * 60)
print("  4/7 — RealESRGAN")
print("=" * 60)
esrgan = "RealESRGAN_x4plus.pth"
if os.path.exists(esrgan):
    print(f"  [OK] {esrgan} ({os.path.getsize(esrgan)/1e6:.1f} MB)")
else:
    print("  [..] Indiriliyor...")
    try:
        url = "https://github.com/xinntao/Real-ESRGAN/releases/download/v0.1.0/RealESRGAN_x4plus.pth"
        urllib.request.urlretrieve(url, esrgan)
        print(f"  [OK] {esrgan} indirildi")
    except Exception as e:
        print(f"  [!!] ESRGAN indirilemedi: {e}")

# 5. Ollama + Gemma
print("=" * 60)
print("  5/7 — Ollama & Gemma 4 E2B")
print("=" * 60)
if shutil.which("ollama"):
    print("  [OK] Ollama kurulu")
    run(["ollama", "pull", "gemma4:e2b"], "ollama pull gemma4:e2b")
else:
    print("  [!!] Ollama kurulu degil!")
    print("  [..] Kurmaya calisiliyor...")
    if run(["winget", "install", "Ollama.Ollama", "--silent"], "winget install Ollama"):
        print("  [OK] Ollama kuruldu, restart sonrasi: ollama pull gemma4:e2b")
    else:
        print("  [!!] Ollama kurulamadi. Manuel kurun: winget install Ollama.Ollama")

# 6. RIFE
print("=" * 60)
print("  6/7 — RIFE ncnn-vulkan")
print("=" * 60)
rife_dir = "tools"
os.makedirs(rife_dir, exist_ok=True)
rife_bin = os.path.join(rife_dir, "rife-ncnn-vulkan.exe")
if os.path.exists(rife_bin) or shutil.which("rife-ncnn-vulkan"):
    print(f"  [OK] RIFE mevcut")
else:
    print("  [!!] RIFE binary bulunamadi!")
    print("  [!!] Manuel indirin:")
    print("       https://github.com/nihui/rife-ncnn-vulkan/releases")
    print(f"       Cikarip {rife_bin} olarak koyun")

# 7. Health
print("=" * 60)
print("  7/7 — Sistem Kontrolu")
print("=" * 60)
print(f"  [OK] Python: {sys.version.split()[0]}")
if shutil.which("ffmpeg"):
    print("  [OK] FFmpeg: mevcut")
if os.path.exists(yolo):
    print(f"  [OK] YOLO: {yolo}")
if os.path.exists(esrgan):
    print(f"  [OK] ESRGAN: {esrgan}")
try:
    import onnxruntime as ort
    provs = ort.get_available_providers()
    dml = "DmlExecutionProvider" in provs
    tag = "[OK]" if dml else "[!!]"
    print(f"  {tag} DirectML: {provs}")
except:
    print("  [!!] onnxruntime bulunamadi")
try:
    import supertonic
    print("  [OK] Supertonic TTS: kurulu")
except ImportError:
    print("  [!!] Supertonic TTS: pip install supertonic")

print()
print("=" * 60)
print("  KURULUM TAMAMLANDI!")
print("  Baslatmak icin: python -m pipeline.main")
print("=" * 60)


────────────────────────────────────────────────────────────
FILE: upgrade_v103b.py
SIZE: 29.4 KB
────────────────────────────────────────────────────────────
#!/usr/bin/env python3
"""upgrade_v103b.py — Add resemblyzer audio-visual speaker verification."""
import os, base64

FILES = {
    "pipeline/voice_extractor.py": "IiIidm9pY2VfZXh0cmFjdG9yLnB5IOKAlCBWb2ljZSBleHRyYWN0aW9uICsgcmVzZW1ibHl6ZXIgZW1iZWRkaW5nIG1hdGNoaW5nLiIiIgppbXBvcnQgb3MsIHN1YnByb2Nlc3MsIHNodXRpbCwgZ2xvYgppbXBvcnQgbnVtcHkgYXMgbnAKZnJvbSBwaXBlbGluZS5jb3JlIGltcG9ydCBjb25maWcsIGxvZ2dlciwgcnVuX3N1YgoKCmNsYXNzIFZvaWNlRXh0cmFjdG9yOgogICAgZGVmIF9faW5pdF9fKHNlbGYpOgogICAgICAgIHNlbGYudm9pY2VzX2RpciA9IHN0cihnZXRhdHRyKGNvbmZpZywgInZvaWNlc19kaXIiLCAiLi92b2ljZXMiKSkKICAgICAgICBvcy5tYWtlZGlycyhzZWxmLnZvaWNlc19kaXIsIGV4aXN0X29rPVRydWUpCiAgICAgICAgc2VsZi5fZW5jb2RlciA9IE5vbmUKICAgICAgICBzZWxmLl9pbml0X2VuY29kZXIoKQoKICAgIGRlZiBfaW5pdF9lbmNvZGVyKHNlbGYpOgogICAgICAgIHRyeToKICAgICAgICAgICAgZnJvbSByZXNlbWJseXplciBpbXBvcnQgVm9pY2VFbmNvZGVyCiAgICAgICAgICAgIHNlbGYuX2VuY29kZXIgPSBWb2ljZUVuY29kZXIoImNwdSIpCiAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJWb2ljZUV4dHJhY3RvcjogcmVzZW1ibHl6ZXIgT0siKQogICAgICAgIGV4Y2VwdCBJbXBvcnRFcnJvcjoKICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoIlZvaWNlRXh0cmFjdG9yOiByZXNlbWJseXplciBub3QgaW5zdGFsbGVkIChwaXAgaW5zdGFsbCByZXNlbWJseXplcikiKQogICAgICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZToKICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoIlZvaWNlRXh0cmFjdG9yOiBlbmNvZGVyIGluaXQgZmFpbGVkOiAlcyIsIGUpCgogICAgQHByb3BlcnR5CiAgICBkZWYgYXZhaWxhYmxlKHNlbGYpOgogICAgICAgIHJldHVybiBUcnVlCgogICAgQHByb3BlcnR5CiAgICBkZWYgY2FuX21hdGNoKHNlbGYpOgogICAgICAgIHJldHVybiBzZWxmLl9lbmNvZGVyIGlzIG5vdCBOb25lCgogICAgZGVmIF9zYWZlX25hbWUoc2VsZiwgbmFtZSk6CiAgICAgICAgcmV0dXJuICIiLmpvaW4oYyBpZiBjLmlzYWxudW0oKSBvciBjIGluICJfLSIgZWxzZSAiXyIgZm9yIGMgaW4gbmFtZS5sb3dlcigpLnN0cmlwKCkpCgogICAgZGVmIF93YXZfcGF0aChzZWxmLCBuYW1lKToKICAgICAgICByZXR1cm4gb3MucGF0aC5qb2luKHNlbGYudm9pY2VzX2Rpciwgc2VsZi5fc2FmZV9uYW1lKG5hbWUpICsgIi53YXYiKQoKICAgIGRlZiBfZW1iX3BhdGgoc2VsZiwgbmFtZSk6CiAgICAgICAgcmV0dXJuIG9zLnBhdGguam9pbihzZWxmLnZvaWNlc19kaXIsIHNlbGYuX3NhZmVfbmFtZShuYW1lKSArICIubnB5IikKCiAgICBkZWYgaGFzX3ZvaWNlKHNlbGYsIGNoYXJhY3Rlcl9uYW1lKToKICAgICAgICByZXR1cm4gb3MucGF0aC5pc2ZpbGUoc2VsZi5fd2F2X3BhdGgoY2hhcmFjdGVyX25hbWUpKQoKICAgIGRlZiBnZXRfdm9pY2VfcGF0aChzZWxmLCBjaGFyYWN0ZXJfbmFtZSk6CiAgICAgICAgcCA9IHNlbGYuX3dhdl9wYXRoKGNoYXJhY3Rlcl9uYW1lKQogICAgICAgIHJldHVybiBwIGlmIG9zLnBhdGguaXNmaWxlKHApIGVsc2UgTm9uZQoKICAgICMg4pSA4pSAIEVtYmVkZGluZyBvcHMg4pSA4pSACgogICAgZGVmIGdldF9lbWJlZGRpbmcoc2VsZiwgd2F2X3BhdGgpOgogICAgICAgIGlmIG5vdCBzZWxmLl9lbmNvZGVyOgogICAgICAgICAgICByZXR1cm4gTm9uZQogICAgICAgIHRyeToKICAgICAgICAgICAgZnJvbSByZXNlbWJseXplciBpbXBvcnQgcHJlcHJvY2Vzc193YXYKICAgICAgICAgICAgd2F2ID0gcHJlcHJvY2Vzc193YXYod2F2X3BhdGgpCiAgICAgICAgICAgIGlmIGxlbih3YXYpIDwgMTYwMDoKICAgICAgICAgICAgICAgIGxvZ2dlci53YXJuaW5nKCJBdWRpbyB0b28gc2hvcnQgZm9yIGVtYmVkZGluZzogJXMiLCB3YXZfcGF0aCkKICAgICAgICAgICAgICAgIHJldHVybiBOb25lCiAgICAgICAgICAgIGVtYiA9IHNlbGYuX2VuY29kZXIuZW1iZWRfdXR0ZXJhbmNlKHdhdikKICAgICAgICAgICAgcmV0dXJuIGVtYgogICAgICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZToKICAgICAgICAgICAgbG9nZ2VyLmVycm9yKCJFbWJlZGRpbmcgZXh0cmFjdGlvbiBmYWlsZWQ6ICVzIiwgZSkKICAgICAgICAgICAgcmV0dXJuIE5vbmUKCiAgICBkZWYgc2F2ZV9lbWJlZGRpbmcoc2VsZiwgY2hhcmFjdGVyX25hbWUsIGVtYmVkZGluZyk6CiAgICAgICAgcGF0aCA9IHNlbGYuX2VtYl9wYXRoKGNoYXJhY3Rlcl9uYW1lKQogICAgICAgIG5wLnNhdmUocGF0aCwgZW1iZWRkaW5nKQogICAgICAgIGxvZ2dlci5pbmZvKCJFbWJlZGRpbmcgc2F2ZWQ6ICVzIiwgcGF0aCkKCiAgICBkZWYgbG9hZF9lbWJlZGRpbmcoc2VsZiwgY2hhcmFjdGVyX25hbWUpOgogICAgICAgIHBhdGggPSBzZWxmLl9lbWJfcGF0aChjaGFyYWN0ZXJfbmFtZSkKICAgICAgICBpZiBvcy5wYXRoLmlzZmlsZShwYXRoKToKICAgICAgICAgICAgcmV0dXJuIG5wLmxvYWQocGF0aCkKICAgICAgICByZXR1cm4gTm9uZQoKICAgIGRlZiBfbGlzdF9lbWJlZGRpbmdzKHNlbGYpOgogICAgICAgIHJlc3VsdCA9IHt9CiAgICAgICAgZm9yIGYgaW4gb3MubGlzdGRpcihzZWxmLnZvaWNlc19kaXIpOgogICAgICAgICAgICBpZiBmLmVuZHN3aXRoKCIubnB5Iik6CiAgICAgICAgICAgICAgICBuYW1lID0gZls6LTRdCiAgICAgICAgICAgICAgICBlbWIgPSBucC5sb2FkKG9zLnBhdGguam9pbihzZWxmLnZvaWNlc19kaXIsIGYpKQogICAgICAgICAgICAgICAgcmVzdWx0W25hbWVdID0gZW1iCiAgICAgICAgcmV0dXJuIHJlc3VsdAoKICAgIGRlZiBtYXRjaF92b2ljZShzZWxmLCB3YXZfcGF0aCwgdGhyZXNob2xkPTAuNzUpOgogICAgICAgIGlmIG5vdCBzZWxmLl9lbmNvZGVyOgogICAgICAgICAgICByZXR1cm4gTm9uZQogICAgICAgIGVtYiA9IHNlbGYuZ2V0X2VtYmVkZGluZyh3YXZfcGF0aCkKICAgICAgICBpZiBlbWIgaXMgTm9uZToKICAgICAgICAgICAgcmV0dXJuIE5vbmUKICAgICAgICBsaWJyYXJ5ID0gc2VsZi5fbGlzdF9lbWJlZGRpbmdzKCkKICAgICAgICBpZiBub3QgbGlicmFyeToKICAgICAgICAgICAgcmV0dXJuIE5vbmUKICAgICAgICBiZXN0X25hbWUgPSBOb25lCiAgICAgICAgYmVzdF9zaW0gPSAwLjAKICAgICAgICBmb3IgbmFtZSwgc3RvcmVkX2VtYiBpbiBsaWJyYXJ5Lml0ZW1zKCk6CiAgICAgICAgICAgIHNpbSA9IGZsb2F0KG5wLmRvdChlbWIsIHN0b3JlZF9lbWIpIC8gKG5wLmxpbmFsZy5ub3JtKGVtYikgKiBucC5saW5hbGcubm9ybShzdG9yZWRfZW1iKSArIDFlLTgpKQogICAgICAgICAgICBpZiBzaW0gPiBiZXN0X3NpbToKICAgICAgICAgICAgICAgIGJlc3Rfc2ltID0gc2ltCiAgICAgICAgICAgICAgICBiZXN0X25hbWUgPSBuYW1lCiAgICAgICAgaWYgYmVzdF9zaW0gPj0gdGhyZXNob2xkIGFuZCBiZXN0X25hbWU6CiAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJWb2ljZSBtYXRjaDogJXMgKHNpbT0lLjNmKSIsIGJlc3RfbmFtZSwgYmVzdF9zaW0pCiAgICAgICAgICAgIHJldHVybiB7ImNoYXJhY3RlciI6IGJlc3RfbmFtZSwgInNpbWlsYXJpdHkiOiBiZXN0X3NpbX0KICAgICAgICBsb2dnZXIuaW5mbygiTm8gdm9pY2UgbWF0Y2ggYWJvdmUgdGhyZXNob2xkIChiZXN0OiAlcyAlLjNmKSIsIGJlc3RfbmFtZSwgYmVzdF9zaW0pCiAgICAgICAgcmV0dXJuIE5vbmUKCiAgICAjIOKUgOKUgCBBdWRpbyBleHRyYWN0aW9uIOKUgOKUgAoKICAgIGRlZiBfY3V0X2F1ZGlvKHNlbGYsIHZpZGVvX3BhdGgsIHN0YXJ0X3NlYywgZW5kX3NlYywgb3V0cHV0X3dhdik6CiAgICAgICAgZHVyYXRpb24gPSBlbmRfc2VjIC0gc3RhcnRfc2VjCiAgICAgICAgaWYgZHVyYXRpb24gPCAyLjA6CiAgICAgICAgICAgIGR1cmF0aW9uID0gNS4wCiAgICAgICAgaWYgZHVyYXRpb24gPiAzMC4wOgogICAgICAgICAgICBkdXJhdGlvbiA9IDMwLjAKICAgICAgICBjbWQgPSBbCiAgICAgICAgICAgIGNvbmZpZy5mZm1wZWcsICIteSIsCiAgICAgICAgICAgICItc3MiLCBzdHIoc3RhcnRfc2VjKSwKICAgICAgICAgICAgIi10Iiwgc3RyKGR1cmF0aW9uKSwKICAgICAgICAgICAgIi1pIiwgdmlkZW9fcGF0aCwKICAgICAgICAgICAgIi12biIsICItYWNvZGVjIiwgInBjbV9zMTZsZSIsCiAgICAgICAgICAgICItYXIiLCAiMTYwMDAiLCAiLWFjIiwgIjEiLAogICAgICAgICAgICBvdXRwdXRfd2F2CiAgICAgICAgXQogICAgICAgIHRyeToKICAgICAgICAgICAgcnVuX3N1YihjbWQsICJ2b2ljZS1jdXQiLCBpZ25vcmU9VHJ1ZSkKICAgICAgICAgICAgcmV0dXJuIG9zLnBhdGguaXNmaWxlKG91dHB1dF93YXYpCiAgICAgICAgZXhjZXB0IEV4Y2VwdGlvbiBhcyBlOgogICAgICAgICAgICBsb2dnZXIuZXJyb3IoIkZGbXBlZyB2b2ljZSBjdXQgZmFpbGVkOiAlcyIsIGUpCiAgICAgICAgICAgIHJldHVybiBGYWxzZQoKICAgIGRlZiBfaXNvbGF0ZV92b2NhbHMoc2VsZiwgcmF3X3dhdiwgdG1wX2Rpcik6CiAgICAgICAgdHJ5OgogICAgICAgICAgICBjbWQgPSBbCiAgICAgICAgICAgICAgICAiZGVtdWNzIiwgIi0tdHdvLXN0ZW1zIiwgInZvY2FscyIsCiAgICAgICAgICAgICAgICAiLW4iLCAiaHRkZW11Y3MiLAogICAgICAgICAgICAgICAgIi0tb3V0IiwgdG1wX2RpciwKICAgICAgICAgICAgICAgIHJhd193YXYKICAgICAgICAgICAgXQogICAgICAgICAgICBydW5fc3ViKGNtZCwgImRlbXVjcyIsIGlnbm9yZT1UcnVlKQogICAgICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZToKICAgICAgICAgICAgbG9nZ2VyLndhcm5pbmcoIkRlbXVjcyBmYWlsZWQ6ICVzIiwgZSkKICAgICAgICAgICAgcmV0dXJuIE5vbmUKICAgICAgICBwYXR0ZXJucyA9IFsKICAgICAgICAgICAgb3MucGF0aC5qb2luKHRtcF9kaXIsICJodGRlbXVjcyIsICIqKiIsICJ2b2NhbHMud2F2IiksCiAgICAgICAgICAgIG9zLnBhdGguam9pbih0bXBfZGlyLCAiKioiLCAidm9jYWxzLndhdiIpLAogICAgICAgIF0KICAgICAgICBmb3IgcGF0IGluIHBhdHRlcm5zOgogICAgICAgICAgICBtYXRjaGVzID0gZ2xvYi5nbG9iKHBhdCwgcmVjdXJzaXZlPVRydWUpCiAgICAgICAgICAgIGlmIG1hdGNoZXM6CiAgICAgICAgICAgICAgICByZXR1cm4gbWF0Y2hlc1swXQogICAgICAgIHJldHVybiBOb25lCgogICAgZGVmIGV4dHJhY3Rfdm9pY2Uoc2VsZiwgdmlkZW9fcGF0aCwgc3RhcnRfc2VjLCBlbmRfc2VjLCBjaGFyYWN0ZXJfbmFtZSk6CiAgICAgICAgc2FmZSA9IHNlbGYuX3NhZmVfbmFtZShjaGFyYWN0ZXJfbmFtZSkKICAgICAgICBvdXRfd2F2ID0gc2VsZi5fd2F2X3BhdGgoY2hhcmFjdGVyX25hbWUpCiAgICAgICAgaWYgb3MucGF0aC5pc2ZpbGUob3V0X3dhdikgYW5kIG9zLnBhdGguZ2V0c2l6ZShvdXRfd2F2KSA+IDEwMDAwOgogICAgICAgICAgICBsb2dnZXIuaW5mbygiVm9pY2UgYWxyZWFkeSBleGlzdHM6ICVzIiwgb3V0X3dhdikKICAgICAgICAgICAgcmV0dXJuIG91dF93YXYKICAgICAgICB0bXBfZGlyID0gb3MucGF0aC5qb2luKHN0cihjb25maWcuZGlyX3RlbXApLCAidm9pY2VfZXh0cmFjdCIpCiAgICAgICAgb3MubWFrZWRpcnModG1wX2RpciwgZXhpc3Rfb2s9VHJ1ZSkKICAgICAgICByYXdfd2F2ID0gb3MucGF0aC5qb2luKHRtcF9kaXIsIHNhZmUgKyAiX3Jhdy53YXYiKQogICAgICAgIGlmIG5vdCBzZWxmLl9jdXRfYXVkaW8odmlkZW9fcGF0aCwgc3RhcnRfc2VjLCBlbmRfc2VjLCByYXdfd2F2KToKICAgICAgICAgICAgcmV0dXJuIE5vbmUKICAgICAgICB2b2NhbCA9IHNlbGYuX2lzb2xhdGVfdm9jYWxzKHJhd193YXYsIHRtcF9kaXIpCiAgICAgICAgaWYgdm9jYWwgYW5kIG9zLnBhdGguaXNmaWxlKHZvY2FsKToKICAgICAgICAgICAgc2h1dGlsLmNvcHkyKHZvY2FsLCBvdXRfd2F2KQogICAgICAgICAgICBsb2dnZXIuaW5mbygiVm9pY2UgZXh0cmFjdGVkIChkZW11Y3MpOiAlcyIsIG91dF93YXYpCiAgICAgICAgZWxzZToKICAgICAgICAgICAgc2h1dGlsLmNvcHkyKHJhd193YXYsIG91dF93YXYpCiAgICAgICAgICAgIGxvZ2dlci53YXJuaW5nKCJEZW11Y3MgdW5hdmFpbGFibGUsIHVzaW5nIHJhdyBhdWRpbzogJXMiLCBvdXRfd2F2KQogICAgICAgIGVtYiA9IHNlbGYuZ2V0X2VtYmVkZGluZyhvdXRfd2F2KQogICAgICAgIGlmIGVtYiBpcyBub3QgTm9uZToKICAgICAgICAgICAgc2VsZi5zYXZlX2VtYmVkZGluZyhjaGFyYWN0ZXJfbmFtZSwgZW1iKQogICAgICAgICAgICBsb2dnZXIuaW5mbygiVm9pY2UgKyBlbWJlZGRpbmcgc2F2ZWQ6ICVzIiwgY2hhcmFjdGVyX25hbWUpCiAgICAgICAgcmV0dXJuIG91dF93YXYKCiAgICAjIOKUgOKUgCBBdWRpby1WaXN1YWwgVmVyaWZpY2F0aW9uIOKUgOKUgAoKICAgIGRlZiB2ZXJpZnlfc3BlYWtlcihzZWxmLCB2aWRlb19wYXRoLCBzdGFydF9zZWMsIGVuZF9zZWMsIHZpc3VhbF9ndWVzcz0idW5rbm93biIpOgogICAgICAgIHRtcF9kaXIgPSBvcy5wYXRoLmpvaW4oc3RyKGNvbmZpZy5kaXJfdGVtcCksICJ2b2ljZV92ZXJpZnkiKQogICAgICAgIG9zLm1ha2VkaXJzKHRtcF9kaXIsIGV4aXN0X29rPVRydWUpCiAgICAgICAgdG1wX3dhdiA9IG9zLnBhdGguam9pbih0bXBfZGlyLCAidmVyaWZ5X3NlZ21lbnQud2F2IikKICAgICAgICBpZiBub3Qgc2VsZi5fY3V0X2F1ZGlvKHZpZGVvX3BhdGgsIHN0YXJ0X3NlYywgZW5kX3NlYywgdG1wX3dhdik6CiAgICAgICAgICAgIHJldHVybiB7ImNoYXJhY3RlciI6IHZpc3VhbF9ndWVzcywgIm1ldGhvZCI6ICJ2aXN1YWxfb25seSIsICJjb25maWRlbmNlIjogMC41fQogICAgICAgIHZvY2FsID0gc2VsZi5faXNvbGF0ZV92b2NhbHModG1wX3dhdiwgdG1wX2RpcikKICAgICAgICBjaGVja193YXYgPSB2b2NhbCBpZiB2b2NhbCBhbmQgb3MucGF0aC5pc2ZpbGUodm9jYWwpIGVsc2UgdG1wX3dhdgogICAgICAgIGF1ZGlvX21hdGNoID0gc2VsZi5tYXRjaF92b2ljZShjaGVja193YXYpCiAgICAgICAgaWYgYXVkaW9fbWF0Y2g6CiAgICAgICAgICAgIGF1ZGlvX2NoYXIgPSBhdWRpb19tYXRjaFsiY2hhcmFjdGVyIl0KICAgICAgICAgICAgYXVkaW9fc2ltID0gYXVkaW9fbWF0Y2hbInNpbWlsYXJpdHkiXQogICAgICAgICAgICBpZiB2aXN1YWxfZ3Vlc3MgIT0gInVua25vd24iIGFuZCBzZWxmLl9zYWZlX25hbWUodmlzdWFsX2d1ZXNzKSA9PSBhdWRpb19jaGFyOgogICAgICAgICAgICAgICAgY29uZmlkZW5jZSA9IG1pbigxLjAsIChhdWRpb19zaW0gKyAwLjg1KSAvIDIpCiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiQ09ORklSTUVEIChhdWRpbyt2aXN1YWwpOiAlcyAoY29uZj0lLjJmKSIsIHZpc3VhbF9ndWVzcywgY29uZmlkZW5jZSkKICAgICAgICAgICAgICAgIHJldHVybiB7ImNoYXJhY3RlciI6IHZpc3VhbF9ndWVzcywgIm1ldGhvZCI6ICJhdWRpbyt2aXN1YWwiLCAiY29uZmlkZW5jZSI6IGNvbmZpZGVuY2V9CiAgICAgICAgICAgIGVsaWYgYXVkaW9fc2ltID4gMC44NToKICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJBVURJTyBPVkVSUklERTogdmlzdWFsPSVzLCBhdWRpbz0lcyAoc2ltPSUuM2YpIiwgdmlzdWFsX2d1ZXNzLCBhdWRpb19jaGFyLCBhdWRpb19zaW0pCiAgICAgICAgICAgICAgICByZXR1cm4geyJjaGFyYWN0ZXIiOiBhdWRpb19jaGFyLCAibWV0aG9kIjogImF1ZGlvX292ZXJyaWRlIiwgImNvbmZpZGVuY2UiOiBhdWRpb19zaW19CiAgICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiQ09ORkxJQ1Q6IHZpc3VhbD0lcywgYXVkaW89JXMgKHNpbT0lLjNmKSDigJQgdXNpbmcgdmlzdWFsIiwgdmlzdWFsX2d1ZXNzLCBhdWRpb19jaGFyLCBhdWRpb19zaW0pCiAgICAgICAgICAgICAgICByZXR1cm4geyJjaGFyYWN0ZXIiOiB2aXN1YWxfZ3Vlc3MsICJtZXRob2QiOiAidmlzdWFsX3ByZWZlcnJlZCIsICJjb25maWRlbmNlIjogMC42fQogICAgICAgIGVsc2U6CiAgICAgICAgICAgIGlmIHZpc3VhbF9ndWVzcyAhPSAidW5rbm93biI6CiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiTkVXIFZPSUNFOiAlcyAobm8gYXVkaW8gbWF0Y2gsIHNhdmluZyBlbWJlZGRpbmcpIiwgdmlzdWFsX2d1ZXNzKQogICAgICAgICAgICAgICAgZW1iID0gc2VsZi5nZXRfZW1iZWRkaW5nKGNoZWNrX3dhdikKICAgICAgICAgICAgICAgIGlmIGVtYiBpcyBub3QgTm9uZToKICAgICAgICAgICAgICAgICAgICBzZWxmLnNhdmVfZW1iZWRkaW5nKHZpc3VhbF9ndWVzcywgZW1iKQogICAgICAgICAgICAgICAgcmV0dXJuIHsiY2hhcmFjdGVyIjogdmlzdWFsX2d1ZXNzLCAibWV0aG9kIjogInZpc3VhbF9uZXdfdm9pY2UiLCAiY29uZmlkZW5jZSI6IDAuN30KICAgICAgICAgICAgcmV0dXJuIHsiY2hhcmFjdGVyIjogInVua25vd24iLCAibWV0aG9kIjogIm5vbmUiLCAiY29uZmlkZW5jZSI6IDAuMH0KCiAgICBkZWYgbGlzdF92b2ljZXMoc2VsZik6CiAgICAgICAgaWYgbm90IG9zLnBhdGguaXNkaXIoc2VsZi52b2ljZXNfZGlyKToKICAgICAgICAgICAgcmV0dXJuIFtdCiAgICAgICAgdm9pY2VzID0gW10KICAgICAgICBmb3IgZiBpbiBvcy5saXN0ZGlyKHNlbGYudm9pY2VzX2Rpcik6CiAgICAgICAgICAgIGlmIGYuZW5kc3dpdGgoIi53YXYiKToKICAgICAgICAgICAgICAgIG5hbWUgPSBmWzotNF0KICAgICAgICAgICAgICAgIHdhdl9zaXplID0gb3MucGF0aC5nZXRzaXplKG9zLnBhdGguam9pbihzZWxmLnZvaWNlc19kaXIsIGYpKSAvIDEwMjQKICAgICAgICAgICAgICAgIGhhc19lbWIgPSBvcy5wYXRoLmlzZmlsZShvcy5wYXRoLmpvaW4oc2VsZi52b2ljZXNfZGlyLCBuYW1lICsgIi5ucHkiKSkKICAgICAgICAgICAgICAgIHZvaWNlcy5hcHBlbmQoeyJuYW1lIjogbmFtZSwgInNpemVfa2IiOiB3YXZfc2l6ZSwgImhhc19lbWJlZGRpbmciOiBoYXNfZW1ifSkKICAgICAgICByZXR1cm4gdm9pY2VzCg==",
    "pipeline/brain.py": "IiIiYnJhaW4ucHkg4oCUIFBpcGVsaW5lQnJhaW4gd2l0aCBhdWRpby12aXN1YWwgY2hhcmFjdGVyIHZlcmlmaWNhdGlvbi4iIiIKaW1wb3J0IG9zLCBqc29uLCBiYXNlNjQKZnJvbSBwaXBlbGluZS5jb3JlIGltcG9ydCBjb25maWcsIGxvZ2dlcgoKCmNsYXNzIFBpcGVsaW5lQnJhaW46CiAgICBkZWYgX19pbml0X18oc2VsZik6CiAgICAgICAgc2VsZi5fbW9kZWwgPSBjb25maWcub2xsYW1hX21vZGVsCiAgICAgICAgc2VsZi5fdXJsID0gY29uZmlnLm9sbGFtYV91cmwKICAgICAgICBzZWxmLmF2YWlsYWJsZSA9IEZhbHNlCiAgICAgICAgdHJ5OgogICAgICAgICAgICBpbXBvcnQgcmVxdWVzdHMKICAgICAgICAgICAgciA9IHJlcXVlc3RzLmdldChzZWxmLl91cmwsIHRpbWVvdXQ9MykKICAgICAgICAgICAgaWYgci5zdGF0dXNfY29kZSA9PSAyMDA6CiAgICAgICAgICAgICAgICBzZWxmLmF2YWlsYWJsZSA9IFRydWUKICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJCcmFpbjogT2xsYW1hIE9LIChtb2RlbD0lcykiLCBzZWxmLl9tb2RlbCkKICAgICAgICBleGNlcHQgRXhjZXB0aW9uOgogICAgICAgICAgICBsb2dnZXIud2FybmluZygiQnJhaW46IE9sbGFtYSBub3QgcmVhY2hhYmxlIGF0ICVzIiwgc2VsZi5fdXJsKQoKICAgIGRlZiBfY2hhdChzZWxmLCBtZXNzYWdlcywgdGltZW91dD02MCk6CiAgICAgICAgaW1wb3J0IHJlcXVlc3RzCiAgICAgICAgdXJsID0gc2VsZi5fdXJsLnJzdHJpcCgiLyIpICsgIi9hcGkvY2hhdCIKICAgICAgICBwYXlsb2FkID0geyJtb2RlbCI6IHNlbGYuX21vZGVsLCAibWVzc2FnZXMiOiBtZXNzYWdlcywgInN0cmVhbSI6IEZhbHNlfQogICAgICAgIHRyeToKICAgICAgICAgICAgciA9IHJlcXVlc3RzLnBvc3QodXJsLCBqc29uPXBheWxvYWQsIHRpbWVvdXQ9dGltZW91dCkKICAgICAgICAgICAgZGF0YSA9IHIuanNvbigpCiAgICAgICAgICAgIHJldHVybiBkYXRhLmdldCgibWVzc2FnZSIsIHt9KS5nZXQoImNvbnRlbnQiLCAiIikuc3RyaXAoKQogICAgICAgIGV4Y2VwdCBFeGNlcHRpb24gYXMgZToKICAgICAgICAgICAgbG9nZ2VyLmVycm9yKCJCcmFpbiBjaGF0IGVycm9yOiAlcyIsIGUpCiAgICAgICAgICAgIHJldHVybiAiIgoKICAgIGRlZiBfZnJhbWVfdG9fYjY0KHNlbGYsIGZyYW1lKToKICAgICAgICBpbXBvcnQgY3YyCiAgICAgICAgb2ssIGJ1ZiA9IGN2Mi5pbWVuY29kZSgiLmpwZyIsIGZyYW1lLCBbY3YyLklNV1JJVEVfSlBFR19RVUFMSVRZLCA3MF0pCiAgICAgICAgaWYgb2s6CiAgICAgICAgICAgIHJldHVybiBiYXNlNjQuYjY0ZW5jb2RlKGJ1Zi50b2J5dGVzKCkpLmRlY29kZSgiYXNjaWkiKQogICAgICAgIHJldHVybiAiIgoKICAgIGRlZiBfZXh0cmFjdF9qc29uKHNlbGYsIHJhdyk6CiAgICAgICAgdHJ5OgogICAgICAgICAgICBzdGFydCA9IHJhdy5maW5kKCJ7IikKICAgICAgICAgICAgZW5kID0gcmF3LnJmaW5kKCJ9IikgKyAxCiAgICAgICAgICAgIGlmIHN0YXJ0ID49IDAgYW5kIGVuZCA+IHN0YXJ0OgogICAgICAgICAgICAgICAgcmV0dXJuIGpzb24ubG9hZHMocmF3W3N0YXJ0OmVuZF0pCiAgICAgICAgZXhjZXB0IChqc29uLkpTT05EZWNvZGVFcnJvciwgVmFsdWVFcnJvcik6CiAgICAgICAgICAgIHBhc3MKICAgICAgICByZXR1cm4gTm9uZQoKICAgIGRlZiBhbmFseXplX3NjZW5lKHNlbGYsIGZyYW1lcywgY29udGV4dD0iYW5pbWUiKToKICAgICAgICBpZiBub3Qgc2VsZi5hdmFpbGFibGUgb3Igbm90IGZyYW1lczoKICAgICAgICAgICAgcmV0dXJuIHsibW9vZCI6ICJ1bmtub3duIiwgImFjdGlvbiI6ICJ1bmtub3duIiwgImNoYXJhY3RlciI6ICJ1bmtub3duIn0KICAgICAgICBpbWdzID0gW10KICAgICAgICBmb3IgZiBpbiBmcmFtZXNbOjNdOgogICAgICAgICAgICBiNjQgPSBzZWxmLl9mcmFtZV90b19iNjQoZikKICAgICAgICAgICAgaWYgYjY0OgogICAgICAgICAgICAgICAgaW1ncy5hcHBlbmQoYjY0KQogICAgICAgIGNvbnRlbnRfcGFydHMgPSBbXQogICAgICAgIGZvciBpbWcgaW4gaW1nczoKICAgICAgICAgICAgY29udGVudF9wYXJ0cy5hcHBlbmQoeyJ0eXBlIjogImltYWdlX3VybCIsICJpbWFnZV91cmwiOiB7InVybCI6ICJkYXRhOmltYWdlL2pwZWc7YmFzZTY0LCIgKyBpbWd9fSkKICAgICAgICBjb250ZW50X3BhcnRzLmFwcGVuZCh7CiAgICAgICAgICAgICJ0eXBlIjogInRleHQiLAogICAgICAgICAgICAidGV4dCI6ICgKICAgICAgICAgICAgICAgICJCdSBhbmltZSBzYWhuZXNpbmkgYW5hbGl6IGV0LiBKU09OIGZvcm1hdGluZGEgY2V2YXAgdmVyOlxuIgogICAgICAgICAgICAgICAgJ3sibW9vZCI6ICIuLi4iLCAiYWN0aW9uIjogIi4uLiIsICJjaGFyYWN0ZXIiOiAia2FyYWt0ZXJfYWRpIiwgJwogICAgICAgICAgICAgICAgJyJpbnRlbnNpdHkiOiAxLTEwLCAic3BlYWtpbmciOiB0cnVlL2ZhbHNlfVxuJwogICAgICAgICAgICAgICAgIkVnZXIga2FyYWt0ZXJpIHRhbmltbGF5YW1peW9yc2FuIGNoYXJhY3RlcjogXCJ1bmtub3duXCIgeWF6LlxuIgogICAgICAgICAgICAgICAgIlNhZGVjZSBKU09OIGRvbiwgYmFza2EgYmlyIHNleSB5YXptYS4iCiAgICAgICAgICAgICkKICAgICAgICB9KQogICAgICAgIHJhdyA9IHNlbGYuX2NoYXQoW3sicm9sZSI6ICJ1c2VyIiwgImNvbnRlbnQiOiBjb250ZW50X3BhcnRzfV0sIHRpbWVvdXQ9OTApCiAgICAgICAgcmVzdWx0ID0gc2VsZi5fZXh0cmFjdF9qc29uKHJhdykKICAgICAgICBpZiByZXN1bHQ6CiAgICAgICAgICAgIHJldHVybiByZXN1bHQKICAgICAgICByZXR1cm4geyJtb29kIjogInVua25vd24iLCAiYWN0aW9uIjogInVua25vd24iLCAiY2hhcmFjdGVyIjogInVua25vd24ifQoKICAgIGRlZiBpZGVudGlmeV9zcGVha2VycyhzZWxmLCB2aWRlb19wYXRoLCBzY2VuZXMsIHZvaWNlX2V4PU5vbmUpOgogICAgICAgIGlmIG5vdCBzZWxmLmF2YWlsYWJsZToKICAgICAgICAgICAgcmV0dXJuIFtdCiAgICAgICAgaW1wb3J0IGN2MgogICAgICAgIHJlc3VsdHMgPSBbXQogICAgICAgIGZvciBpLCBzYyBpbiBlbnVtZXJhdGUoc2NlbmVzWzo4XSk6CiAgICAgICAgICAgIHNzID0gc2MuZ2V0KCJzdGFydF9zZWMiLCAwKQogICAgICAgICAgICBlcyA9IHNjLmdldCgiZW5kX3NlYyIsIHNzICsgNSkKICAgICAgICAgICAgY2FwID0gY3YyLlZpZGVvQ2FwdHVyZSh2aWRlb19wYXRoKQogICAgICAgICAgICBmcHMgPSBjYXAuZ2V0KGN2Mi5DQVBfUFJPUF9GUFMpIG9yIDI0CiAgICAgICAgICAgIG1pZCA9IGludCgoc3MgKyBlcykgLyAyICogZnBzKQogICAgICAgICAgICBjYXAuc2V0KGN2Mi5DQVBfUFJPUF9QT1NfRlJBTUVTLCBtaWQpCiAgICAgICAgICAgIG9rLCBmcmFtZSA9IGNhcC5yZWFkKCkKICAgICAgICAgICAgY2FwLnJlbGVhc2UoKQogICAgICAgICAgICBpZiBub3Qgb2sgb3IgZnJhbWUgaXMgTm9uZToKICAgICAgICAgICAgICAgIGNvbnRpbnVlCiAgICAgICAgICAgIGI2NCA9IHNlbGYuX2ZyYW1lX3RvX2I2NChmcmFtZSkKICAgICAgICAgICAgaWYgbm90IGI2NDoKICAgICAgICAgICAgICAgIGNvbnRpbnVlCiAgICAgICAgICAgIGNvbnRlbnRfcGFydHMgPSBbCiAgICAgICAgICAgICAgICB7InR5cGUiOiAiaW1hZ2VfdXJsIiwgImltYWdlX3VybCI6IHsidXJsIjogImRhdGE6aW1hZ2UvanBlZztiYXNlNjQsIiArIGI2NH19LAogICAgICAgICAgICAgICAgeyJ0eXBlIjogInRleHQiLCAidGV4dCI6ICgKICAgICAgICAgICAgICAgICAgICAiQnUgYW5pbWUgc2FobmVzaW5kZWtpIGtvbnVzYW4ga2FyYWt0ZXJpIHRhbmltbGEuXG4iCiAgICAgICAgICAgICAgICAgICAgJ0pTT046IHsiY2hhcmFjdGVyIjogImlzaW0iLCAiY29uZmlkZW5jZSI6IDAuMC0xLjAsICJzcGVha2luZyI6IHRydWUvZmFsc2V9XG4nCiAgICAgICAgICAgICAgICAgICAgIlRhbmltaXlvcnNhbjoge1wiY2hhcmFjdGVyXCI6IFwidW5rbm93blwiLCBcImNvbmZpZGVuY2VcIjogMC4wLCBcInNwZWFraW5nXCI6IGZhbHNlfVxuIgogICAgICAgICAgICAgICAgICAgICJTYWRlY2UgSlNPTiBkb24uIgogICAgICAgICAgICAgICAgKX0KICAgICAgICAgICAgXQogICAgICAgICAgICByYXcgPSBzZWxmLl9jaGF0KFt7InJvbGUiOiAidXNlciIsICJjb250ZW50IjogY29udGVudF9wYXJ0c31dLCB0aW1lb3V0PTYwKQogICAgICAgICAgICBpbmZvID0gc2VsZi5fZXh0cmFjdF9qc29uKHJhdykKICAgICAgICAgICAgaWYgbm90IGluZm86CiAgICAgICAgICAgICAgICBjb250aW51ZQogICAgICAgICAgICB2aXN1YWxfY2hhciA9IGluZm8uZ2V0KCJjaGFyYWN0ZXIiLCAidW5rbm93biIpCiAgICAgICAgICAgIGNvbmYgPSBpbmZvLmdldCgiY29uZmlkZW5jZSIsIDApCiAgICAgICAgICAgIHNwZWFraW5nID0gaW5mby5nZXQoInNwZWFraW5nIiwgRmFsc2UpCiAgICAgICAgICAgIGlmIHZpc3VhbF9jaGFyID09ICJ1bmtub3duIiBvciBjb25mIDwgMC40IG9yIG5vdCBzcGVha2luZzoKICAgICAgICAgICAgICAgIGNvbnRpbnVlCiAgICAgICAgICAgICMgQXVkaW8tdmlzdWFsIHZlcmlmaWNhdGlvbgogICAgICAgICAgICBpZiB2b2ljZV9leCBhbmQgdm9pY2VfZXguY2FuX21hdGNoOgogICAgICAgICAgICAgICAgdmVyaWZpZWQgPSB2b2ljZV9leC52ZXJpZnlfc3BlYWtlcih2aWRlb19wYXRoLCBzcywgZXMsIHZpc3VhbF9ndWVzcz12aXN1YWxfY2hhcikKICAgICAgICAgICAgICAgIGZpbmFsX2NoYXIgPSB2ZXJpZmllZC5nZXQoImNoYXJhY3RlciIsIHZpc3VhbF9jaGFyKQogICAgICAgICAgICAgICAgbWV0aG9kID0gdmVyaWZpZWQuZ2V0KCJtZXRob2QiLCAidmlzdWFsX29ubHkiKQogICAgICAgICAgICAgICAgZmluYWxfY29uZiA9IHZlcmlmaWVkLmdldCgiY29uZmlkZW5jZSIsIGNvbmYpCiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiU2NlbmUgJWQ6ICVzIC0+ICVzICglcywgY29uZj0lLjJmKSIsIGksIHZpc3VhbF9jaGFyLCBmaW5hbF9jaGFyLCBtZXRob2QsIGZpbmFsX2NvbmYpCiAgICAgICAgICAgICAgICByZXN1bHRzLmFwcGVuZCh7CiAgICAgICAgICAgICAgICAgICAgImNoYXJhY3RlciI6IGZpbmFsX2NoYXIsCiAgICAgICAgICAgICAgICAgICAgImNvbmZpZGVuY2UiOiBmaW5hbF9jb25mLAogICAgICAgICAgICAgICAgICAgICJtZXRob2QiOiBtZXRob2QsCiAgICAgICAgICAgICAgICAgICAgInN0YXJ0X3NlYyI6IHNzLAogICAgICAgICAgICAgICAgICAgICJlbmRfc2VjIjogZXMsCiAgICAgICAgICAgICAgICAgICAgInNjZW5lX2lkeCI6IGkKICAgICAgICAgICAgICAgIH0pCiAgICAgICAgICAgIGVsc2U6CiAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiU2NlbmUgJWQ6ICVzICh2aXN1YWwgb25seSwgY29uZj0lLjJmKSIsIGksIHZpc3VhbF9jaGFyLCBjb25mKQogICAgICAgICAgICAgICAgcmVzdWx0cy5hcHBlbmQoewogICAgICAgICAgICAgICAgICAgICJjaGFyYWN0ZXIiOiB2aXN1YWxfY2hhciwKICAgICAgICAgICAgICAgICAgICAiY29uZmlkZW5jZSI6IGNvbmYsCiAgICAgICAgICAgICAgICAgICAgIm1ldGhvZCI6ICJ2aXN1YWxfb25seSIsCiAgICAgICAgICAgICAgICAgICAgInN0YXJ0X3NlYyI6IHNzLAogICAgICAgICAgICAgICAgICAgICJlbmRfc2VjIjogZXMsCiAgICAgICAgICAgICAgICAgICAgInNjZW5lX2lkeCI6IGkKICAgICAgICAgICAgICAgIH0pCiAgICAgICAgcmV0dXJuIHJlc3VsdHMKCiAgICBkZWYgd3JpdGVfbmFycmF0aW9uKHNlbGYsIHN0eWxlPSJlcGljIiwgY2hhcmFjdGVyPU5vbmUpOgogICAgICAgIGlmIG5vdCBzZWxmLmF2YWlsYWJsZToKICAgICAgICAgICAgcmV0dXJuICIiCiAgICAgICAgaWYgY2hhcmFjdGVyIGFuZCBjaGFyYWN0ZXIgIT0gInVua25vd24iOgogICAgICAgICAgICBwcm9tcHQgPSAoCiAgICAgICAgICAgICAgICAiU2VuIHt9IGthcmFrdGVyaXNpbi4gQW5pbWUgbWVtZSBpY2VyaWdpIGljaW4gYnUga2FyYWt0ZXJlIHV5Z3VuLCAiCiAgICAgICAgICAgICAgICAiZHJhbWF0aWsgdmUga29taWsgYmlyIFR1cmtjZSBuYXJyYXRpb24geWF6LiAyLTMgY3VtbGUuICIKICAgICAgICAgICAgICAgICJLYXJha3RlcmluIGtvbnVzbWEgdGFyemkgdmUga2lzaWxpZ2luaSB5YW5zaXQuIgogICAgICAgICAgICApLmZvcm1hdChjaGFyYWN0ZXIpCiAgICAgICAgZWxzZToKICAgICAgICAgICAgcHJvbXB0ID0gKAogICAgICAgICAgICAgICAgIkFuaW1lIG1lbWUgaWNlcmlnaSBpY2luIHt9IHRhcnppbmRhIGRyYW1hdGlrIHZlIGtvbWlrICIKICAgICAgICAgICAgICAgICJiaXIgVHVya2NlIG5hcnJhdGlvbiB5YXouIDItMyBjdW1sZS4gS2lzYSB2ZSBldGtpbGV5aWNpIG9sc3VuLiIKICAgICAgICAgICAgKS5mb3JtYXQoc3R5bGUpCiAgICAgICAgcmV0dXJuIHNlbGYuX2NoYXQoW3sicm9sZSI6ICJ1c2VyIiwgImNvbnRlbnQiOiBwcm9tcHR9XSkK",
    "pipeline/engine.py": "IiIiZW5naW5lLnB5IOKAlCBWaWRlbyBlbmdpbmUgd2l0aCBhdWRpby12aXN1YWwgdm9pY2UgZXh0cmFjdGlvbi4iIiIKaW1wb3J0IG9zLCB0aW1lLCBzaHV0aWwKZnJvbSBwaXBlbGluZS5jb3JlIGltcG9ydCBjb25maWcsIGxvZ2dlciwgbWV0cmljcywgcnVuX3N1YgoKCmNsYXNzIFZpZGVvRW5naW5lOgogICAgZGVmIF9faW5pdF9fKHNlbGYsIHVwc2NhbGVyPU5vbmUsIGRldGVjdG9yPU5vbmUsIG1lbWVfZng9Tm9uZSwgaW50ZXJwb2xhdG9yPU5vbmUpOgogICAgICAgIHNlbGYuX3VwcyA9IHVwc2NhbGVyCiAgICAgICAgc2VsZi5fZGV0ID0gZGV0ZWN0b3IKICAgICAgICBzZWxmLl9tZnggPSBtZW1lX2Z4CiAgICAgICAgc2VsZi5fcmlmZSA9IGludGVycG9sYXRvcgoKICAgIGRlZiBwcm9jZXNzX3NlZ21lbnQoc2VsZiwgdmlkZW9fcGF0aCwgb3V0cHV0X3BhdGgsIHN0YXJ0PU5vbmUsIGR1cj1Ob25lKToKICAgICAgICBmcm9tIHBpcGVsaW5lLmRlY29kZXIgaW1wb3J0IEZGbXBlZ0RlYwogICAgICAgIGZyb20gcGlwZWxpbmUuZW5jb2RlciBpbXBvcnQgRkZtcGVnRW5jCiAgICAgICAgZnJvbSBwaXBlbGluZS5wYXJhbGxlbCBpbXBvcnQgUGFyYVBpcGUKICAgICAgICBkZWMgPSBGRm1wZWdEZWModmlkZW9fcGF0aCwgc3M9c3RhcnQsIHRvPWR1cikKICAgICAgICB3LCBoLCBmcHMgPSBkZWMuX3csIGRlYy5faCwgZGVjLl9mcHMKICAgICAgICBpZiBzZWxmLl91cHMgYW5kIHNlbGYuX3Vwcy5hdmFpbGFibGU6CiAgICAgICAgICAgIHcgPSB3ICogY29uZmlnLnVwc2NhbGVfZmFjdG9yCiAgICAgICAgICAgIGggPSBoICogY29uZmlnLnVwc2NhbGVfZmFjdG9yCiAgICAgICAgZW5jID0gRkZtcGVnRW5jKG91dHB1dF9wYXRoLCB3LCBoLCBmcHMpCiAgICAgICAgZGVmIHByb2Nlc3NvcihmcmFtZSk6CiAgICAgICAgICAgIGlmIHNlbGYuX3VwcyBhbmQgc2VsZi5fdXBzLmF2YWlsYWJsZToKICAgICAgICAgICAgICAgIGZyYW1lID0gc2VsZi5fdXBzLmVuaGFuY2UoZnJhbWUpCiAgICAgICAgICAgIHJldHVybiBmcmFtZQogICAgICAgIHBpcGUgPSBQYXJhUGlwZShkZWMsIGVuYywgcHJvY2Vzc29yKQogICAgICAgIHQwID0gdGltZS50aW1lKCkKICAgICAgICBwaXBlLnJ1bigpCiAgICAgICAgZW5jLmNsb3NlKCkKICAgICAgICBkZWMuY2xvc2UoKQogICAgICAgIGVsYXBzZWQgPSB0aW1lLnRpbWUoKSAtIHQwCiAgICAgICAgbG9nZ2VyLmluZm8oIlNlZ21lbnQgZG9uZTogJXMgKCUuMWZzKSIsIG91dHB1dF9wYXRoLCBlbGFwc2VkKQogICAgICAgIHJldHVybiBvdXRwdXRfcGF0aAoKICAgIGRlZiBjb21wb3NlX2ZpbmFsKHNlbGYsIHNlZ21lbnRzLCBhdWRpb19wYXRoLCBvdXRwdXRfcGF0aCwgdHRzX3BhdGg9Tm9uZSk6CiAgICAgICAgaWYgbm90IHNlZ21lbnRzOgogICAgICAgICAgICByZXR1cm4gTm9uZQogICAgICAgIHRtcCA9IG9zLnBhdGguam9pbihzdHIoY29uZmlnLmRpcl90ZW1wKSwgImNvbmNhdC50eHQiKQogICAgICAgIG9zLm1ha2VkaXJzKHN0cihjb25maWcuZGlyX3RlbXApLCBleGlzdF9vaz1UcnVlKQogICAgICAgIHdpdGggb3Blbih0bXAsICJ3IikgYXMgZjoKICAgICAgICAgICAgZm9yIHMgaW4gc2VnbWVudHM6CiAgICAgICAgICAgICAgICBmLndyaXRlKCJmaWxlICIgKyByZXByKG9zLnBhdGguYWJzcGF0aChzKSkgKyAiXG4iKQogICAgICAgIGNtZCA9IFtjb25maWcuZmZtcGVnLCAiLXkiLCAiLWYiLCAiY29uY2F0IiwgIi1zYWZlIiwgIjAiLCAiLWkiLCB0bXBdCiAgICAgICAgYXVkaW9faW5wdXRzID0gW10KICAgICAgICBpZiBhdWRpb19wYXRoIGFuZCBvcy5wYXRoLmlzZmlsZShhdWRpb19wYXRoKToKICAgICAgICAgICAgY21kICs9IFsiLWkiLCBhdWRpb19wYXRoXQogICAgICAgICAgICBhdWRpb19pbnB1dHMuYXBwZW5kKCJtdXNpYyIpCiAgICAgICAgaWYgdHRzX3BhdGggYW5kIG9zLnBhdGguaXNmaWxlKHR0c19wYXRoKToKICAgICAgICAgICAgY21kICs9IFsiLWkiLCB0dHNfcGF0aF0KICAgICAgICAgICAgYXVkaW9faW5wdXRzLmFwcGVuZCgidHRzIikKICAgICAgICBpZiBsZW4oYXVkaW9faW5wdXRzKSA9PSAyOgogICAgICAgICAgICBjbWQgKz0gWyItZmlsdGVyX2NvbXBsZXgiLAogICAgICAgICAgICAgICAgICAgICJbMTphXXZvbHVtZT0wLjdbbV07WzI6YV12b2x1bWU9MS4yW3RdO1ttXVt0XWFtaXg9aW5wdXRzPTJbYV0iLAogICAgICAgICAgICAgICAgICAgICItbWFwIiwgIjA6diIsICItbWFwIiwgIlthXSJdCiAgICAgICAgZWxpZiBsZW4oYXVkaW9faW5wdXRzKSA9PSAxOgogICAgICAgICAgICBjbWQgKz0gWyItbWFwIiwgIjA6diIsICItbWFwIiwgIjE6YSJdCiAgICAgICAgZWxzZToKICAgICAgICAgICAgY21kICs9IFsiLW1hcCIsICIwOnYiXQogICAgICAgIGNtZCArPSBbIi1jOnYiLCAiY29weSIsICItYzphIiwgImFhYyIsICItc2hvcnRlc3QiLCBvdXRwdXRfcGF0aF0KICAgICAgICBydW5fc3ViKGNtZCwgImNvbXBvc2UiLCBpZ25vcmU9VHJ1ZSkKICAgICAgICBsb2dnZXIuaW5mbygiRmluYWwgY29tcG9zZWQ6ICVzIiwgb3V0cHV0X3BhdGgpCiAgICAgICAgcmV0dXJuIG91dHB1dF9wYXRoCgogICAgZGVmIGZ1bGxfcHJvY2VzcyhzZWxmLCB2aWRlb19wYXRoLCBhdWRpb19wYXRoLCBvdXRwdXRfcGF0aCwKICAgICAgICAgICAgICAgICAgICAgYnJhaW49Tm9uZSwgdHRzPU5vbmUsIHNjZW5lX2FuPU5vbmUsIHZvaWNlX2V4PU5vbmUpOgogICAgICAgIG9zLm1ha2VkaXJzKHN0cihjb25maWcuZGlyX3RlbXApLCBleGlzdF9vaz1UcnVlKQogICAgICAgIG9zLm1ha2VkaXJzKHN0cihjb25maWcuZGlyX291dHB1dCksIGV4aXN0X29rPVRydWUpCiAgICAgICAgIyBTY2VuZSBkZXRlY3Rpb24KICAgICAgICBzY2VuZXMgPSBOb25lCiAgICAgICAgaWYgc2NlbmVfYW46CiAgICAgICAgICAgIHNjZW5lcywgXyA9IHNjZW5lX2FuLmZpbmRfc2NlbmVzKHZpZGVvX3BhdGgpCiAgICAgICAgaWYgbm90IHNjZW5lczoKICAgICAgICAgICAgc2NlbmVzID0gW3sic3RhcnRfc2VjIjogMCwgImVuZF9zZWMiOiBOb25lLCAiZHVyYXRpb24iOiAwfV0KICAgICAgICAjIENoYXJhY3RlciBpZGVudGlmaWNhdGlvbiB3aXRoIGF1ZGlvLXZpc3VhbCB2ZXJpZmljYXRpb24KICAgICAgICBjaGFyYWN0ZXIgPSBOb25lCiAgICAgICAgc3BlYWtlcnMgPSBbXQogICAgICAgIGlmIGJyYWluIGFuZCBicmFpbi5hdmFpbGFibGU6CiAgICAgICAgICAgIHNwZWFrZXJzID0gYnJhaW4uaWRlbnRpZnlfc3BlYWtlcnModmlkZW9fcGF0aCwgc2NlbmVzLCB2b2ljZV9leD12b2ljZV9leCkKICAgICAgICAgICAgaWYgc3BlYWtlcnM6CiAgICAgICAgICAgICAgICBwcmltYXJ5ID0gc3BlYWtlcnNbMF0KICAgICAgICAgICAgICAgIGNoYXJhY3RlciA9IHByaW1hcnkuZ2V0KCJjaGFyYWN0ZXIiLCAidW5rbm93biIpCiAgICAgICAgICAgICAgICBtZXRob2QgPSBwcmltYXJ5LmdldCgibWV0aG9kIiwgInVua25vd24iKQogICAgICAgICAgICAgICAgY29uZiA9IHByaW1hcnkuZ2V0KCJjb25maWRlbmNlIiwgMCkKICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJQcmltYXJ5IGNoYXJhY3RlcjogJXMgKG1ldGhvZD0lcywgY29uZj0lLjJmKSIsIGNoYXJhY3RlciwgbWV0aG9kLCBjb25mKQogICAgICAgICAgICAgICAgIyBBdXRvLWV4dHJhY3Qgdm9pY2UgaWYgbmVlZGVkCiAgICAgICAgICAgICAgICBpZiBjaGFyYWN0ZXIgIT0gInVua25vd24iIGFuZCB2b2ljZV9leDoKICAgICAgICAgICAgICAgICAgICBpZiBub3Qgdm9pY2VfZXguaGFzX3ZvaWNlKGNoYXJhY3Rlcik6CiAgICAgICAgICAgICAgICAgICAgICAgIHNzID0gcHJpbWFyeS5nZXQoInN0YXJ0X3NlYyIsIDApCiAgICAgICAgICAgICAgICAgICAgICAgIGVzID0gcHJpbWFyeS5nZXQoImVuZF9zZWMiLCBzcyArIDEwKQogICAgICAgICAgICAgICAgICAgICAgICBsb2dnZXIuaW5mbygiQXV0by1leHRyYWN0aW5nIHZvaWNlOiAlcyAoJS4xZi0lLjFmKSIsIGNoYXJhY3Rlciwgc3MsIGVzKQogICAgICAgICAgICAgICAgICAgICAgICB2b2ljZV9leC5leHRyYWN0X3ZvaWNlKHZpZGVvX3BhdGgsIHNzLCBlcywgY2hhcmFjdGVyKQogICAgICAgICMgTG9nIHZvaWNlIGxpYnJhcnkKICAgICAgICBpZiB2b2ljZV9leDoKICAgICAgICAgICAgdmxpc3QgPSB2b2ljZV9leC5saXN0X3ZvaWNlcygpCiAgICAgICAgICAgIGZvciB2IGluIHZsaXN0OgogICAgICAgICAgICAgICAgdm4gPSB2WyJuYW1lIl0KICAgICAgICAgICAgICAgIHZzID0gdlsic2l6ZV9rYiJdCiAgICAgICAgICAgICAgICB2ZSA9ICIrIiBpZiB2WyJoYXNfZW1iZWRkaW5nIl0gZWxzZSAiLSIKICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJWb2ljZSBsaWJyYXJ5OiAlcyAoJS4wZiBLQikgZW1iPSVzIiwgdm4sIHZzLCB2ZSkKICAgICAgICAjIFRUUyBuYXJyYXRpb24gd2l0aCBjaGFyYWN0ZXIgdm9pY2UKICAgICAgICB0dHNfcGF0aCA9IE5vbmUKICAgICAgICBpZiBicmFpbiBhbmQgYnJhaW4uYXZhaWxhYmxlIGFuZCB0dHMgYW5kIHR0cy5hdmFpbGFibGU6CiAgICAgICAgICAgIHRleHQgPSBicmFpbi53cml0ZV9uYXJyYXRpb24oImVwaWMiLCBjaGFyYWN0ZXI9Y2hhcmFjdGVyKQogICAgICAgICAgICBpZiB0ZXh0OgogICAgICAgICAgICAgICAgdHRzX3BhdGggPSBvcy5wYXRoLmpvaW4oc3RyKGNvbmZpZy5kaXJfdGVtcCksICJuYXJyYXRpb24ud2F2IikKICAgICAgICAgICAgICAgIGlmIGNoYXJhY3RlciBhbmQgY2hhcmFjdGVyICE9ICJ1bmtub3duIjoKICAgICAgICAgICAgICAgICAgICB0dHMuc3ludGhlc2l6ZV9hcyh0ZXh0LCBjaGFyYWN0ZXIsIHR0c19wYXRoKQogICAgICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJOYXJyYXRpb24gYXMgJXM6ICVzIiwgY2hhcmFjdGVyLCB0ZXh0Wzo4MF0pCiAgICAgICAgICAgICAgICBlbHNlOgogICAgICAgICAgICAgICAgICAgIHR0cy5zeW50aGVzaXplKHRleHQsIHR0c19wYXRoKQogICAgICAgICAgICAgICAgICAgIGxvZ2dlci5pbmZvKCJOYXJyYXRpb24gKGRlZmF1bHQpOiAlcyIsIHRleHRbOjgwXSkKICAgICAgICAjIFByb2Nlc3MgdmlkZW8gc2VnbWVudHMKICAgICAgICBzZWdfcGF0aHMgPSBbXQogICAgICAgIG1heF9zZWdzID0gbWluKGxlbihzY2VuZXMpLCBjb25maWcubWF4X3NlZ21lbnRzKQogICAgICAgIGZvciBpIGluIHJhbmdlKG1heF9zZWdzKToKICAgICAgICAgICAgc2MgPSBzY2VuZXNbaV0KICAgICAgICAgICAgc2VnX291dCA9IG9zLnBhdGguam9pbihzdHIoY29uZmlnLmRpcl90ZW1wKSwgInNlZ197fS5tcDQiLmZvcm1hdChpKSkKICAgICAgICAgICAgc3MgPSBzYy5nZXQoInN0YXJ0X3NlYyIsIDApCiAgICAgICAgICAgIGR1ciA9IHNjLmdldCgiZHVyYXRpb24iLCBOb25lKQogICAgICAgICAgICBzZWxmLnByb2Nlc3Nfc2VnbWVudCh2aWRlb19wYXRoLCBzZWdfb3V0LCBzdGFydD1zcywgZHVyPWR1cikKICAgICAgICAgICAgaWYgc2VsZi5fcmlmZSBhbmQgc2VsZi5fcmlmZS5hdmFpbGFibGUgYW5kIGNvbmZpZy5yaWZlX2VuYWJsZWQ6CiAgICAgICAgICAgICAgICByaWZlX291dCA9IG9zLnBhdGguam9pbihzdHIoY29uZmlnLmRpcl90ZW1wKSwgInNlZ197fV9yaWZlLm1wNCIuZm9ybWF0KGkpKQogICAgICAgICAgICAgICAgc2VsZi5fcmlmZS5pbnRlcnBvbGF0ZV92aWRlbyhzZWdfb3V0LCByaWZlX291dCwgY29uZmlnLnJpZmVfZmFjdG9yKQogICAgICAgICAgICAgICAgc2VnX3BhdGhzLmFwcGVuZChyaWZlX291dCkKICAgICAgICAgICAgZWxzZToKICAgICAgICAgICAgICAgIHNlZ19wYXRocy5hcHBlbmQoc2VnX291dCkKICAgICAgICBmaW5hbCA9IHNlbGYuY29tcG9zZV9maW5hbChzZWdfcGF0aHMsIGF1ZGlvX3BhdGgsIG91dHB1dF9wYXRoLCB0dHNfcGF0aCkKICAgICAgICByZXR1cm4gZmluYWwK",
}


def main():
    if not os.path.isdir("pipeline"):
        print("[!!] pipeline/ klasoru bulunamadi!")
        print("     Bu scripti pipeline_project/ icinden calistirin.")
        return
    os.makedirs("voices", exist_ok=True)
    for path, b64 in FILES.items():
        dp = os.path.dirname(path)
        if dp:
            os.makedirs(dp, exist_ok=True)
        with open(path, "wb") as f:
            f.write(base64.b64decode(b64))
        print("  [OK] " + path)
    env_path = ".env"
    if os.path.isfile(env_path):
        with open(env_path, "r") as f:
            content = f.read()
        if "VOICES_DIR" not in content:
            with open(env_path, "a") as f:
                f.write("\nVOICES_DIR=./voices\n")
            print("  [OK] .env: VOICES_DIR eklendi")
    print()
    print("=" * 60)
    print("  Audio-Visual Speaker Verification TAMAM!")
    print()
    print("  Kurulum:")
    print("    pip install resemblyzer demucs")
    print()
    print("  Nasil calisir:")
    print("    1. Brain frame'den karakter tanimlar (Gemma vision)")
    print("    2. FFmpeg o segmentin sesini keser")
    print("    3. Demucs BGM/SFX ayirir -> saf vokal")
    print("    4. Resemblyzer ses parmak izi cikarir")
    print("    5. Kutuphaneyle karsilastirir -> onaylar/duzeltir")
    print("    6. Bir sonraki videoda ayni sesi otomatik tanir")
    print()
    print("  python -m pipeline.main")
    print("=" * 60)

if __name__ == "__main__":
    main()


────────────────────────────────────────────────────────────
FILE: pipeline\__init__.py
SIZE: 0.0 KB
────────────────────────────────────────────────────────────
"""Pipeline v10.2"""
__version__="10.2.0"


────────────────────────────────────────────────────────────
FILE: pipeline\acquisition.py
SIZE: 1.8 KB
────────────────────────────────────────────────────────────
"""acquisition.py — YouTube search & download via yt-dlp."""
import os, json, subprocess
from pipeline.core import config, logger, run_sub, Database

def search_download(db=None):
    query = config.search_query
    if not query:
        logger.warning("SEARCH_QUERY is empty")
        return []
    outdir = str(config.dir_video)
    os.makedirs(outdir, exist_ok=True)
    cmd = ["yt-dlp", "--dump-json", "--no-download",
           f"ytsearch10:{query}",
           "--match-filter", f"view_count>={config.min_views}"]
    try:
        r = run_sub(cmd, "yt-dlp-search", ignore=True)
    except Exception as e:
        logger.error("yt-dlp search failed: %s", e)
        return []
    videos = []
    for line in r.stdout.strip().split("\n"):
        if not line.strip():
            continue
        try:
            info = json.loads(line)
            vid = info.get("id", "")
            title = info.get("title", "unknown")
            if db and db.is_downloaded(vid):
                logger.info("Skip (already downloaded): %s", vid)
                continue
            videos.append({"id": vid, "title": title, "url": info.get("webpage_url", "")})
        except json.JSONDecodeError:
            continue
    downloaded = []
    for v in videos[:5]:
        out_tpl = os.path.join(outdir, "%(id)s.%(ext)s")
        dl_cmd = ["yt-dlp", "-f", "bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4",
                   "-o", out_tpl, v["url"]]
        try:
            run_sub(dl_cmd, "yt-dlp-dl")
            fp = os.path.join(outdir, v["id"] + ".mp4")
            if os.path.isfile(fp):
                if db:
                    db.record_dl(v["id"], "youtube")
                downloaded.append(fp)
                logger.info("Downloaded: %s -> %s", v["title"], fp)
        except Exception as e:
            logger.error("Download failed for %s: %s", v["id"], e)
    return downloaded


────────────────────────────────────────────────────────────
FILE: pipeline\audio.py
SIZE: 1.1 KB
────────────────────────────────────────────────────────────
"""audio.py — Beat analysis with librosa."""
import numpy as np
from pipeline.core import config,logger,IAudio

class AudioAn(IAudio):
    def compute_beats(self,path):
        import librosa
        y,sr=librosa.load(path,sr=22050,mono=True)
        tempo,beats=librosa.beat.beat_track(y=y,sr=sr)
        bt=librosa.frames_to_time(beats,sr=sr)
        onset_env=librosa.onset.onset_strength(y=y,sr=sr)
        onsets=librosa.onset.onset_detect(y=y,sr=sr,onset_envelope=onset_env,units="time")
        sc=librosa.feature.spectral_centroid(y=y,sr=sr)[0]
        sc_times=librosa.frames_to_time(np.arange(len(sc)),sr=sr)
        peaks=sorted(set(list(bt)+list(onsets)))
        result=[]
        for t in peaks:
            idx=np.argmin(np.abs(sc_times-t))
            score=float(sc[idx])/4000.0
            result.append({"time":float(t),"score":min(1.0,score)})
        gap=0.08
        filtered=[]
        for p in result:
            if not filtered or p["time"]-filtered[-1]["time"]>=gap:
                filtered.append(p)
        logger.info(f"Audio: {len(filtered)} peaks, BPM={float(tempo) if np.ndim(tempo)==0 else float(tempo[0]):.0f}")
        return filtered


────────────────────────────────────────────────────────────
FILE: pipeline\brain.py
SIZE: 6.6 KB
────────────────────────────────────────────────────────────
"""brain.py — PipelineBrain with audio-visual character verification."""
import os, json, base64
from pipeline.core import config, logger


class PipelineBrain:
    def __init__(self):
        self._model = config.ollama_model
        self._url = config.ollama_url
        self.available = False
        try:
            import requests
            r = requests.get(self._url, timeout=3)
            if r.status_code == 200:
                self.available = True
                logger.info("Brain: Ollama OK (model=%s)", self._model)
        except Exception:
            logger.warning("Brain: Ollama not reachable at %s", self._url)

    def _chat(self, messages, timeout=60):
        import requests
        url = self._url.rstrip("/") + "/api/chat"
        payload = {"model": self._model, "messages": messages, "stream": False}
        try:
            r = requests.post(url, json=payload, timeout=timeout)
            data = r.json()
            return data.get("message", {}).get("content", "").strip()
        except Exception as e:
            logger.error("Brain chat error: %s", e)
            return ""

    def _frame_to_b64(self, frame):
        import cv2
        ok, buf = cv2.imencode(".jpg", frame, [cv2.IMWRITE_JPEG_QUALITY, 70])
        if ok:
            return base64.b64encode(buf.tobytes()).decode("ascii")
        return ""

    def _extract_json(self, raw):
        try:
            start = raw.find("{")
            end = raw.rfind("}") + 1
            if start >= 0 and end > start:
                return json.loads(raw[start:end])
        except (json.JSONDecodeError, ValueError):
            pass
        return None

    def analyze_scene(self, frames, context="anime"):
        if not self.available or not frames:
            return {"mood": "unknown", "action": "unknown", "character": "unknown"}
        imgs = []
        for f in frames[:3]:
            b64 = self._frame_to_b64(f)
            if b64:
                imgs.append(b64)
        content_parts = []
        for img in imgs:
            content_parts.append({"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + img}})
        content_parts.append({
            "type": "text",
            "text": (
                "Bu anime sahnesini analiz et. JSON formatinda cevap ver:\n"
                '{"mood": "...", "action": "...", "character": "karakter_adi", '
                '"intensity": 1-10, "speaking": true/false}\n'
                "Eger karakteri tanimlayamiyorsan character: \"unknown\" yaz.\n"
                "Sadece JSON don, baska bir sey yazma."
            )
        })
        raw = self._chat([{"role": "user", "content": content_parts}], timeout=90)
        result = self._extract_json(raw)
        if result:
            return result
        return {"mood": "unknown", "action": "unknown", "character": "unknown"}

    def identify_speakers(self, video_path, scenes, voice_ex=None):
        if not self.available:
            return []
        import cv2
        results = []
        for i, sc in enumerate(scenes[:8]):
            ss = sc.get("start_sec", 0)
            es = sc.get("end_sec", ss + 5)
            cap = cv2.VideoCapture(video_path)
            fps = cap.get(cv2.CAP_PROP_FPS) or 24
            mid = int((ss + es) / 2 * fps)
            cap.set(cv2.CAP_PROP_POS_FRAMES, mid)
            ok, frame = cap.read()
            cap.release()
            if not ok or frame is None:
                continue
            b64 = self._frame_to_b64(frame)
            if not b64:
                continue
            content_parts = [
                {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64," + b64}},
                {"type": "text", "text": (
                    "Bu anime sahnesindeki konusan karakteri tanimla.\n"
                    'JSON: {"character": "isim", "confidence": 0.0-1.0, "speaking": true/false}\n'
                    "Tanimiyorsan: {\"character\": \"unknown\", \"confidence\": 0.0, \"speaking\": false}\n"
                    "Sadece JSON don."
                )}
            ]
            raw = self._chat([{"role": "user", "content": content_parts}], timeout=60)
            info = self._extract_json(raw)
            if not info:
                continue
            visual_char = info.get("character", "unknown")
            conf = info.get("confidence", 0)
            speaking = info.get("speaking", False)
            if visual_char == "unknown" or conf < 0.4 or not speaking:
                continue
            # Audio-visual verification
            if voice_ex and voice_ex.can_match:
                verified = voice_ex.verify_speaker(video_path, ss, es, visual_guess=visual_char)
                final_char = verified.get("character", visual_char)
                method = verified.get("method", "visual_only")
                final_conf = verified.get("confidence", conf)
                logger.info("Scene %d: %s -> %s (%s, conf=%.2f)", i, visual_char, final_char, method, final_conf)
                results.append({
                    "character": final_char,
                    "confidence": final_conf,
                    "method": method,
                    "start_sec": ss,
                    "end_sec": es,
                    "scene_idx": i
                })
            else:
                logger.info("Scene %d: %s (visual only, conf=%.2f)", i, visual_char, conf)
                results.append({
                    "character": visual_char,
                    "confidence": conf,
                    "method": "visual_only",
                    "start_sec": ss,
                    "end_sec": es,
                    "scene_idx": i
                })
        return results

    def write_narration(self, style="epic", character=None):
        if not self.available:
            return ""
        if character and character != "unknown":
            prompt = (
                "Sen {} karakterisin. Anime meme icerigi icin bu karaktere uygun, "
                "dramatik ve komik bir Turkce narration yaz. KURALLAR: Sadece duz metin yaz. Markdown kullanma. Secenekler listeleme. Tirnak isareti kullanma. Baslik yazma. Tek bir versiyon yaz. Maksimum 2-3 cumle. "
                "Karakterin konusma tarzi ve kisiligini yansit."
            ).format(character)
        else:
            prompt = (
                "Anime meme icerigi icin {} tarzinda dramatik ve komik "
                "bir Turkce narration yaz. 2-3 cumle. Kisa ve etkileyici olsun."
            ).format(style)
        return self._chat([{"role": "user", "content": prompt}])


────────────────────────────────────────────────────────────
FILE: pipeline\core.py
SIZE: 13.1 KB
────────────────────────────────────────────────────────────
from pipeline import dml_guard  # auto-fix DirectML
"""core.py"""
import sys,os,time,json,sqlite3,subprocess,shutil
import signal,atexit,logging,threading,gc
from abc import ABC,abstractmethod
from dataclasses import dataclass,field
from pathlib import Path
from datetime import datetime,timezone
from typing import Optional,List,Dict,Tuple
from logging.handlers import RotatingFileHandler
import numpy as np

def resolve_dependencies():
    return  # disabled� deps managed externally:
    deps=["yt-dlp","scenedetect[opencv]","librosa>=0.10.2","pandas>=2.2.0",
          "opencv-python>=4.9.0","numpy>=1.26,<2.0","requests>=2.31.0",
          "python-dotenv>=1.0.0","basicsr>=1.4.2","realesrgan>=0.3.0",
          "onnxruntime-directml>=1.17.0","colorama","supertonic"]
    with open("requirements.txt","w") as f:
        for d in deps: f.write(d+"\n")
    try:
        import importlib.metadata as im; miss=False
        for n in ["yt_dlp","scenedetect","librosa","pandas","cv2","numpy",
                   "requests","dotenv","basicsr","realesrgan","onnxruntime","colorama","supertonic"]:
            try: im.version(n.replace("cv2","opencv-python").replace("dotenv","python-dotenv"))
            except im.PackageNotFoundError: miss=True;break
        if miss: subprocess.run([sys.executable,"-m","pip","install","-r","requirements.txt","--quiet"],check=True)
    except Exception as e: print(f"[FATAL] Bootstrap: {e}"); sys.exit(1)
pass  # resolve_dependencies disabled� deps managed externally
from dotenv import load_dotenv; load_dotenv()
import cv2,librosa,onnxruntime as ort
from scenedetect import open_video,SceneManager
from scenedetect.detectors import AdaptiveDetector

class GPUBackend:
    def __init__(self):
        self.ort_providers=["DmlExecutionProvider","CPUExecutionProvider"]
        self.torch_device="cpu"; self.use_half=False
        self.hw_decode=["-hwaccel","d3d11va"]
        self.hw_h264="h264_amf"; self.hw_hevc="hevc_amf"; self.hw_av1="av1_amf"
        try:
            import torch
            if torch.cuda.is_available(): self.torch_device="cuda"; self.use_half=True
        except ImportError: pass
    def summary(self): return {"ort":self.ort_providers,"dev":self.torch_device,"h264":self.hw_h264}
gpu_backend=GPUBackend()

@dataclass
class Cfg:
    version:str="10.2.0"
    pipeline_mode:int=int(os.getenv("PIPELINE_MODE","1"))
    search_query:str=os.getenv("SEARCH_QUERY","")
    min_views:int=int(os.getenv("MIN_VIEW_COUNT","100000"))
    scene_var:float=float(os.getenv("SCENE_VARIANCE_THRESHOLD","3.0"))
    max_segments:int=int(os.getenv("MAX_SEGMENT_COUNT","15"))
    yolo_path:str=os.getenv("YOLO_ONNX_PATH","yolov8n.onnx")
    esrgan_path:str=os.getenv("ESRGAN_WEIGHTS_PATH","RealESRGAN_x4plus.pth")
    yolo_class:int=int(os.getenv("YOLO_TARGET_CLASS","0"))
    yolo_conf:float=float(os.getenv("YOLO_CONF_THRESHOLD","0.35"))
    batch_size:int=int(os.getenv("INFERENCE_BATCH_SIZE","16"))
    upscale_factor:int=int(os.getenv("UPSCALE_FACTOR","2"))
    max_retries:int=int(os.getenv("MAX_RETRY_COUNT","3"))
    queue_depth:int=int(os.getenv("PIPELINE_QUEUE_DEPTH","4"))
    enc_quality:int=int(os.getenv("ENCODE_QUALITY_LEVEL","28"))
    enc_bitrate:str=os.getenv("ENCODE_BITRATE_CEILING","12M")
    tts_enabled:bool=os.getenv("TTS_ENABLED","true").lower()=="true"
    tts_lang:str=os.getenv("TTS_LANGUAGE","tr")
    tts_voice:str=os.getenv("TTS_VOICE","M1")
    tts_steps:int=int(os.getenv("TTS_STEPS","8"))
    tts_speed:float=float(os.getenv("TTS_SPEED","1.0"))
    tts_voice_ref:str=os.getenv("TTS_VOICE_REF","")
    rife_factor:int=int(os.getenv("RIFE_FACTOR","2"))
    rife_enabled:bool=os.getenv("RIFE_ENABLED","false").lower()=="true"
    ollama_model:str=os.getenv("OLLAMA_MODEL","gemma4:e2b")
    ollama_url:str=os.getenv("OLLAMA_URL","http://localhost:11434")
    dir_video:Path=Path(os.getenv("DIR_INPUT_VIDEO","./1_input_videos")).resolve()
    dir_audio:Path=Path(os.getenv("DIR_INPUT_AUDIO","./2_input_music")).resolve()
    dir_temp:Path=Path(os.getenv("DIR_TEMP","./temp_processing")).resolve()
    dir_output:Path=Path(os.getenv("DIR_OUTPUT","./output_ready")).resolve()
    dir_tools:Path=Path(os.getenv("DIR_TOOLS","./tools")).resolve()
    db_file:str=os.getenv("DATABASE_FILE","pipeline_state.db")
    ffmpeg:str=field(init=False); ffprobe:str=field(init=False); rife_bin:str=field(init=False)
    def __post_init__(self):
        self.ffmpeg=shutil.which("ffmpeg") or str(self.dir_tools/"ffmpeg.exe")
        self.ffprobe=shutil.which("ffprobe") or str(self.dir_tools/"ffprobe.exe")
        self.rife_bin=shutil.which("rife-ncnn-vulkan") or str(self.dir_tools/"rife-ncnn-vulkan.exe")
config=Cfg()

@dataclass
class ErrCtx:
    job_id:int=0;stage:str=""
    ts:str=field(default_factory=lambda:datetime.now(timezone.utc).isoformat())
class StructFmt(logging.Formatter):
    def format(self,r):
        return json.dumps({"ts":self.formatTime(r),"lvl":r.levelname,"mod":r.module,"msg":r.getMessage()},ensure_ascii=False)
def setup_logger():
    lg=logging.getLogger("pipeline");lg.setLevel(logging.INFO)
    if not lg.handlers:
        os.makedirs("logs",exist_ok=True)
        fh=RotatingFileHandler("logs/pipeline.log",maxBytes=5_000_000,backupCount=3,encoding="utf-8")
        fh.setFormatter(StructFmt());lg.addHandler(fh)
        sh=logging.StreamHandler(sys.stdout)
        sh.setFormatter(logging.Formatter("%(asctime)s [%(levelname)s] %(message)s"));lg.addHandler(sh)
    return lg
logger=setup_logger()
class PipeErr(Exception):
    def __init__(self,msg,ctx=None):self.ctx=ctx;super().__init__(msg)
class SubprocErr(PipeErr):pass
class GPUErr(PipeErr):pass
class FatalErr(PipeErr):pass
class Metrics:
    def __init__(self):
        self._lk=threading.Lock();self.times={};self.frames=0;self.jobs=0
    def rec(self,s,d):
        with self._lk:self.times.setdefault(s,[]).append(d)
    def add_frames(self,n):
        with self._lk:self.frames+=n
    def summary(self):
        with self._lk:
            r={}
            for s,d in self.times.items():
                a=np.array(d);r[s]={"mean_ms":round(float(np.mean(a)*1000),1),"count":len(d)}
            r["frames"]=self.frames;r["jobs"]=self.jobs;return r
metrics=Metrics()

class IDecoder(ABC):
    @abstractmethod
    def read_frame(self):...
    @abstractmethod
    def read_batch(self,n):...
    @abstractmethod
    def close(self):...
class IEncoder(ABC):
    @abstractmethod
    def write_frame(self,f):...
    @abstractmethod
    def close(self):...
class IDetector(ABC):
    @abstractmethod
    def detect_batch(self,frames):...
class IUpscaler(ABC):
    @abstractmethod
    def enhance(self,f):...
class IAudio(ABC):
    @abstractmethod
    def compute_beats(self,p):...
class IScene(ABC):
    @abstractmethod
    def find_scenes(self,p,w):...
class ITTS(ABC):
    @abstractmethod
    def synthesize(self,text,out,lang="tr"):...

class CircuitBreaker:
    def __init__(self,name,thr=5,timeout=60):
        self.name=name;self._thr=thr;self._to=timeout
        self._state="CLOSED";self._fails=0;self._last=0;self._lk=threading.Lock()
    def can_exec(self):
        with self._lk:
            if self._state=="CLOSED":return True
            if self._state=="OPEN" and time.time()-self._last>=self._to:
                self._state="HALF";return True
            return self._state=="HALF"
    def ok(self):
        with self._lk:self._state="CLOSED";self._fails=0
    def fail(self):
        with self._lk:
            self._fails+=1;self._last=time.time()
            if self._fails>=self._thr or self._state=="HALF":self._state="OPEN"
    def stats(self):
        with self._lk:return{"name":self.name,"state":self._state,"fails":self._fails}
class CBReg:
    def __init__(self):
        self._b={"dec":CircuitBreaker("dec"),"enc":CircuitBreaker("enc"),
                  "det":CircuitBreaker("det",3,30),"ups":CircuitBreaker("ups",3,45),
                  "tts":CircuitBreaker("tts",3,30)}
    def get(self,n):return self._b[n]
    def stats(self):return{n:b.stats()for n,b in self._b.items()}
breakers=CBReg()

class Resources:
    def opt_tile(self,base=400):
        try:
            import torch
            if torch.cuda.is_available():
                f,_=torch.cuda.mem_get_info()
                if f/1e9<1:return max(128,base//4)
                if f/1e9<2:return max(128,base//2)
        except:pass
        return base
resources=Resources()

class Adaptive:
    def __init__(self,db):
        self._db=db;self._lk=threading.Lock()
        c=sqlite3.connect(db,timeout=10)
        c.execute("CREATE TABLE IF NOT EXISTS adapt(id INTEGER PRIMARY KEY AUTOINCREMENT,"
                  "job_id INT,seg INT,dur REAL,frames INT,errs INT,batch INT,tile INT,"
                  "ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP)")
        c.commit();c.close()
    def record(self,job_id,seg,dur,frames,errs,batch,tile):
        with self._lk:
            c=sqlite3.connect(self._db,timeout=10)
            c.execute("INSERT INTO adapt(job_id,seg,dur,frames,errs,batch,tile)VALUES(?,?,?,?,?,?,?)",
                      (job_id,seg,dur,frames,errs,batch,tile));c.commit();c.close()
    def get_params(self,job_id):
        with self._lk:
            c=sqlite3.connect(self._db,timeout=10)
            rows=c.execute("SELECT dur,frames,errs,batch,tile FROM adapt WHERE job_id=? ORDER BY seg",(job_id,)).fetchall();c.close()
        if not rows:return{"batch":config.batch_size,"tile":400,"action":"default"}
        te=sum(r[2]for r in rows);lb,lt=rows[-1][3],rows[-1][4]
        if te>0:return{"batch":max(1,lb//2),"tile":max(128,lt//2),"action":"downgrade"}
        return{"batch":lb,"tile":lt,"action":"stable"}
adaptive=Adaptive(config.db_file)

class SubReg:
    def __init__(self):
        self.active=[];self._lk=threading.Lock();atexit.register(self.kill_all)
    def add(self,p):
        with self._lk:self.active.append(p)
    def remove(self,p):
        with self._lk:
            if p in self.active:self.active.remove(p)
    def kill_all(self):
        with self._lk:
            for p in self.active:
                if p.poll()is None:
                    try:p.kill()
                    except:pass
proc_reg=SubReg()
def _si():
    si=subprocess.STARTUPINFO();si.dwFlags|=subprocess.STARTF_USESHOWWINDOW;si.wShowWindow=0;return si
def run_sub(cmd,label,ignore=False):
    p=subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.PIPE,
                       text=True,encoding="utf-8",errors="replace",startupinfo=_si())
    proc_reg.add(p)
    try:out,err=p.communicate(timeout=600)
    except subprocess.TimeoutExpired:p.kill();out,err=p.communicate()
    finally:proc_reg.remove(p)
    if p.returncode!=0 and not ignore:
        raise SubprocErr(f"[{label}] exit {p.returncode}\n{err[:500]}",ctx=ErrCtx(stage=label))
    return subprocess.CompletedProcess(p.args,p.returncode,out,err)

class Database:
    def __init__(self,path):
        self._path=path;self._local=threading.local();self._schema(self._c())
    def _c(self):
        if not hasattr(self._local,"conn")or self._local.conn is None:
            c=sqlite3.connect(self._path,timeout=30)
            c.execute("PRAGMA journal_mode=WAL");c.execute("PRAGMA busy_timeout=30000")
            self._local.conn=c
        return self._local.conn
    def _schema(self,c):
        c.executescript("""
            CREATE TABLE IF NOT EXISTS downloads(cid TEXT PRIMARY KEY,cat TEXT,ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
            CREATE TABLE IF NOT EXISTS jobs(id INTEGER PRIMARY KEY AUTOINCREMENT,vpath TEXT,apath TEXT,status TEXT DEFAULT 'PENDING',retries INT DEFAULT 0,created TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
            CREATE TABLE IF NOT EXISTS outputs(id INTEGER PRIMARY KEY AUTOINCREMENT,job_id INT,path TEXT,quality REAL DEFAULT 0,ts TIMESTAMP DEFAULT CURRENT_TIMESTAMP);
        """);c.commit()
    def is_downloaded(self,cid):return self._c().execute("SELECT 1 FROM downloads WHERE cid=?",(cid,)).fetchone()is not None
    def record_dl(self,cid,cat):c=self._c();c.execute("INSERT OR IGNORE INTO downloads(cid,cat)VALUES(?,?)",(cid,cat));c.commit()
    def add_job(self,vp,ap=None):c=self._c();cur=c.execute("INSERT INTO jobs(vpath,apath)VALUES(?,?)",(vp,ap));c.commit();return cur.lastrowid
    def claim(self):
        c=self._c()
        cur=c.execute("UPDATE jobs SET status='PROC' WHERE id=(SELECT id FROM jobs WHERE status='PENDING' OR(status='FAIL' AND retries<?)ORDER BY created LIMIT 1)RETURNING id,vpath,apath,retries",(config.max_retries,))
        row=cur.fetchone();c.commit()
        if row:return{"id":row[0],"vpath":row[1],"apath":row[2],"retries":row[3]}
        return None
    def done(self,jid,path,q=0):
        c=self._c();c.execute("UPDATE jobs SET status='DONE' WHERE id=?",(jid,))
        c.execute("INSERT INTO outputs(job_id,path,quality)VALUES(?,?,?)",(jid,path,q));c.commit()
    def fail(self,jid,msg,fatal=False):
        c=self._c();inc=config.max_retries if fatal else 1
        c.execute("UPDATE jobs SET status='FAIL',retries=retries+? WHERE id=?",(inc,jid));c.commit()
    def stats(self):
        c=self._c();return{r[0]:r[1]for r in c.execute("SELECT status,COUNT(*)FROM jobs GROUP BY status").fetchall()}


────────────────────────────────────────────────────────────
FILE: pipeline\decoder.py
SIZE: 2.5 KB
────────────────────────────────────────────────────────────
"""decoder.py — FFmpeg d3d11va decode."""
import subprocess,time,json
import numpy as np
from pipeline.core import config,logger,metrics,breakers,run_sub,IDecoder,_si,proc_reg

class FFmpegDec(IDecoder):
    def __init__(self,path,ss=None,to=None):
        self.path=path;self._w=0;self._h=0;self._fps=30.0;self._proc=None
        self._probe(path);self._open(ss,to)
    def _probe(self,path):
        cmd=[config.ffprobe,"-v","quiet","-print_format","json","-show_streams",path]
        r=run_sub(cmd,"ffprobe")
        info=json.loads(r.stdout)
        for s in info.get("streams",[]):
            if s.get("codec_type")=="video":
                self._w=int(s["width"]);self._h=int(s["height"])
                fr=s.get("r_frame_rate","30/1").split("/")
                self._fps=int(fr[0])/max(int(fr[1]),1);break
        if not self._w:raise RuntimeError(f"No video stream: {path}")
    def _open(self,ss,to):
        cmd=[config.ffmpeg]+config.gpu_backend_hw_decode
        if ss:cmd+=["-ss",str(ss)]
        if to:cmd+=["-to",str(to)]
        cmd+=["-i",self.path,"-f","rawvideo","-pix_fmt","bgr24","-v","quiet","-"]
        si=_si()
        self._proc=subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.PIPE,startupinfo=si)
        proc_reg.add(self._proc)
    @property
    def gpu_backend_hw_decode(self):return getattr(config,"hw_decode",["-hwaccel","d3d11va"])
    @property
    def width(self):return self._w
    @property
    def height(self):return self._h
    @property
    def fps(self):return self._fps
    def read_frame(self):
        raw=self._proc.stdout.read(self._w*self._h*3)
        if len(raw)!=self._w*self._h*3:return None
        return np.frombuffer(raw,dtype=np.uint8).reshape((self._h,self._w,3))
    def read_batch(self,n):
        frames=[]
        for _ in range(n):
            f=self.read_frame()
            if f is None:break
            frames.append(f)
        return frames
    def close(self):
        if self._proc:
            try:self._proc.stdout.close()
            except:pass
            try:self._proc.kill()
            except:pass
            proc_reg.remove(self._proc);self._proc=None
    def _open(self,ss,to):
        from pipeline.core import gpu_backend
        cmd=[config.ffmpeg]+gpu_backend.hw_decode
        if ss:cmd+=["-ss",str(ss)]
        if to:cmd+=["-to",str(to)]
        cmd+=["-i",self.path,"-f","rawvideo","-pix_fmt","bgr24","-v","quiet","-"]
        si=_si()
        self._proc=subprocess.Popen(cmd,stdout=subprocess.PIPE,stderr=subprocess.PIPE,startupinfo=si)
        proc_reg.add(self._proc)


────────────────────────────────────────────────────────────
FILE: pipeline\detector.py
SIZE: 2.1 KB
────────────────────────────────────────────────────────────
"""detector.py — YOLOv8 + DirectML."""
import os,time
import numpy as np
import cv2
import onnxruntime as ort
from pipeline.core import config,logger,metrics,gpu_backend,IDetector

class ONNXDet(IDetector):
    def __init__(self,path=None):
        p=path or config.yolo_path;self._sess=None
        if os.path.exists(p):
            so=ort.SessionOptions();so.graph_optimization_level=ort.GraphOptimizationLevel.ORT_ENABLE_ALL
            self._sess=ort.InferenceSession(p,so,providers=gpu_backend.ort_providers)
            self._in=self._sess.get_inputs()[0].name
            logger.info(f"YOLO loaded: {p}")
        else:logger.warning(f"YOLO not found: {p}")
        self.available=self._sess is not None
    def _letterbox(self,img,sz=640):
        h,w=img.shape[:2];scale=min(sz/h,sz/w)
        nh,nw=int(h*scale),int(w*scale)
        resized=cv2.resize(img,(nw,nh))
        canvas=np.full((sz,sz,3),114,dtype=np.uint8)
        dy,dx=(sz-nh)//2,(sz-nw)//2
        canvas[dy:dy+nh,dx:dx+nw]=resized
        return canvas,scale,dx,dy
    def _post(self,out,scale,dx,dy,conf_thr=None):
        ct=conf_thr or config.yolo_conf;dets=[]
        data=out[0] if isinstance(out,list) else out
        if data.ndim==3:data=data[0]
        if data.shape[0]<data.shape[1]:data=data.T
        for row in data:
            scores=row[4:]
            cid=int(np.argmax(scores));conf=float(scores[cid])
            if conf<ct:continue
            cx,cy,bw,bh=row[:4]
            x1=int((cx-bw/2-dx)/scale);y1=int((cy-bh/2-dy)/scale)
            x2=int((cx+bw/2-dx)/scale);y2=int((cy+bh/2-dy)/scale)
            dets.append({"class":cid,"conf":round(conf,3),"box":[x1,y1,x2,y2]})
        return dets
    def detect_batch(self,frames):
        if not self.available:return[[]]*len(frames)
        results=[]
        for fr in frames:
            t0=time.time()
            lb,sc,dx,dy=self._letterbox(fr)
            blob=lb.astype(np.float32).transpose(2,0,1)[None]/255.0
            out=self._sess.run(None,{self._in:blob})
            dets=self._post(out,sc,dx,dy)
            metrics.rec("det",time.time()-t0)
            results.append(dets)
        return results


────────────────────────────────────────────────────────────
FILE: pipeline\dml_guard.py
SIZE: 0.6 KB
────────────────────────────────────────────────────────────
# Auto-fix DirectML — imported by core.py
import importlib, subprocess, sys

def ensure_directml():
    try:
        ort = importlib.import_module("onnxruntime")
        provs = ort.get_available_providers()
        if "DmlExecutionProvider" not in provs:
            raise ImportError("DmlExecutionProvider missing")
    except Exception:
        subprocess.run([sys.executable, "-m", "pip", "uninstall", "onnxruntime", "-y"],
                       capture_output=True)
        subprocess.run([sys.executable, "-m", "pip", "install", "onnxruntime-directml", "--no-deps"],
                       capture_output=True)

ensure_directml()


────────────────────────────────────────────────────────────
FILE: pipeline\encoder.py
SIZE: 1.9 KB
────────────────────────────────────────────────────────────
"""encoder.py — h264_amf + libx264 fallback."""
import subprocess
from pipeline.core import config,logger,breakers,run_sub,IEncoder,_si,proc_reg,gpu_backend

class FFmpegEnc(IEncoder):
    def __init__(self,path,w,h,fps=30.0):
        self.path=path;self._w=w;self._h=h;self._fps=fps;self._proc=None;self._open()
    def _open(self):
        try:self._try_amf()
        except:self._try_x264()
    def _try_amf(self):
        cmd=[config.ffmpeg,"-y","-f","rawvideo","-pix_fmt","bgr24",
             "-s",f"{self._w}x{self._h}","-r",str(self._fps),"-i","-",
             "-c:v",getattr(gpu_backend, 'hw_hevc', gpu_backend.hw_h264),"-quality","speed",
             "-rc","vbr_latency","-qp_i","22","-qp_p","22",
             "-maxrate",config.enc_bitrate,"-bufsize","24M",
             "-pix_fmt","nv12",self.path]
        si=_si()
        self._proc=subprocess.Popen(cmd,stdin=subprocess.PIPE,stderr=subprocess.PIPE,startupinfo=si)
        proc_reg.add(self._proc)
        logger.info("Encoder: hevc_amf H.265 (GPU)")
    def _try_x264(self):
        logger.warning("AMF failed, falling back to libx264")
        cmd=[config.ffmpeg,"-y","-f","rawvideo","-pix_fmt","bgr24",
             "-s",f"{self._w}x{self._h}","-r",str(self._fps),"-i","-",
             "-c:v","libx264","-preset","fast","-crf",str(config.enc_quality),
             "-pix_fmt","yuv420p",self.path]
        si=_si()
        self._proc=subprocess.Popen(cmd,stdin=subprocess.PIPE,stderr=subprocess.PIPE,startupinfo=si)
        proc_reg.add(self._proc)
    def write_frame(self,frame):
        if self._proc and self._proc.stdin:
            try:self._proc.stdin.write(frame.tobytes())
            except BrokenPipeError:logger.error("Encoder pipe broken")
    def close(self):
        if self._proc:
            try:self._proc.stdin.close()
            except:pass
            self._proc.wait(timeout=1800)
            proc_reg.remove(self._proc);self._proc=None


────────────────────────────────────────────────────────────
FILE: pipeline\engine.py
SIZE: 9.8 KB
────────────────────────────────────────────────────────────
"""engine.py — Video engine with verbose logging."""
import os, time, shutil, subprocess, json
from pipeline.core import config, logger, metrics, run_sub


class VideoEngine:
    def __init__(self, upscaler=None, detector=None, meme_fx=None, interpolator=None):
        self._ups = upscaler
        self._det = detector
        self._mfx = meme_fx
        self._rife = interpolator

    def _probe_frames(self, video_path):
        try:
            ffprobe = config.ffmpeg.replace("ffmpeg", "ffprobe")
            probe = subprocess.run([
                ffprobe, "-v", "error", "-select_streams", "v:0",
                "-show_entries", "stream=nb_frames,duration,r_frame_rate",
                "-of", "json", video_path
            ], capture_output=True, text=True, timeout=10)
            info = json.loads(probe.stdout)
            stream = info.get("streams", [{}])[0]
            nb = stream.get("nb_frames", "")
            if nb and nb.isdigit():
                return int(nb)
            dur = float(stream.get("duration", 0))
            rfr = stream.get("r_frame_rate", "24/1")
            parts = rfr.split("/")
            fps_val = float(parts[0]) / float(parts[1]) if len(parts) == 2 else 24.0
            return int(dur * fps_val)
        except Exception:
            return 0

    def process_segment(self, video_path, output_path, start=None, dur=None):
        from pipeline.decoder import FFmpegDec
        from pipeline.encoder import FFmpegEnc
        from pipeline.parallel import ParaPipe
        dec = FFmpegDec(video_path, ss=start, to=dur)
        w, h, fps = dec._w, dec._h, dec._fps
        if self._ups and self._ups.available:
            w = w * config.upscale_factor
            h = h * config.upscale_factor
        enc = FFmpegEnc(output_path, w, h, fps)
        def processor(frame):
            if self._ups and self._ups.available:
                frame = self._ups.enhance(frame)
            return frame
        total_frames = self._probe_frames(video_path)
        if self._ups and self._ups.available:
            self._ups.reset(total_frames)
        pipe = ParaPipe(dec, enc, processor)
        t0 = time.time()
        pipe.run()
        enc.close()
        dec.close()
        elapsed = time.time() - t0
        logger.info("Segment done: %s (%.1fs)", output_path, elapsed)
        return output_path

    def compose_final(self, segments, audio_path, output_path, tts_path=None):
        if not segments:
            return None
        tmp = os.path.join(str(config.dir_temp), "concat.txt")
        os.makedirs(str(config.dir_temp), exist_ok=True)
        with open(tmp, "w") as f:
            for s in segments:
                f.write("file " + repr(os.path.abspath(s)) + "\n")
        cmd = [config.ffmpeg, "-y", "-f", "concat", "-safe", "0", "-i", tmp]
        audio_inputs = []
        if audio_path and os.path.isfile(audio_path):
            cmd += ["-i", audio_path]
            audio_inputs.append("music")
        if tts_path and os.path.isfile(tts_path):
            cmd += ["-i", tts_path]
            audio_inputs.append("tts")
        if len(audio_inputs) == 2:
            cmd += ["-filter_complex",
                    "[1:a]volume=0.7[m];[2:a]volume=1.2[t];[m][t]amix=inputs=2[a]",
                    "-map", "0:v", "-map", "[a]"]
        elif len(audio_inputs) == 1:
            cmd += ["-map", "0:v", "-map", "1:a"]
        else:
            cmd += ["-map", "0:v"]
        cmd += ["-c:v", "copy", "-c:a", "aac", "-shortest", output_path]
        run_sub(cmd, "compose", ignore=True)
        logger.info("Final composed: %s", output_path)
        return output_path

    def full_process(self, video_path, audio_path, output_path,
                     brain=None, tts=None, scene_an=None, voice_ex=None):
        os.makedirs(str(config.dir_temp), exist_ok=True)
        os.makedirs(str(config.dir_output), exist_ok=True)
        vname = os.path.basename(video_path)
        logger.info("")
        logger.info("=" * 60)
        logger.info("  JOB START: %s", vname)
        logger.info("=" * 60)

        # Step 1: Scene detection
        logger.info("[1/8] Sahne tespiti baslatiliyor...")
        scenes = None
        if scene_an:
            scenes, _ = scene_an.find_scenes(video_path)
        if not scenes:
            scenes = [{"start_sec": 0, "end_sec": None, "duration": 0}]
        logger.info("[1/8] Sahne tespiti TAMAM — %d sahne bulundu", len(scenes))

        # Step 2+3: Brain analyze + speaker identification
        character = None
        speakers = []
        if brain and brain.available:
            logger.info("[2/8] Brain sahne analizi baslatiliyor...")
            for i, sc in enumerate(scenes[:8]):
                ss = sc.get("start_sec", 0)
                es = sc.get("end_sec", ss + 5)
                logger.info("  Sahne %d/%d analiz ediliyor (%.1fs-%.1fs)...", i+1, min(len(scenes), 8), ss, es)

            logger.info("[3/8] Konusan karakter tespiti...")
            speakers = brain.identify_speakers(video_path, scenes, voice_ex=voice_ex)
            if speakers:
                primary = speakers[0]
                character = primary.get("character", "unknown")
                method = primary.get("method", "unknown")
                conf = primary.get("confidence", 0)
                logger.info("[3/8] Karakter: %s (method=%s, conf=%.2f)", character, method, conf)
            else:
                logger.info("[3/8] Karakter tespit edilemedi — default ses kullanilacak")

            # Step 4: Voice extraction
            if character and character != "unknown" and voice_ex:
                if not voice_ex.has_voice(character):
                    ss = speakers[0].get("start_sec", 0)
                    es = speakers[0].get("end_sec", ss + 10)
                    logger.info("[4/8] Ses cikariliyor: %s (%.1fs-%.1fs)...", character, ss, es)
                    voice_ex.extract_voice(video_path, ss, es, character)
                    logger.info("[4/8] Ses cikarildi: voices/%s.wav", character)
                else:
                    logger.info("[4/8] Ses zaten mevcut: %s", character)
            else:
                logger.info("[4/8] Ses cikarma atlandi (karakter bilinmiyor)")

            # Voice library status
            if voice_ex:
                vlist = voice_ex.list_voices()
                if vlist:
                    names = [v["name"] for v in vlist]
                    logger.info("  Ses kutuphanesi: %s", ", ".join(names))
        else:
            logger.info("[2/8] Brain kulanilamiyor — atlaniyor")
            logger.info("[3/8] Karakter tespiti atlandi")
            logger.info("[4/8] Ses cikarma atlandi")

        # Step 5+6: Narration + TTS
        tts_path = None
        if brain and brain.available and tts and tts.available:
            logger.info("[5/8] Brain narration yaziyor (karakter=%s)...", character or "genel")
            text = brain.write_narration("epic", character=character)
            if text:
                txt_preview = text[:80]
                logger.info("[5/8] Narration: '%s' (%d karakter)", txt_preview, len(text))
                tts_path = os.path.join(str(config.dir_temp), "narration.wav")
                if character and character != "unknown":
                    logger.info("[6/8] TTS sentezleniyor (%s sesiyle)...", character)
                    tts.synthesize_as(text, character, tts_path)
                else:
                    logger.info("[6/8] TTS sentezleniyor (default ses)...")
                    tts.synthesize(text, tts_path)
                if os.path.isfile(tts_path):
                    sz = os.path.getsize(tts_path) / 1024
                    logger.info("[6/8] TTS TAMAM — narration.wav (%.0f KB)", sz)
                else:
                    logger.info("[6/8] TTS basarisiz")
            else:
                logger.info("[5/8] Brain narration yazamadi")
                logger.info("[6/8] TTS atlandi")
        else:
            logger.info("[5/8] Narration atlandi")
            logger.info("[6/8] TTS atlandi")

        # Step 7: Process segments
        seg_paths = []
        max_segs = min(len(scenes), config.max_segments)
        logger.info("[7/8] %d segment isleniyor (upscale + encode)...", max_segs)
        for i in range(max_segs):
            sc = scenes[i]
            seg_out = os.path.join(str(config.dir_temp), "seg_%d.mp4" % i)
            ss = sc.get("start_sec", 0)
            es = sc.get("end_sec", ss + 6)
            dur = es - ss
            es = sc.get("end_sec", ss + 6)
            logger.info("  Segment %d/%d (%.1fs-%.1fs)...", i+1, max_segs, ss, es)
            t0 = time.time()
            self.process_segment(video_path, seg_out, start=ss, dur=dur)
            elapsed = time.time() - t0
            logger.info("  Segment %d/%d TAMAM (%.1fs)", i+1, max_segs, elapsed)
            if self._rife and self._rife.available and config.rife_enabled:
                rife_out = os.path.join(str(config.dir_temp), "seg_%d_rife.mp4" % i)
                self._rife.interpolate_video(seg_out, rife_out, config.rife_factor)
                seg_paths.append(rife_out)
            else:
                seg_paths.append(seg_out)
        logger.info("[7/8] Tum segmentler tamamlandi")

        # Step 8: Compose final
        logger.info("[8/8] Final birlestirme...")
        final = self.compose_final(seg_paths, audio_path, output_path, tts_path)
        if final and os.path.isfile(final) and os.path.getsize(final) > 1000:
            sz = os.path.getsize(final) / (1024 * 1024)
            logger.info("")
            logger.info("=" * 60)
            logger.info("  JOB DONE: %s (%.1f MB)", os.path.basename(final), sz)
            logger.info("=" * 60)
        else:
            logger.info("  JOB FAILED: %s", vname)
        return final


────────────────────────────────────────────────────────────
FILE: pipeline\interpolator.py
SIZE: 3.7 KB
────────────────────────────────────────────────────────────
"""interpolator.py — RIFE frame interpolation via rife-ncnn-vulkan."""
import os, subprocess, shutil, tempfile, glob
import cv2
from pipeline.core import config, logger, run_sub

class RIFEInterpolator:
    def __init__(self):
        self.available = os.path.isfile(config.rife_bin)
        if self.available:
            logger.info("RIFE: binary found at %s", config.rife_bin)
        else:
            logger.warning("RIFE: binary not found at %s", config.rife_bin)

    def interpolate_pair(self, frame1, frame2, factor=2):
        if not self.available or factor < 2:
            return []
        tmpdir = tempfile.mkdtemp(prefix="rife_")
        indir = os.path.join(tmpdir, "input")
        outdir = os.path.join(tmpdir, "output")
        os.makedirs(indir); os.makedirs(outdir)
        try:
            cv2.imwrite(os.path.join(indir, "00000.png"), frame1)
            cv2.imwrite(os.path.join(indir, "00001.png"), frame2)
            cmd = [
                config.rife_bin,
                "-i", indir, "-o", outdir,
                "-n", str(factor - 1),
                "-f", "%05d.png",
            ]
            run_sub(cmd, "RIFE-pair", ignore=True)
            results = []
            for fp in sorted(glob.glob(os.path.join(outdir, "*.png"))):
                img = cv2.imread(fp)
                if img is not None:
                    results.append(img)
            return results
        except Exception as e:
            logger.error("RIFE pair failed: %s", e)
            return []
        finally:
            shutil.rmtree(tmpdir, ignore_errors=True)

    def interpolate_video(self, input_path, output_path, factor=2):
        if not self.available:
            logger.warning("RIFE not available, copying input to output")
            shutil.copy2(input_path, output_path)
            return output_path
        tmpdir = tempfile.mkdtemp(prefix="rife_vid_")
        indir = os.path.join(tmpdir, "input")
        outdir = os.path.join(tmpdir, "output")
        os.makedirs(indir); os.makedirs(outdir)
        try:
            extract_cmd = [
                config.ffmpeg, "-i", input_path,
                "-qscale:v", "2",
                os.path.join(indir, "%08d.png")
            ]
            run_sub(extract_cmd, "RIFE-extract")
            rife_cmd = [
                config.rife_bin,
                "-i", indir, "-o", outdir,
                "-n", str(factor - 1),
                "-f", "%08d.png",
            ]
            run_sub(rife_cmd, "RIFE-interp")
            probe_cmd = [
                config.ffprobe, "-v", "error",
                "-select_streams", "v:0",
                "-show_entries", "stream=r_frame_rate",
                "-of", "csv=p=0", input_path
            ]
            r = run_sub(probe_cmd, "RIFE-probe", ignore=True)
            try:
                num, den = r.stdout.strip().split("/")
                fps = float(num) / float(den) * factor
            except Exception:
                fps = 30.0 * factor
            encode_cmd = [
                config.ffmpeg, "-y",
                "-framerate", str(fps),
                "-i", os.path.join(outdir, "%08d.png"),
                "-i", input_path,
                "-map", "0:v", "-map", "1:a?",
                "-c:v", "libx264", "-crf", "18",
                "-c:a", "copy",
                "-pix_fmt", "yuv420p",
                output_path
            ]
            run_sub(encode_cmd, "RIFE-encode")
            logger.info("RIFE: interpolated %s -> %s (factor=%d)", input_path, output_path, factor)
            return output_path
        except Exception as e:
            logger.error("RIFE video failed: %s", e)
            shutil.copy2(input_path, output_path)
            return output_path
        finally:
            shutil.rmtree(tmpdir, ignore_errors=True)


────────────────────────────────────────────────────────────
FILE: pipeline\main.py
SIZE: 5.6 KB
────────────────────────────────────────────────────────────
"""main.py — Pipeline entry point."""
import sys, os, glob
from pipeline.core import config, logger, metrics, Database
from pipeline.utilities import preflight, setup_models
from pipeline.menu import show_menu, show_health, show_stats


def _get_audio():
    music = glob.glob(os.path.join(str(config.dir_audio), "*.mp3"))
    if music:
        return music[0]
    return None


def full_auto(db):
    from pipeline.acquisition import search_download
    from pipeline.scene import SceneAn
    from pipeline.brain import PipelineBrain
    from pipeline.engine import VideoEngine
    from pipeline.tts import TTSEngine
    from pipeline.upscaler import Upsc
    from pipeline.meme_fx import MemeFX
    from pipeline.interpolator import RIFEInterpolator

    brain = PipelineBrain()
    tts = TTSEngine() if config.tts_enabled else None
    ups = Upsc()
    mfx = MemeFX()
    rife = RIFEInterpolator()
    scene_an = SceneAn()
    engine = VideoEngine(upscaler=ups, meme_fx=mfx, interpolator=rife)

    videos = search_download(db)
    if not videos:
        videos = glob.glob(os.path.join(str(config.dir_video), "*.mp4"))
    if not videos:
        logger.warning("No videos found")
        return

    audio = _get_audio()
    for vp in videos:
        jid = db.add_job(vp, audio)
        try:
            final_out = os.path.join(str(config.dir_output), "final_{}.mp4".format(jid))
            result = engine.full_process(vp, audio, final_out,
                                         brain=brain, tts=tts, scene_an=scene_an)
            if result:
                db.done(jid, result)
                metrics.jobs += 1
                logger.info("Job %d complete: %s", jid, result)
            else:
                db.fail(jid, "full_process returned None")
        except Exception as e:
            logger.error("Job %d failed: %s", jid, e)
            db.fail(jid, str(e))


def main_loop():
    preflight()
    setup_models()
    db = Database(config.db_file)

    while True:
        try:
            ch = show_menu()
        except (EOFError, KeyboardInterrupt):
            break

        if ch == "0":
            break
        elif ch == "1":
            full_auto(db)
        elif ch == "2":
            p = input("Video yolu: ").strip().strip('"')
            if os.path.exists(p):
                from pipeline.engine import VideoEngine
                from pipeline.upscaler import Upsc
                from pipeline.meme_fx import MemeFX
                ups = Upsc()
                eng = VideoEngine(upscaler=ups, meme_fx=MemeFX())
                jid = db.add_job(p)
                o = os.path.join(str(config.dir_output), "manual_{}.mp4".format(jid))
                eng.process_segment(p, o)
                print("OK: " + o)
        elif ch == "3":
            from pipeline.tts import TTSEngine
            t = TTSEngine()
            if t.available:
                text = input("Metin: ")
                o = os.path.join(str(config.dir_output), "tts.wav")
                t.synthesize(text, o)
                print("OK: " + o)
            else:
                print("TTS kulanilamiyor")
        elif ch == "4":
            from pipeline.upscaler import Upsc
            import cv2
            p = input("Goruntu yolu: ").strip()
            img = cv2.imread(p)
            if img is not None:
                u = Upsc()
                if u.available:
                    out = u.enhance(img)
                    opath = p.rsplit(".", 1)[0] + "_up.png"
                    cv2.imwrite(opath, out)
                    print("OK: " + opath)
                else:
                    print("ESRGAN kulanilamiyor")
        elif ch == "5":
            from pipeline.brain import PipelineBrain
            b = PipelineBrain()
            if b.available:
                print(b.write_narration("epic"))
            else:
                print("Ollama kulanilamiyor")
        elif ch == "6":
            from pipeline.meme_fx import MemeFX
            import cv2
            p = input("Goruntu: ").strip()
            img = cv2.imread(p)
            if img is not None:
                fx = MemeFX()
                r = fx.zoom_cut(fx.speed_lines(fx.camera_shake(img)))
                opath = p.rsplit(".", 1)[0] + "_meme.png"
                cv2.imwrite(opath, r)
                print("OK: " + opath)
        elif ch == "7":
            from pipeline.scene import SceneAn
            p = input("Video: ").strip()
            scenes, _ = SceneAn().find_scenes(p)
            for i, s in enumerate(scenes):
                ss = s["start_sec"]
                es = s["end_sec"]
                print("  {}: {:.1f}s - {:.1f}s".format(i, ss, es))
        elif ch == "8":
            from pipeline.audio import AudioAn
            p = input("Ses dosyasi: ").strip()
            beats = AudioAn().compute_beats(p)
            for x in beats[:10]:
                bt = x["time"]
                bs = x["score"]
                print("  t={:.3f} sc={:.2f}".format(bt, bs))
        elif ch == "9":
            show_health()
            show_stats()
        elif ch == "10":
            from pipeline.interpolator import RIFEInterpolator
            rife = RIFEInterpolator()
            if rife.available:
                p = input("Video yolu: ").strip().strip('"')
                factor = input("Factor (2/4): ").strip()
                factor = int(factor) if factor.isdigit() else 2
                o = p.rsplit(".", 1)[0] + "_rife.mp4"
                rife.interpolate_video(p, o, factor)
                print("OK: " + o)
            else:
                print("RIFE binary bulunamadi: " + config.rife_bin)

    print("Bye!")


if __name__ == "__main__":
    main_loop()


────────────────────────────────────────────────────────────
FILE: pipeline\meme_fx.py
SIZE: 2.4 KB
────────────────────────────────────────────────────────────
"""meme_fx.py — Visual effects for meme content."""
import numpy as np
import cv2
import math

class MemeFX:
    def zoom_cut(self, frame, factor=1.3):
        h, w = frame.shape[:2]
        ch, cw = int(h / factor), int(w / factor)
        y1 = (h - ch) // 2
        x1 = (w - cw) // 2
        crop = frame[y1:y1+ch, x1:x1+cw]
        return cv2.resize(crop, (w, h), interpolation=cv2.INTER_LANCZOS4)

    def speed_lines(self, frame, n=30, thickness=2):
        out = frame.copy()
        h, w = out.shape[:2]
        cx, cy = w // 2, h // 2
        for i in range(n):
            angle = (2 * math.pi * i) / n
            x1 = int(cx + 80 * math.cos(angle))
            y1 = int(cy + 80 * math.sin(angle))
            x2 = int(cx + max(w, h) * math.cos(angle))
            y2 = int(cy + max(w, h) * math.sin(angle))
            cv2.line(out, (x1, y1), (x2, y2), (255, 255, 255), thickness)
        return out

    def camera_shake(self, frame, intensity=8):
        dx = np.random.randint(-intensity, intensity + 1)
        dy = np.random.randint(-intensity, intensity + 1)
        M = np.float32([[1, 0, dx], [0, 1, dy]])
        return cv2.warpAffine(frame, M, (frame.shape[1], frame.shape[0]),
                              borderMode=cv2.BORDER_REFLECT)

    def subtitle_bounce(self, frame, text, t=0, font_scale=1.5, thickness=3):
        out = frame.copy()
        h, w = out.shape[:2]
        y_offset = int(15 * math.sin(t * 4))
        sz = cv2.getTextSize(text, cv2.FONT_HERSHEY_DUPLEX, font_scale, thickness)[0]
        x = (w - sz[0]) // 2
        y = h - 60 + y_offset
        cv2.putText(out, text, (x+2, y+2), cv2.FONT_HERSHEY_DUPLEX,
                    font_scale, (0, 0, 0), thickness + 2)
        cv2.putText(out, text, (x, y), cv2.FONT_HERSHEY_DUPLEX,
                    font_scale, (255, 255, 0), thickness)
        return out

    def frame_freeze(self, frame, count=6):
        return [frame.copy() for _ in range(count)]

    def apply_chain(self, frame, effects, t=0):
        result = frame
        for fx in effects:
            if fx == "zoom_cut":
                result = self.zoom_cut(result)
            elif fx == "speed_lines":
                result = self.speed_lines(result)
            elif fx == "camera_shake":
                result = self.camera_shake(result)
            elif fx == "subtitle_bounce":
                result = self.subtitle_bounce(result, "NANI?!", t)
        return result


────────────────────────────────────────────────────────────
FILE: pipeline\menu.py
SIZE: 0.9 KB
────────────────────────────────────────────────────────────
"""menu.py — Interactive menu."""
from pipeline.core import config, metrics

BANNER = """
 ============================================================
   PIPELINE v10.2 -- Anime Meme Content Factory
   Windows 11 + RX 9070 XT 16GB | Supertonic 3 + Gemma 4 E2B
 ============================================================
"""

MENU = """
[1] Full Auto Pipeline      [2] Manual Video Select
[3] TTS Only                [4] Upscale Only
[5] Brain Analysis          [6] Meme FX Only
[7] Scene Analysis          [8] Audio Analysis
[9] Settings / Health       [10] RIFE Interpolation
[0] Exit
"""

def show_menu():
    print(BANNER)
    print(MENU)
    return input("Secim: ").strip()

def show_health():
    from pipeline.utilities import health_check
    print("\n=== Sistem Sagligi ===")
    health_check()

def show_stats():
    print("\n=== Metrikler ===")
    s = metrics.summary()
    for k, v in s.items():
        print(f"  {k}: {v}")


────────────────────────────────────────────────────────────
FILE: pipeline\parallel.py
SIZE: 1.4 KB
────────────────────────────────────────────────────────────
"""parallel.py — 3-thread pipeline."""
import threading,queue,time
import numpy as np
import cv2
from pipeline.core import config,logger,metrics

class ParaPipe:
    def __init__(self,decoder,encoder,processor=None):
        self._dec=decoder;self._enc=encoder;self._proc=processor
        self._q1=queue.Queue(config.queue_depth)
        self._q2=queue.Queue(config.queue_depth)
    def run(self):
        t1=threading.Thread(target=self._t_dec,daemon=True)
        t2=threading.Thread(target=self._t_proc,daemon=True)
        t3=threading.Thread(target=self._t_enc,daemon=True)
        t1.start();t2.start();t3.start()
        t1.join();t2.join();t3.join()
    def _t_dec(self):
        while True:
            f=self._dec.read_frame()
            if f is None:self._q1.put(None);break
            self._q1.put(f);metrics.add_frames(1)
    def _t_proc(self):
        while True:
            f=self._q1.get()
            if f is None:self._q2.put(None);break
            if self._proc:
                try:f=self._proc(f)
                except Exception as e:logger.error(f"Proc: {e}")
            self._q2.put(f)
    def _t_enc(self):
        while True:
            f=self._q2.get()
            if f is None:break
            self._enc.write_frame(f)
    @staticmethod
    def motion_energy(prev,cur):
        g1=cv2.cvtColor(prev,cv2.COLOR_BGR2GRAY)
        g2=cv2.cvtColor(cur,cv2.COLOR_BGR2GRAY)
        diff=cv2.absdiff(g1,g2)
        return float(np.mean(diff))/255.0


────────────────────────────────────────────────────────────
FILE: pipeline\scene.py
SIZE: 0.8 KB
────────────────────────────────────────────────────────────
"""scene.py — Scene detection."""
from pipeline.core import config,logger,IScene

class SceneAn(IScene):
    def find_scenes(self,path,var=None):
        from scenedetect import open_video,SceneManager
        from scenedetect.detectors import AdaptiveDetector
        v=var or config.scene_var
        video=open_video(path)
        sm=SceneManager()
        sm.add_detector(AdaptiveDetector(adaptive_threshold=v))
        sm.detect_scenes(video)
        sl=sm.get_scene_list()
        scenes=[]
        for s,e in sl:
            ss=s.get_seconds();es=e.get_seconds()
            scenes.append({"start_sec":ss,"end_sec":es,"duration":es-ss})
        scenes.sort(key=lambda x:x["duration"],reverse=True)
        mx=config.max_segments
        scenes=scenes[:mx]
        logger.info(f"Scenes: {len(scenes)} found")
        return scenes,sl


────────────────────────────────────────────────────────────
FILE: pipeline\tts.py
SIZE: 2.4 KB
────────────────────────────────────────────────────────────
"""tts.py — Supertonic 3 TTS (ONNX, on-device, 31 languages)."""
import os
from pipeline.core import config, logger, breakers

class TTSEngine:
    def __init__(self):
        self.available = False
        self._tts = None
        self._style = None
        try:
            from supertonic import TTS
            self._tts = TTS(auto_download=True)
            voice = config.tts_voice if config.tts_voice else "M1"
            if config.tts_voice_ref and os.path.exists(config.tts_voice_ref):
                self._style = self._tts.get_voice_style_from_path(config.tts_voice_ref)
            else:
                self._style = self._tts.get_voice_style(voice_name=voice)
            self.available = True
            logger.info("TTS: Supertonic 3 ready (voice=%s, lang=%s)", voice, config.tts_lang)
        except ImportError:
            logger.warning("TTS: supertonic not installed (pip install supertonic)")
        except Exception as e:
            logger.warning("TTS init failed: %s", e)

    def synthesize(self, text, output_path, lang=None):
        if not self.available:
            logger.warning("TTS not available")
            return None
        cb = breakers.get("tts")
        if not cb.can_exec():
            logger.warning("TTS circuit breaker OPEN")
            return None
        try:
            wav, duration = self._tts.synthesize(
                text=text,
                voice_style=self._style,
                lang=lang or config.tts_lang,
                total_steps=config.tts_steps,
                speed=config.tts_speed,
            )
            self._tts.save_audio(wav, output_path)
            dur_val = float(duration[0]) if hasattr(duration, "__len__") else float(duration)
            logger.info("TTS: generated %.2fs -> %s", dur_val, output_path)
            cb.ok()
            return output_path
        except Exception as e:
            logger.error("TTS synthesis failed: %s", e)
            cb.fail()
            return None

    def list_voices(self):
        return ["M1", "M2", "M3", "M4", "M5", "F1", "F2", "F3", "F4", "F5"]

    def set_voice(self, voice_name):
        if not self._tts:
            return False
        try:
            self._style = self._tts.get_voice_style(voice_name=voice_name)
            logger.info("TTS: voice changed to %s", voice_name)
            return True
        except Exception as e:
            logger.error("TTS set_voice failed: %s", e)
            return False


────────────────────────────────────────────────────────────
FILE: pipeline\upscaler.py
SIZE: 3.5 KB
────────────────────────────────────────────────────────────
"""upscaler.py — Real-ESRGAN Vulkan GPU, progress + OS-level fd suppress."""
import os, sys, time
import cv2
from pipeline.core import config, logger


def _suppress():
    saved_out = os.dup(1)
    saved_err = os.dup(2)
    devnull = os.open(os.devnull, os.O_WRONLY)
    os.dup2(devnull, 1)
    os.dup2(devnull, 2)
    os.close(devnull)
    return saved_out, saved_err


def _restore(saved):
    saved_out, saved_err = saved
    os.dup2(saved_out, 1)
    os.dup2(saved_err, 2)
    os.close(saved_out)
    os.close(saved_err)


class Upsc:
    """Real-ESRGAN ncnn Vulkan GPU upscaler.

    Bu GERCEK AI upscale:
      - realesr-animevideov3 derin sinir agi modeli kullanir
      - Orijinalde OLMAYAN detaylari olusturur (hatlar, dokular, kenarlar)
      - Bicubic/lanczos gibi basit interpolasyon DEGILDIR
      - Ornek: 640x360 -> 1280x720 (2x), 640x360 -> 1920x1080 (3x)
    """

    def __init__(self):
        self.available = False
        self._up = None
        self._count = 0
        self._total = 0
        self._t0 = time.time()
        self._scale = getattr(config, "upscale_factor", 2)
        model_id = {2: 0, 3: 1, 4: 2}.get(self._scale, 0)
        try:
            from realesrgan_ncnn_py import Realesrgan
            saved = _suppress()
            try:
                self._up = Realesrgan(gpuid=0, model=model_id, tilesize=640)
            finally:
                _restore(saved)
            self.available = True
            logger.info("ESRGAN Vulkan GPU ready (model=%d, scale=%dx)", model_id, self._scale)
        except ImportError:
            logger.warning("ESRGAN: pip install realesrgan-ncnn-py")
        except Exception as e:
            logger.warning("ESRGAN init: %s", e)

    def reset(self, total=0):
        self._count = 0
        self._total = total
        self._t0 = time.time()

    def _format_eta(self, seconds):
        if seconds < 60:
            return "%ds" % seconds
        elif seconds < 3600:
            m = int(seconds // 60)
            s = int(seconds % 60)
            return "%dm %ds" % (m, s)
        else:
            h = int(seconds // 3600)
            m = int((seconds % 3600) // 60)
            return "%dh %dm" % (h, m)

    def enhance(self, frame):
        if not self._up:
            return frame
        try:
            saved = _suppress()
            try:
                result = self._up.process_cv2(frame)
            finally:
                _restore(saved)
            self._count += 1
            if self._count % 5 == 0:
                elapsed = time.time() - self._t0
                fps = self._count / max(elapsed, 0.01)
                if self._total > 0:
                    pct = self._count / self._total * 100
                    remain = (self._total - self._count) / max(fps, 0.01)
                    eta = self._format_eta(remain)
                    h, w = frame.shape[:2]
                    oh = h * self._scale
                    ow = w * self._scale
                    msg = "\r  Upscale: %d/%d (%.1f%%) | %.1f fps | %dx%d->%dx%d | ETA: %s     " % (
                        self._count, self._total, pct, fps, w, h, ow, oh, eta)
                else:
                    msg = "\r  Upscale: %d frames | %.1f fps     " % (self._count, fps)
                sys.stderr.write(msg)
                sys.stderr.flush()
            if result is not None:
                return result
        except Exception as e:
            logger.error("ESRGAN: %s", e)
        return frame


────────────────────────────────────────────────────────────
FILE: pipeline\utilities.py
SIZE: 2.2 KB
────────────────────────────────────────────────────────────
"""utilities.py — Preflight, health, model setup."""
import os, shutil
from pipeline.core import config, logger, gpu_backend
import onnxruntime as ort

def preflight():
    for d in [config.dir_video, config.dir_audio, config.dir_temp, config.dir_output, config.dir_tools]:
        os.makedirs(str(d), exist_ok=True)
    if not os.path.isfile(config.ffmpeg):
        logger.error("FFmpeg not found: %s", config.ffmpeg)
    else:
        logger.info("FFmpeg: %s", config.ffmpeg)
    logger.info("Preflight OK")

def setup_models():
    if os.path.isfile(config.yolo_path):
        sz = os.path.getsize(config.yolo_path) / 1e6
        logger.info("YOLO: %s (%.1f MB)", config.yolo_path, sz)
    else:
        logger.warning("YOLO model missing: %s", config.yolo_path)
    if os.path.isfile(config.esrgan_path):
        sz = os.path.getsize(config.esrgan_path) / 1e6
        logger.info("ESRGAN: %s (%.1f MB)", config.esrgan_path, sz)
    else:
        logger.warning("ESRGAN weights missing: %s", config.esrgan_path)

def health_check():
    checks = []
    checks.append(("Python", True, __import__("sys").version.split()[0]))
    checks.append(("FFmpeg", os.path.isfile(config.ffmpeg) or shutil.which("ffmpeg") is not None, config.ffmpeg))
    checks.append(("YOLO", os.path.isfile(config.yolo_path), config.yolo_path))
    checks.append(("ESRGAN", os.path.isfile(config.esrgan_path), config.esrgan_path))
    provs = ort.get_available_providers()
    dml = "DmlExecutionProvider" in provs
    checks.append(("DirectML", dml, str(provs)))
    rife_ok = os.path.isfile(config.rife_bin)
    checks.append(("RIFE", rife_ok, config.rife_bin))
    tts_ok = False
    try:
        import supertonic
        tts_ok = True
    except ImportError:
        pass
    checks.append(("Supertonic TTS", tts_ok, "pip install supertonic" if not tts_ok else "OK"))
    ollama_ok = False
    try:
        import requests
        r = requests.get(config.ollama_url, timeout=3)
        ollama_ok = r.status_code == 200
    except Exception:
        pass
    checks.append(("Ollama", ollama_ok, config.ollama_url))
    for name, ok, detail in checks:
        tag = "[OK]" if ok else "[!!]"
        print(f"  {tag} {name}: {detail}")
    return checks


────────────────────────────────────────────────────────────
FILE: pipeline\voice_extractor.py
SIZE: 8.7 KB
────────────────────────────────────────────────────────────
"""voice_extractor.py — Voice extraction + resemblyzer embedding matching."""
import os, subprocess, shutil, glob
import numpy as np
from pipeline.core import config, logger, run_sub


class VoiceExtractor:
    def __init__(self):
        self.voices_dir = str(getattr(config, "voices_dir", "./voices"))
        os.makedirs(self.voices_dir, exist_ok=True)
        self._encoder = None
        self._init_encoder()

    def _init_encoder(self):
        try:
            from resemblyzer import VoiceEncoder
            self._encoder = VoiceEncoder("cpu")
            logger.info("VoiceExtractor: resemblyzer OK")
        except ImportError:
            logger.warning("VoiceExtractor: resemblyzer not installed (pip install resemblyzer)")
        except Exception as e:
            logger.warning("VoiceExtractor: encoder init failed: %s", e)

    @property
    def available(self):
        return True

    @property
    def can_match(self):
        return self._encoder is not None

    def _safe_name(self, name):
        return "".join(c if c.isalnum() or c in "_-" else "_" for c in name.lower().strip())

    def _wav_path(self, name):
        return os.path.join(self.voices_dir, self._safe_name(name) + ".wav")

    def _emb_path(self, name):
        return os.path.join(self.voices_dir, self._safe_name(name) + ".npy")

    def has_voice(self, character_name):
        return os.path.isfile(self._wav_path(character_name))

    def get_voice_path(self, character_name):
        p = self._wav_path(character_name)
        return p if os.path.isfile(p) else None

    # ── Embedding ops ──

    def get_embedding(self, wav_path):
        if not self._encoder:
            return None
        try:
            from resemblyzer import preprocess_wav
            wav = preprocess_wav(wav_path)
            if len(wav) < 1600:
                logger.warning("Audio too short for embedding: %s", wav_path)
                return None
            emb = self._encoder.embed_utterance(wav)
            return emb
        except Exception as e:
            logger.error("Embedding extraction failed: %s", e)
            return None

    def save_embedding(self, character_name, embedding):
        path = self._emb_path(character_name)
        np.save(path, embedding)
        logger.info("Embedding saved: %s", path)

    def load_embedding(self, character_name):
        path = self._emb_path(character_name)
        if os.path.isfile(path):
            return np.load(path)
        return None

    def _list_embeddings(self):
        result = {}
        for f in os.listdir(self.voices_dir):
            if f.endswith(".npy"):
                name = f[:-4]
                emb = np.load(os.path.join(self.voices_dir, f))
                result[name] = emb
        return result

    def match_voice(self, wav_path, threshold=0.75):
        if not self._encoder:
            return None
        emb = self.get_embedding(wav_path)
        if emb is None:
            return None
        library = self._list_embeddings()
        if not library:
            return None
        best_name = None
        best_sim = 0.0
        for name, stored_emb in library.items():
            sim = float(np.dot(emb, stored_emb) / (np.linalg.norm(emb) * np.linalg.norm(stored_emb) + 1e-8))
            if sim > best_sim:
                best_sim = sim
                best_name = name
        if best_sim >= threshold and best_name:
            logger.info("Voice match: %s (sim=%.3f)", best_name, best_sim)
            return {"character": best_name, "similarity": best_sim}
        logger.info("No voice match above threshold (best: %s %.3f)", best_name, best_sim)
        return None

    # ── Audio extraction ──

    def _cut_audio(self, video_path, start_sec, end_sec, output_wav):
        duration = end_sec - start_sec
        if duration < 2.0:
            duration = 5.0
        if duration > 30.0:
            duration = 30.0
        cmd = [
            config.ffmpeg, "-y",
            "-ss", str(start_sec),
            "-t", str(duration),
            "-i", video_path,
            "-vn", "-acodec", "pcm_s16le",
            "-ar", "16000", "-ac", "1",
            output_wav
        ]
        try:
            run_sub(cmd, "voice-cut", ignore=True)
            return os.path.isfile(output_wav)
        except Exception as e:
            logger.error("FFmpeg voice cut failed: %s", e)
            return False

    def _isolate_vocals(self, raw_wav, tmp_dir):
        try:
            cmd = [
                "demucs", "--two-stems", "vocals",
                "-n", "htdemucs",
                "--out", tmp_dir,
                raw_wav
            ]
            run_sub(cmd, "demucs", ignore=True)
        except Exception as e:
            logger.warning("Demucs failed: %s", e)
            return None
        patterns = [
            os.path.join(tmp_dir, "htdemucs", "**", "vocals.wav"),
            os.path.join(tmp_dir, "**", "vocals.wav"),
        ]
        for pat in patterns:
            matches = glob.glob(pat, recursive=True)
            if matches:
                return matches[0]
        return None

    def extract_voice(self, video_path, start_sec, end_sec, character_name):
        safe = self._safe_name(character_name)
        out_wav = self._wav_path(character_name)
        if os.path.isfile(out_wav) and os.path.getsize(out_wav) > 10000:
            logger.info("Voice already exists: %s", out_wav)
            return out_wav
        tmp_dir = os.path.join(str(config.dir_temp), "voice_extract")
        os.makedirs(tmp_dir, exist_ok=True)
        raw_wav = os.path.join(tmp_dir, safe + "_raw.wav")
        if not self._cut_audio(video_path, start_sec, end_sec, raw_wav):
            return None
        vocal = self._isolate_vocals(raw_wav, tmp_dir)
        if vocal and os.path.isfile(vocal):
            shutil.copy2(vocal, out_wav)
            logger.info("Voice extracted (demucs): %s", out_wav)
        else:
            shutil.copy2(raw_wav, out_wav)
            logger.warning("Demucs unavailable, using raw audio: %s", out_wav)
        emb = self.get_embedding(out_wav)
        if emb is not None:
            self.save_embedding(character_name, emb)
            logger.info("Voice + embedding saved: %s", character_name)
        return out_wav

    # ── Audio-Visual Verification ──

    def verify_speaker(self, video_path, start_sec, end_sec, visual_guess="unknown"):
        tmp_dir = os.path.join(str(config.dir_temp), "voice_verify")
        os.makedirs(tmp_dir, exist_ok=True)
        tmp_wav = os.path.join(tmp_dir, "verify_segment.wav")
        if not self._cut_audio(video_path, start_sec, end_sec, tmp_wav):
            return {"character": visual_guess, "method": "visual_only", "confidence": 0.5}
        vocal = self._isolate_vocals(tmp_wav, tmp_dir)
        check_wav = vocal if vocal and os.path.isfile(vocal) else tmp_wav
        audio_match = self.match_voice(check_wav)
        if audio_match:
            audio_char = audio_match["character"]
            audio_sim = audio_match["similarity"]
            if visual_guess != "unknown" and self._safe_name(visual_guess) == audio_char:
                confidence = min(1.0, (audio_sim + 0.85) / 2)
                logger.info("CONFIRMED (audio+visual): %s (conf=%.2f)", visual_guess, confidence)
                return {"character": visual_guess, "method": "audio+visual", "confidence": confidence}
            elif audio_sim > 0.85:
                logger.info("AUDIO OVERRIDE: visual=%s, audio=%s (sim=%.3f)", visual_guess, audio_char, audio_sim)
                return {"character": audio_char, "method": "audio_override", "confidence": audio_sim}
            else:
                logger.info("CONFLICT: visual=%s, audio=%s (sim=%.3f) — using visual", visual_guess, audio_char, audio_sim)
                return {"character": visual_guess, "method": "visual_preferred", "confidence": 0.6}
        else:
            if visual_guess != "unknown":
                logger.info("NEW VOICE: %s (no audio match, saving embedding)", visual_guess)
                emb = self.get_embedding(check_wav)
                if emb is not None:
                    self.save_embedding(visual_guess, emb)
                return {"character": visual_guess, "method": "visual_new_voice", "confidence": 0.7}
            return {"character": "unknown", "method": "none", "confidence": 0.0}

    def list_voices(self):
        if not os.path.isdir(self.voices_dir):
            return []
        voices = []
        for f in os.listdir(self.voices_dir):
            if f.endswith(".wav"):
                name = f[:-4]
                wav_size = os.path.getsize(os.path.join(self.voices_dir, f)) / 1024
                has_emb = os.path.isfile(os.path.join(self.voices_dir, name + ".npy"))
                voices.append({"name": name, "size_kb": wav_size, "has_embedding": has_emb})
        return voices


────────────────────────────────────────────────────────────
FILE: temp_processing\concat.txt
SIZE: 1.0 KB
────────────────────────────────────────────────────────────
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_0.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_1.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_2.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_3.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_4.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_5.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_6.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_7.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_8.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_9.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_10.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_11.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_12.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_13.mp4'
file 'E:\\DENEYSEL\\pipeline_project\\temp_processing\\seg_14.mp4'