KINTSUGI Troubleshooting Guide

This guide covers common issues and their solutions when installing or running KINTSUGI.

Table of Contents

Installation Issues

Conda Environment Creation Fails or Hangs

Symptom: conda env create sits at Solving environment for 15+ minutes, stops with no output, or fails with dependency conflicts.

Solutions (check in order):

  1. Remove the defaults channel (Miniconda and Anaconda ship channels: [defaults]). conda merges .condarc channels into every env-file solve; the KINTSUGI env files list nodefaults to block that, but the Anaconda Terms-of-Service plugin (conda >= 25.x) still gates conda env create on the configured channels. In a terminal it prompts; without a terminal (scripts, IDE tasks, AI coding assistants) it raises CondaToSNonInteractiveError. A conda-forge + defaults solve of this environment does not finish in 25 minutes.

    conda config --show channels                 # is 'defaults' listed?
conda config --remove channels defaults
conda config --add channels conda-forge
conda config --set channel_priority strict
# if 'defaults' comes from the install's own .condarc:
conda config --file "$(conda info --base)/.condarc" --remove channels defaults

    Or install Miniforge, which is conda-forge-only.

  2. Set strict channel priority (Miniforge’s default; Miniconda uses flexible):

    conda config --set channel_priority strict

    With flexible, conda 26.7 + libmamba solved the pinned env-linux.yml in 79 s once and was still solving a repeat run after 18 minutes; with strict the same solve took 93 s and 48 s in two runs. The unpinned file never finished within 32 minutes.

  3. Use the libmamba solver (default since conda 23.10):

    conda install -n base conda-libmamba-solver
conda config --set solver libmamba
conda env create -f envs/env-linux.yml

  4. Skip the solver with the lock file (Linux x86_64). envs/env-linux.lock.txt is an explicit, md5-pinned list generated from envs/env-linux.yml; conda installs it without solving (about 90 s), then add the pip-only packages:

    conda create -n KINTSUGI --file envs/env-linux.lock.txt
conda activate KINTSUGI
pip install -e ".[workflow]"

  5. Update conda and the env file:

    conda update -n base conda
git pull        # envs/*.yml carry solver pins and nodefaults

    micromamba 2.9 (the solver core behind Miniforge’s mamba env create) solved both the old and the new env-linux.yml in under 75 s in every run.

  6. Create a minimal environment and add packages:

    conda create -n KINTSUGI -c conda-forge --override-channels python=3.11 pip libvips
conda activate KINTSUGI
pip install -e ".[workflow]"
# Install additional dependencies as needed

Package Installation Errors

Symptom: pip install -e . fails.

Solutions:

  1. Ensure setuptools is updated:

    pip install --upgrade pip setuptools wheel

  2. Check Python version:

    python --version  # Should be 3.10+

  3. Install with verbose output:

    pip install -e . -v

Dependency Issues

libvips Not Found

Symptom: Error message like cannot find library libvips or pyvips.GObject.Error.

Windows

  1. Download from Zenodo:

    • Download PyVips-dev-8.16 from Zenodo

    • Extract to KINTSUGI folder

  2. Set PATH manually:

    $env:PATH = "C:\Users\[username]\KINTSUGI\vips-dev-8.16\bin;$env:PATH"

  3. Add to system PATH permanently:

    • Open System Properties > Environment Variables

    • Add C:\Users\[username]\KINTSUGI\vips-dev-8.16\bin to PATH

Linux

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y libvips-dev

# Fedora/RHEL
sudo dnf install vips-devel

# Verify installation
vips --version

macOS

brew install vips

Verify libvips Installation

import pyvips
print(f"libvips version: {pyvips.version(0)}.{pyvips.version(1)}.{pyvips.version(2)}")

Java/JVM Issues (Legacy)

Note: Java, Maven, and FIJI/CLIJ2 are no longer required. KINTSUGI now uses pure Python implementations for all processing. This section is kept for users of older versions.

If an older notebook, environment file or error message still mentions jpype, scyjava, JAVA_HOME or Maven, you are running code from a pre-1.0 release: update to the current version and recreate the environment from envs/. No current KINTSUGI component installs or starts a JVM.

CUDA/GPU Issues

Symptom: torch.cuda.is_available() returns False.

Verify GPU Detection

import torch
print(f"CUDA available: {torch.cuda.is_available()}")
print(f"CUDA version: {torch.version.cuda}")
if torch.cuda.is_available():
    print(f"GPU: {torch.cuda.get_device_name(0)}")

Solutions

  1. Check NVIDIA driver:

    nvidia-smi

  2. Reinstall PyTorch with CUDA:

    pip uninstall torch torchvision
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

  3. Verify CUDA toolkit:

    nvcc --version

  4. Use CPU fallback:

    • Most KINTSUGI operations work without GPU

    • Set device manually: device = "cpu"

Runtime Issues

Import Errors

Symptom: ModuleNotFoundError or ImportError.

General Solutions

  1. Verify environment is activated:

    conda activate KINTSUGI
which python  # Should show conda environment path

  2. Run dependency check:

    kintsugi check

  3. Reinstall package:

    pip install -e . --force-reinstall

Specific Import Errors

ImportError: cannot import name 'Valis':

# Correct import path
from Kreg.registration import Valis
# or via kintsugi package
from kintsugi.kreg import Valis

ModuleNotFoundError: No module named 'Kreg':

import sys
sys.path.insert(0, '/path/to/KINTSUGI/notebooks')

Memory Issues

Symptom: MemoryError or process killed during processing.

Solutions

  1. Reduce image dimensions:

    config = {
    "max_image_dim_px": 2048,  # Reduce from default
    "max_processed_image_dim_px": 1024,
}

  2. Process images in tiles:

    • Use tiled processing for large images

    • Reduce batch size

  3. Monitor memory usage:

    import psutil
print(f"Memory: {psutil.virtual_memory().percent}%")

  4. Clear pyvips cache:

    import pyvips
pyvips.cache_set_max(0)

  5. Use zarr for large datasets:

    import zarr
# Process in chunks

Stitched Image Quality Issues

Symptom: Stitched images appear blank/white (saturated) or show visible tile grid patterns.

Diagnosing the Problem

from skimage.io import imread
import numpy as np

img = imread('path/to/stitched.tif')
mean_val = img.mean()
pct_max = 100.0 * np.sum(img > 64000) / img.size
std_mean_ratio = img.std() / img.mean()

print(f"Mean: {mean_val:.0f}")
print(f"Pct at max: {pct_max:.1f}%")
print(f"Std/Mean ratio: {std_mean_ratio:.2f}")

# Saturation: pct_max > 50%
# Tile grid: std_mean_ratio > 0.8

Saturation Issues (Blank White Images)

Cause: BaSiC illumination correction amplifies signal excessively when flatfield values are too low.

Solutions:

  1. Ensure flatfield minimum is enforced:

    BASIC_FLATFIELD_MIN = 0.1
flatfield_safe = np.clip(flatfield, BASIC_FLATFIELD_MIN, None)
corrected = (tiles_norm - darkfield) / flatfield_safe

  2. Verify input normalization:

    # Input must be normalized to [0,1] before BaSiC correction
tiles_norm = tiles.astype(np.float64) / 65535

  3. Reprocess affected z-planes:

    python notebooks/reprocess_problematic_images.py --dry-run
python notebooks/reprocess_problematic_images.py

Tile Grid Patterns (Visible Seams)

Cause: Insufficient blending at tile overlap boundaries or stitch model mismatch.

Solutions:

  1. Increase blend sigma:

    from kintsugi.stitch_blend import stitch_with_blending_gpu
stitched = stitch_with_blending_gpu(
    tiles, result_df,
    sigma=10.0,  # Increase from default
    overlap_fraction=(0.3, 0.3)
)

  2. Verify stitch model compatibility:

    • Stitch model computed from one z-plane may not work for all z-planes

    • Use consistent reference z-plane across channels

  3. Check overlap settings:

    • Ensure overlap_fraction matches actual tile overlap

    • Typical values: (0.2, 0.2) to (0.3, 0.3)

Batch Reprocessing

For multiple problematic images, use the reprocessing script:

# First scan for problems
python -c "
from Kio import scan_stitched_quality
issues = scan_stitched_quality('/path/to/stitched')
print(f'Saturated: {len(issues[\"saturated\"])}')
print(f'Tile grid: {len(issues[\"tile_grid\"])}')
"

# Then reprocess
python notebooks/reprocess_problematic_images.py

Registration Failures

Symptom: Registration produces poor results or fails.

Check Input Images

  1. Verify image format:

    import tifffile
img = tifffile.imread('image.tif')
print(f"Shape: {img.shape}, dtype: {img.dtype}")

  2. Check image quality:

    • Ensure images are not corrupted

    • Verify adequate contrast and features

Adjust Parameters

  1. Try different registrars:

    config = {
    "micro_rigid_registrar_cls": "RigidRegistrar",  # or "AffineRegistrar"
}

  2. Adjust image dimensions:

    config = {
    "max_image_dim_px": 4096,  # Increase for more detail
}

  3. Disable non-rigid registration:

    config = {
    "compose_non_rigid": False,
}

HPC/SLURM Issues

“snakemake: command not found”

conda activate KINTSUGI
pip install "snakemake>=8.0" snakemake-executor-plugin-slurm

“No workflow/config.yaml found”

Run kintsugi workflow config . from your project directory first.

CuPy Appears Unavailable

On login nodes (no GPU hardware), import cupy may succeed but GPU operations fail with cudaErrorInsufficientDriver. Functions like gpu.cupy_available test GPU hardware availability, not package installation. CuPy IS installed — do not attempt to reinstall it.

To check if CuPy is actually installed (import-only, no GPU needed):

from kintsugi.gpu import get_gpu_manager
gpu = get_gpu_manager()
print(f"CuPy installed: {gpu.cupy_installed}")   # True if package is importable
print(f"CuPy available: {gpu.cupy_available}")    # True only if GPU hardware present

Jobs Pending Indefinitely (QOS Limits)

Check your account allocations:

sacctmgr show associations user=$(whoami) format=account,partition,qos,grptres -n -P

Reduce -j when running kintsugi workflow run or set jobs: lower in profiles/slurm/config.yaml.

SLURM Jobs Fail with OOM (Out of Memory)

GPU jobs use CuPy (float32 in GPU memory), needing ~48 GB CPU RAM. CPU jobs use SciPy (float64 in system memory), needing ~128 GB. Adjust memory in workflow/config.yaml:

resources:
  mem_stitch: 48000     # MB, GPU jobs
  mem_decon: 48000
  cpu_mem_decon: 128000  # MB, CPU jobs

“Missing output files after job completion” (NFS Latency)

Increase latency wait in workflow/profiles/slurm/config.yaml:

latency-wait: 300  # Increased from 120 to 300 seconds

“CUDA initialization failed” in Job Logs

This is expected on CPU-only nodes. Scripts automatically fall back to CPU mode via KINTSUGI_DEVICE_MODE. To ensure jobs land on GPU nodes, verify the partition setting in workflow/config.yaml points to a GPU partition (e.g., hpg-b200, hpg-turin).

“srun: fatal: SLURM_TRES_PER_TASK is mutually exclusive”

SLURM >= 24.11 sets SLURM_TRES_PER_TASK in GPU job environments, which conflicts with the Snakemake jobstep plugin’s srun call. Fix by patching the jobstep plugin:

# In snakemake_executor_plugin_slurm_jobstep/__init__.py, add to __post_init__():
import os
os.environ.pop("SLURM_TRES_PER_TASK", None)

This patch must be re-applied after any pip upgrade of snakemake-executor-plugin-slurm-jobstep.

Stitch Model Not Found for CH2+

Channel 1 computes the stitching model used by all other channels. If CH1 fails, subsequent channels fail with “No stitch model.” Check the CH1 stitching log first:

tail /path/to/project/slurm/logs/snakemake/stitch_cyc01.log

MCP Server (AI assistants)

The optional KINTSUGI MCP server is started by your MCP client from the configuration kintsugi mcp config --client NAME (or kintsugi init --mcp-client NAME) wrote or printed. Each client shows the server’s state in its own MCP view (in Claude Code and Gemini CLI, /mcp); the client table lists where. Full guide: MCP server.

“Executable not found in $PATH: kintsugi”

The client could not find the kintsugi command — typically because its entry uses a bare kintsugi and the client was launched outside the conda/venv environment. Regenerate the entry so it carries the absolute path of the executable:

conda activate KINTSUGI             # the environment where kintsugi is installed
kintsugi mcp config /path/to/project --client NAME --force
# the entry's "command" is now e.g. /home/you/envs/KINTSUGI/bin/kintsugi

The generated entry is {"type": "stdio", "command": "<absolute path>", "args": ["mcp", "start", "--project-dir", "<project>"]}. Then restart or reconnect the server in your MCP client.

“No project configured”

The server was started without --project-dir, without KINTSUGI_PROJECT_DIR, and in a working directory that is not a KINTSUGI project, so it cannot guess which project a tool means. Regenerate the client configuration (kintsugi mcp config /path/to/project --client NAME --force, which adds --project-dir), or pass project_path in the tool call.

Server shows “Pending approval”

Most clients start a project’s servers only in a trusted folder or workspace and may ask you to enable the server; the last column of the client table says what each one needs. For Claude Code, the approval kintsugi mcp config --client claude-code writes is .claude/settings.local.json → {"enabledMcpjsonServers": ["kintsugi"]}; if an earlier decision rejected the server, clear it with claude mcp reset-project-choices and reopen the project.

“mcp 1.x is installed but KINTSUGI requires mcp>=2.0,<3”

The server targets the 2.x SDK (mcp.server.mcpserver.MCPServer); the 1.x mcp.server.fastmcp API is not supported. Upgrade in the environment the server starts from:

pip install "mcp>=2.0,<3"           # or: pip install -e ".[mcp]"  /  kintsugi install mcp
kintsugi check                      # should list the MCP server as available

kintsugi mcp start prints the same message to stderr (stdout is reserved for JSON-RPC). Never run pip install kintsugi[mcp] — it installs an unrelated PyPI package.

Platform-Specific Issues

Windows

Long Path Issues

Symptom: FileNotFoundError with long paths.

Solution:

  1. Enable long paths in Windows:

    • Run gpedit.msc

    • Navigate to: Computer Configuration > Administrative Templates > System > Filesystem

    • Enable “Enable Win32 long paths”

  2. Or move KINTSUGI to shorter path (e.g., C:\KINTSUGI)

DLL Load Failures

Symptom: ImportError: DLL load failed.

Solutions:

  1. Install Visual C++ Redistributable

  2. Verify all Zenodo dependencies are extracted

  3. Add paths to system PATH

Linux

Permission Issues

Symptom: Permission denied errors.

Solutions:

# Fix permissions
chmod +x scripts/install.sh
chmod -R u+rw KINTSUGI/

# Don't run as root
# If needed, use: sudo chown -R $USER:$USER KINTSUGI/

Display Issues (Napari)

Symptom: Napari doesn’t display.

Solutions:

# Check display
echo $DISPLAY

# For headless servers, use Xvfb
Xvfb :99 -screen 0 1024x768x24 &
export DISPLAY=:99

macOS

Apple Silicon (M1/M2)

Symptom: Package incompatibility on ARM.

Solutions:

  1. Use Rosetta 2 for x86 compatibility:

    arch -x86_64 conda create -n KINTSUGI python=3.10

  2. Use native ARM packages where available:

    conda config --add channels apple

Gatekeeper Blocks

Symptom: “Cannot be opened because the developer cannot be verified”.

Solution:

xattr -d com.apple.quarantine /path/to/file

Getting Help

If you’re still experiencing issues:

  1. Run full diagnostics:

    kintsugi check --verbose > diagnostics.txt
python -c "import sys; print(sys.version)"
conda list > packages.txt

  2. Check GitHub Issues: https://github.com/smith6jt-cop/KINTSUGI/issues

  3. Create a new issue with:

    • Operating system and version

    • Python version

    • Error message (full traceback)

    • Steps to reproduce

    • Output of kintsugi check

Quick Reference

Issue

Quick Fix

libvips not found

Windows: Download from Zenodo; Linux: apt install libvips-dev

GPU not detected

Check nvidia-smi; kintsugi install gpu

CuPy unavailable on login node

Normal — no GPU on login nodes; test on compute node

Import errors

conda activate KINTSUGI && pip install -e .

Memory errors

Reduce max_image_dim_px in config

Blank white stitched images

Check BaSiC flatfield min (0.1); run reprocess_problematic_images.py

Tile grid in stitched images

Increase blend sigma (10.0); verify stitch model compatibility

Snakemake not found

pip install "snakemake>=8.0" snakemake-executor-plugin-slurm

SLURM OOM kill

GPU: 48 GB RAM; CPU: 128 GB RAM — increase in config.yaml

SLURM_TRES_PER_TASK error

Patch jobstep plugin (see HPC section above)

Executable not found in $PATH: kintsugi (MCP)

kintsugi mcp config /path/to/project --client NAME --force from the activated env

MCP server “Pending approval”

Trust the folder and enable kintsugi in your client (see the client table)

mcp 1.x is installed (MCP)

pip install "mcp>=2.0,<3"