KINTSUGI

Getting Started

  • Installation
    • Quick Start (All Platforms)
    • Windows Installation
      • Option A: Using Installation Script (Recommended)
      • Option B: Manual Installation
      • Download Windows Dependencies
    • Linux Installation
      • Option A: Using Installation Script (Recommended)
      • Option B: Manual Installation
    • macOS Installation
    • GPU Acceleration (Optional)
      • Basic GPU Support (CuPy + PyTorch)
      • Multi-GPU Support
      • GPU-Accelerated Image Processing
      • GPU-Accelerated Single-Cell Analysis
    • AI assistants (MCP server, optional)
    • External Dependencies
    • HPC Installation (SLURM Clusters)
      • Quick Start (Recommended)
      • Manual HPC Installation
      • Fixing a Broken HPC Environment
      • env-hpc.yml vs env-linux.yml
      • HPC-Specific Notes
    • Updating Existing Projects
    • Verifying Installation
  • Quick Start Guide
    • Verify Installation
    • Create a Project
      • Channel Names
    • Command Line Interface
    • Python API
    • AI Assistants (optional)
      • Setup
      • Usage
    • Jupyter Notebooks
      • 1. Parameter Tuning and Testing
      • 2. Batch Processing
      • 3. Signal Isolation & Quality Control (NEW)
      • 4. Segmentation Analysis
      • 2.5. 3D Vessel Segmentation
      • Running Notebooks
    • Data Organization
    • HPC Quick Start (SLURM)
    • Next Steps

User Guide

  • Processing Workflows
    • Overview
    • Workflow 1: Single Channel Evaluation
      • Steps:
    • Workflow 2: Cycle Processing
      • Steps:
      • QC Plot Output
      • QC Module
    • Workflow 3: Signal Isolation
      • Option A: AI-Assisted Workflow (optional MCP server)
      • Option B: Python API
      • Option C: Interactive Tuners (Notebook)
      • Steps:
    • Workflow 4: Segmentation Analysis
      • Steps:
    • Registration Module (Kreg)
      • Basic Usage
      • Configuration Options
    • Visualization Module (Kview2)
      • Basic Usage
    • Stitching Module (Kstitch)
      • Basic Usage
    • Workflow 5: Quality Control
      • Image-Level QC
      • Cell-Level QC
      • Marker Validation
      • Batch Effects
    • Denoising Module
      • Traditional Filters
      • Deep Learning Denoising
      • Adaptive Denoising
    • Segmentation Module
      • Classical Segmentation
      • SAM Segmentation
      • Post-Processing
    • HPC/SLURM Batch Processing (Snakemake)
      • Processing Stages
      • Setup and Execution
      • Multi-Account Architecture
      • Skip-Existing and Recovery
      • Output Structure
    • Tips for Large Datasets
  • MCP Server
    • What is different about this server
    • Sixty-second setup
    • Connect your client
      • Any other client
    • Architecture
    • Tools
      • Resources and prompt
      • Progress and images
    • The learning loop
    • Workflow state
    • Security model
      • Why the tools are not prefixed kintsugi_
    • Evaluation
    • A recorded session
    • Compatibility
    • Troubleshooting
  • Signal Isolation
    • What is Autofluorescence?
    • Methods
      • Global Subtraction
      • Weighted Multi-Range Subtraction
    • Usage
      • AI-Assisted (MCP, optional)
      • Python API
        • AutofluorescenceSubtractor class
        • Direct functions
        • Batch processing
      • Batch Bootstrap (CLI)
    • Parameter Reference
      • Global Subtraction Parameters
      • Weighted Subtraction Parameters
      • Quality Metrics
    • Parameter Learning
      • How it works
      • Database location
      • Checking learning statistics
      • Backwards compatibility
  • Command Line Interface
    • Commands
      • kintsugi check
      • kintsugi info
      • kintsugi template
      • kintsugi register
      • kintsugi scan
      • kintsugi init
      • kintsugi install
    • MCP Server Commands
      • kintsugi mcp start
      • kintsugi mcp tools
      • kintsugi mcp config
      • kintsugi mcp clients
      • kintsugi mcp status
      • kintsugi mcp pretrain
    • Configuration File Format
    • Workflow Commands (HPC/Snakemake)
      • kintsugi workflow config
      • kintsugi workflow check
      • kintsugi workflow run
      • kintsugi workflow status
    • SLURM Commands
      • kintsugi slurm init
      • kintsugi slurm submit
      • kintsugi slurm status
      • kintsugi slurm cancel
    • Environment Variables
  • API Reference
    • Main Package
      • Functions
        • kintsugi.check_dependencies
        • kintsugi.get_config_template
    • Kreg Module (Registration)
      • Valis Class
    • Kview2 Module (Visualization)
      • imshow
      • curtain
      • crop
    • Kstitch Module (Stitching)
    • signal Module
    • qc Module
    • denoise Module
    • edf Module (Extended Depth of Focus)
      • extended_depth_of_focus_variance
    • kcorrect_gpu Module (Illumination Correction)
    • project Module
      • KintsugiProject Class
      • ExperimentConfig Class
    • gpu Module
    • hpc Module
    • parallel_io Module
    • zarr_io Module
    • deps Module
      • DependencyChecker Class
    • cli Module
    • mcp Module (optional MCP server)

Reference

  • KINTSUGI Troubleshooting Guide
    • Table of Contents
    • Installation Issues
      • Conda Environment Creation Fails or Hangs
      • Package Installation Errors
    • Dependency Issues
      • libvips Not Found
        • Windows
        • Linux
        • macOS
      • Verify libvips Installation
      • Java/JVM Issues (Legacy)
      • CUDA/GPU Issues
        • Verify GPU Detection
        • Solutions
    • Runtime Issues
      • Import Errors
        • General Solutions
        • Specific Import Errors
      • Memory Issues
        • Solutions
      • Stitched Image Quality Issues
        • Diagnosing the Problem
        • Saturation Issues (Blank White Images)
        • Tile Grid Patterns (Visible Seams)
        • Batch Reprocessing
      • Registration Failures
        • Check Input Images
        • Adjust Parameters
    • HPC/SLURM Issues
      • “snakemake: command not found”
      • “No workflow/config.yaml found”
      • CuPy Appears Unavailable
      • Jobs Pending Indefinitely (QOS Limits)
      • SLURM Jobs Fail with OOM (Out of Memory)
      • “Missing output files after job completion” (NFS Latency)
      • “CUDA initialization failed” in Job Logs
      • “srun: fatal: SLURM_TRES_PER_TASK is mutually exclusive”
      • Stitch Model Not Found for CH2+
    • MCP Server (AI assistants)
      • “Executable not found in $PATH: kintsugi”
      • “No project configured”
      • Server shows “Pending approval”
      • “mcp 1.x is installed but KINTSUGI requires mcp>=2.0,<3”
    • Platform-Specific Issues
      • Windows
        • Long Path Issues
        • DLL Load Failures
      • Linux
        • Permission Issues
        • Display Issues (Napari)
      • macOS
        • Apple Silicon (M1/M2)
        • Gatekeeper Blocks
    • Getting Help
    • Quick Reference
  • HPC Troubleshooting Guide
    • Issue 1: Packages Show [MISSING] Despite Being Installed
    • Issue 2: kintsugi install all Completes But Packages Are Missing
    • Issue 3: CuPy Imports But CUDA Operations Fail
    • Issue 4: PyTorch is CPU-Only
    • Issue 5: SLURM Jobs Fail With TRES Error
    • Issue 6: torch-cuda Shows [SKIP] on Login Node
    • Verification
    • Getting Help
  • Dependency Safety Guide
    • Safe Install Order
    • Critical Rules
      • Never install torch directly with pip
      • The numpy < 2.0 constraint
      • CuPy needs CUDA runtime libraries
      • SLURM TRES patch (HPC only)
    • Validation
    • Troubleshooting
      • “numpy >= 2.0 detected”
      • “CPU-only PyTorch detected”
      • “CuPy installed but CUDA libraries missing”
      • “SLURM_TRES_PER_TASK patch NOT applied”
      • Full environment rebuild
    • Feature Requirements by Notebook
  • KINTSUGI Dependency Analysis Report
    • Executive Summary
    • 1. Current Project Structure
    • 2. External Dependencies Analysis
      • 2.1 libvips (via pyvips)
      • 2.2 Maven & Java-JDK
      • 2.3 VALIS (valis-wsi)
      • 2.4 Stackview
    • 3. Dependency Management Assessment
      • 3.1 Current State
      • 3.2 What’s Missing
    • 4. Identified Issues
      • 4.1 Critical Issues
      • 4.2 Moderate Issues
      • 4.3 Minor Issues
    • 5. Proposed Improvements
      • 5.1 Package Structure (Priority: High)
      • 5.2 Dependency Validation (Priority: High)
      • 5.3 Environment Consolidation (Priority: Medium)
      • 5.4 Installation Script (Priority: Medium)
      • 5.5 Documentation Updates (Priority: Low)
    • 6. Implementation Roadmap
      • Phase 1: Package Structure (Week 1)
      • Phase 2: Dependency Management (Week 2)
      • Phase 3: Testing & CI (Week 3)
      • Phase 4: Documentation (Week 4)
    • 7. Summary
      • Current State
      • Recommended Actions
      • Risk Assessment

Development

  • Contributing to KINTSUGI
    • Development Setup
    • Running Tests
    • Code Quality
    • Commit Message Format
    • Pre-commit Hooks
    • Pull Request Process
    • Release Process
    • Documentation
    • Reporting Issues
  • Documentation Maintenance Guide
    • Documentation Structure
    • Building Documentation Locally
      • Prerequisites
      • On Windows
      • On Linux/macOS
      • Viewing the Built Documentation
    • Live Preview During Development
    • Deploying to GitHub Pages
      • Automatic Deployment
      • Manual Deployment
      • Accessing the Live Site
      • Enabling GitHub Pages (First Time Setup)
    • Adding New Documentation
      • Adding a New Page
      • Adding API Documentation
      • Adding Images
    • Writing Style Guidelines
    • Common Tasks
      • Clean Build
      • Check for Broken Links
      • Generate PDF (requires LaTeX)
    • Troubleshooting Documentation Builds
      • “sphinx-build not found”
      • “make is not recognized” (Windows)
      • Import errors during build
      • Theme not found
      • MyST parser errors
    • Version Updates
    • Contact
  • Git Submodules Guide
    • Current Submodules
    • Quick Start
      • Cloning the Repository with Submodules
      • Pulling Updates with Submodules
    • Common Operations
      • Check Submodule Status
      • Update Submodules to Tracked Commits
      • Update Submodules to Latest Remote
      • Making Changes in a Submodule
    • Recommended Git Aliases
    • Troubleshooting
      • Detached HEAD in Submodule
      • Submodule Directory is Empty
      • Submodule Changes Not Reflected
      • Reset Submodule to Tracked Commit
    • Adding a New Submodule
    • Removing a Submodule
    • Best Practices
    • References

Technical Notes

  • Algorithm Optimization Plan: BM3D-lite & NLM Denoising
    • Overview
    • Current Implementation Analysis
      • BM3D-lite (denoise_bm3d_lite)
      • NLM (denoise_patch_similarity)
    • Optimization Strategy
      • 1. BM3D-lite: KD-Tree Spatial Indexing
      • 2. NLM: OpenCV Replacement
    • Validation Framework
      • Test Categories
      • Quality Metrics
      • Performance Metrics
      • Test Images
    • Test Implementation
      • Phase 1: Baseline Measurements
      • Phase 2: Optimized Implementation
      • Phase 3: Comparative Testing
      • Phase 4: Validation Criteria
    • Rollback Plan
    • Timeline
    • Files to Create/Modify
      • New Files
      • Modified Files (after validation)
    • Success Criteria Summary
  • Algorithm Optimization Validation Results
    • Executive Summary
    • BM3D-lite KD-tree Optimization
      • Test Results
      • Analysis
      • Recommendation: APPROVE WITH NOTES
    • NLM OpenCV Replacement
      • Test Results
      • Analysis
      • Recommendation: APPROVE
    • Implementation Recommendations
      • 1. BM3D-lite with KD-tree
      • 2. NLM with OpenCV
    • Rollback Considerations
    • Conclusion
  • KINTSUGI Performance Audit Report
    • Executive Summary
    • 1. N+1 Query Patterns (Database)
      • 1.1 Loop with DB Query Per Operation (CRITICAL)
      • 1.2 Loop with DB Write Per Operation (CRITICAL)
      • 1.3 Sequential Queries in Statistics Gathering
      • 1.4 Dual Database Queries in recommend_parameters()
    • 2. Inefficient Algorithms
      • 2.1 O(n²) Patch Similarity Comparison (CRITICAL)
      • 2.2 O(n⁴) Pixel-Level NLM Denoising (CRITICAL)
      • 2.3 Unvectorized Overlap Matrix Computation
      • 2.4 Suboptimal K-NN with Full Sort
      • 2.5 Repeated Morphology Per Label
      • 2.6 Unvectorized Peak Detection
      • 2.7 Custom Otsu’s Threshold Instead of Library
    • 3. Memory Inefficiencies
      • 3.1 Unnecessary Dask Array Copies (CRITICAL)
      • 3.2 Full-Size Mask Allocation Per Tile
      • 3.3 Inefficient Patch List Building
      • 3.4 Multiple Normalization Copies
      • 3.5 Multi-Pass Statistics Computation
      • 3.6 Full Array Load for Line Profile
    • 4. I/O Bottlenecks
      • 4.1 Repeated File Metadata Checks (CRITICAL)
      • 4.2 Sequential Image Loading (CRITICAL)
      • 4.3 Multiple Glob Patterns per Directory
      • 4.4 Missing Caching on File Type Functions
      • 4.5 Inconsistent Parallel vs Sequential I/O
      • 4.6 Full Array Load for Thumbnails
    • Priority Matrix
    • Recommendations
      • Immediate Actions (P0)
      • Short-term (P1)
      • Medium-term (P2)
      • Long-term (P3)
    • Testing Recommendations
  • Evaluation: PyImageJ/CLIJ2 vs Pure Python for Extended Depth of Field (EDF)
    • Executive Summary
    • 1. Current Implementation Analysis
      • 1.1 Architecture Overview
      • 1.2 Algorithm Parameters
      • 1.3 Performance Benchmarks
      • 1.4 Current Dependencies
    • 2. CLIJ2 Algorithm Details
      • 2.1 What extendedDepthOfFocusVarianceProjection Does
    • 3. Pure Python Alternatives
      • 3.1 Available Libraries
      • 3.2 Recommended Pure Python Implementation
      • 3.3 Tiled Processing for Large Images
    • 4. Comparison: PyImageJ/CLIJ2 vs Pure Python
      • 4.1 Performance Comparison
      • 4.2 Pros and Cons
        • PyImageJ/CLIJ2
        • Pure Python
    • 5. Recommendations
      • 5.1 Hybrid Architecture (Recommended)
      • 5.2 Implementation Plan
        • Phase 1: Create Pure Python EDF Module
        • Phase 2: Create Unified Interface
        • Phase 3: Integration
      • 5.3 Use Case Recommendations
    • 6. Validation Strategy
    • 7. Conclusion
    • Appendix: Quick Reference
      • Current EDF Call in Notebooks
      • Proposed Python Equivalent
  • Kstitch Alternatives Evaluation
    • Executive Summary
    • Current Implementation: Kstitch
      • Architecture Overview
      • Key Features
      • Performance Characteristics
      • Critical Integration Points
    • Alternative Libraries Evaluated
      • 1. ASHLAR (Alignment by Simultaneous Harmonization)
      • 2. Original m2stitch
      • 3. RAPIDS cuCIM
      • 4. stitch2d
      • 5. OpenCV Stitching (via stitching package)
      • 6. BigStitcher (via PyImageJ)
    • Detailed Comparison Matrix
    • Efficiency Analysis
      • Current Kstitch Performance Breakdown
      • Theoretical ASHLAR Performance (Same Data)
      • Optimization Opportunities in Current Kstitch
    • Recommendations
      • Primary Recommendation: Keep Kstitch
      • Optional Enhancements (If Performance Issues Arise)
      • When to Reconsider
    • Conclusion
    • Sources
  • KINTSUGI Processing Speed Optimization - Session Summary
    • Background
      • Previous Work Completed
      • Current Implementation Files
    • Repository Review Findings
      • Repositories Analyzed
      • Key Optimization Techniques Found
        • From BaSiCPy (peng-lab)
        • From MIST (NIST)
        • From PyBaSiCCellprofilerPlugin
        • From RAPID
    • Proposed Implementation Plan
      • Phase 1: BaSiC Illumination Correction
      • Phase 2: Stitching Optimization
      • Phase 3: Parallel Processing
      • Phase 4: Output Format
      • Expected Gains
    • IMPORTANT: BaSiC Caching Validation Required
      • User Concern
      • Test Design
        • Test Cases
        • Processing Modes to Compare
        • Metrics to Evaluate
        • Success Criteria
    • Files to Modify
    • Constraints
    • Next Steps
    • References
  • Bug Report: UnboundLocalError in registration.py:register_micro() (v1.2.0)
    • Description
    • Code with Bug
    • Expected Fix
    • Reproduction
    • Note
    • Context
    • Environment
KINTSUGI
  • Search

© Copyright 2024-2026, Smith JT.

Built with Sphinx using a theme provided by Read the Docs.