Hi,

First and foremost, thank you for this wonderful library that you have written.

I am writing this to ask if coloured (3 channels) stack tiffs can be registered and transform from the methods found in this library? Based on the documentation and the tutorials, it seems that image registration can only be applied to single colour channel images, but I would like to confirm if it is true. If so, are there any workarounds?

Thank you for your time.

Hello!

May I ask a question here? Hope it won't bother you too much.
I'm not familiar with the registration method.
I wonder why this package can't apply to images with different sizes?

Thank you!

Hi @glichtner
I'm trying to understand the difference between using skimage wrap and pystackreg.
From the documentation of skimage - https://scikit-image.org/docs/stable/api/skimage.transform.html#skimage.transform.warp I see that the default interpolation is Bi-linear, what is the interpolation used for pystackreg? (Assuming Affine transformation).

also, correct me if I'm wrong, but can you directly use the tmat in skimage wrap function?

for i in range(tmats.shape[0]):
    #tform = tf.AffineTransform(matrix=tmats[i, :, :])
    out[i, :, :] = tf.warp(img1[i, :, :], tmats[i, :, :]) # tform)

Thanks!

I'm having difficulty getting pystackreg to work for a simple example where I rotate the same square image by 90 degrees and try to register the rotation back to the original.

I'm using pystackreg version 0.2.7, python 3.8.12, and a macOS-10.16-x86_64-i386-64bit

I'm having similar issues on other images (all from histopathology). I've tried a few simple things like binarizing the images (i.e. tissue vs. background) or changing the datatypes (np.uint8, float), but nothing helps. My issue is perhaps similar to this issue.

Reproducible example

Here is the original RGB image so you can run the below code yourself

Some setup

from skimage.transform import rotate
from skimage.color import rgb2gray
from skimage.io import imread
from skimage.exposure import equalize_hist

import matplotlib.pyplot as plt
import numpy as np

from pystackreg import StackReg
import pystackreg

save_dir = './'

print("pystackreg version", pystackreg.__version__)

def composite_images(imgs, equalize=False):
    if equalize:
        imgs = [exposure.equalize_hist(img) for img in imgs]
    
    imgs = [img / img.max() for img in imgs]
    
    if len(imgs) < 3:
        imgs += [np.zeros(shape=imgs[0].shape)] * (3-len(imgs))
  
    imgs = np.dstack(imgs)
    
    return imgs

def plot_ctsas(target, source_aligned, source, equalize=False):
    """
    Plots the composite, target, source aligned, and source image.
    """
    composite = composite_images([target, source_aligned], equalize=equalize)
    
    
    plt.subplot(1, 4, 1)
    plt.imshow(composite)
    plt.axis('off')
    plt.title("composite")
    
    plt.subplot(1, 4, 2)
    plt.imshow(target, cmap='gray')
    plt.axis('off')
    plt.title("target image")
    
    
    plt.subplot(1, 4, 3)
    plt.imshow(source_aligned, cmap='gray')
    plt.axis('off')
    plt.title("source aligned")
    
    
    plt.subplot(1, 4, 4)
    plt.imshow(source, cmap='gray')
    plt.axis('off')
    plt.title("source image")
    
   image_rgb = imread(os.path.join(save_dir, 'image.png'))

image_rotated_rgb = rotate(image=image_rgb, angle=90, preserve_range=True).astype(np.uint8)
source = rgb2gray(image_rgb)
target = rgb2gray(image_rotated_rgb)

print("original image rgb shape {}, dtype {}".format(image_rgb.shape, image_rgb.dtype))
print('source shape {}, dtype {}'.format(source.shape, source.dtype))

plt.figure(figsize=(8, 4))
plt.subplot(1, 2, 1)
plt.imshow(source, cmap='gray')
plt.title('source')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.imshow(target, cmap='gray')
plt.title('target')
plt.axis('off')
plt.savefig(os.path.join(save_dir, 'source_and_target.png'), bbox_inches='tight')

pystackreg version 0.2.7
original image rgb shape (256, 256, 3), dtype uint8
source shape (256, 256), dtype float64

Try registering

Now lets try registering the rotated image

transformations = {
    'TRANSLATION': StackReg.TRANSLATION,
    'RIGID_BODY': StackReg.RIGID_BODY,
    'SCALED_ROTATION': StackReg.SCALED_ROTATION,
    'AFFINE': StackReg.AFFINE,
    'BILINEAR': StackReg.BILINEAR
}


for (name, transf) in transformations.items():
    registerer = StackReg(transf)
    source_aligned = registerer.register_transform(ref=target, mov=source)

    plt.figure(figsize=(16, 4))
    plot_ctsas(target=target, source_aligned=source_aligned, source=source)
    
    plt.savefig(os.path.join(save_dir, '{}.png'.format(name)), bbox_inches='tight')

Hi, I'm trying to get pystackreg up and running with Python 3.8. Is there any plan to issue a 3.8 compatible whl?

thanks,
Dave

I have an already established Anaconda image processing environment to which I would like to add PyStackReg. I have successfully installed it via pip, however, I get an error when I try to import any of the PyStackReg modules. Specifically, the error is:

ImportError: numpy.core.multiarray failed to import

I believe the error is due to the NumPy version that is installed (1.13), but another package is preventing upgrading. Any suggestions? Can the code be made more robust against the NumPy version?

Hello,

I've got the following error while pip install on ubuntu.

Collecting pystackreg
  Using cached pystackreg-0.2.5.tar.gz (3.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in /home/frontline/virtalenvs/default/lib/python3.7/site-packages (from pystackreg) (1.19.5)
Collecting tqdm
  Using cached tqdm-4.61.1-py2.py3-none-any.whl (75 kB)
Building wheels for collected packages: pystackreg
  Building wheel for pystackreg (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/frontline/virtalenvs/default/bin/python /home/frontline/virtalenvs/default/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmprvzhdjik
       cwd: /tmp/pip-install-209f_a45/pystackreg_dc4003d641f2439cac83d96e8a0922a2
  Complete output (22 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/pystackreg
  copying pystackreg/version.py -> build/lib.linux-x86_64-3.7/pystackreg
  copying pystackreg/conftest.py -> build/lib.linux-x86_64-3.7/pystackreg
  copying pystackreg/pystackreg.py -> build/lib.linux-x86_64-3.7/pystackreg
  copying pystackreg/__init__.py -> build/lib.linux-x86_64-3.7/pystackreg
  creating build/lib.linux-x86_64-3.7/pystackreg/util
  copying pystackreg/util/__init__.py -> build/lib.linux-x86_64-3.7/pystackreg/util
  running build_ext
  building 'pystackreg.turboreg' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iinc/ -I/tmp/pip-build-env-kgb5b63l/overlay/lib/python3.7/site-packages/numpy/core/include -I/home/frontline/virtalenvs/default/include -I/usr/include/python3.7m -c src/pymain.cpp -o build/temp.linux-x86_64-3.7/src/pymain.o -std=c++11
  src/pymain.cpp:10:10: fatal error: Python.h: No such file or directory
   #include <Python.h>
            ^~~~~~~~~~
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pystackreg
Failed to build pystackreg
ERROR: Could not build wheels for pystackreg which use PEP 517 and cannot be installed directly

I'm trying to multiprocess image pairs to speed up registration:

def regWorker(ref, cur):
    sr = StackReg(StackReg.TRANSLATION)
    ref_img = cv2.imread(ref, -1).astype(np.float64)
    cur_img = cv2.imread(cur, -1).astype(np.float64)
    registered = sr.register_transform(ref_img, cur_img).astype(np.uint16)
    return cur, registered

def register_tseries(img_paths, n_proc):
    l = list(img_paths.values())
    inp = [(l[i], l[i+1]) for i in range(len(l)-1)]
    p = Pool(processes=n_proc)
    res = p.starmap(regWorker, inp)
    for r in res:
        cv2.imwrite(r[0], r[1])

The output from the code doesn't quite match that from the following sequential call and I'm wondering why?

def register_tseries(img_paths, n_proc):
    l = list(img_paths.values())
    inp = [(l[i], l[i + 1]) for i in range(len(l) - 1)]
    sr = StackReg(StackReg.TRANSLATION)
    for i in inp:
        ref_img = cv2.imread(i[0], -1).astype(np.float64)
        cur_img = cv2.imread(i[1], -1).astype(np.float64)
        registered = sr.register_transform(ref_img, cur_img).astype(np.uint16)
        cv2.imwrite(i[1], registered)

Thanks!

Hi,
pystackreg is great as I am trying to get into image processing with Python instead of ImageJ/Fiji.

Unfortunately I am running into some issues regarding the memory usage with pystackreg. I want to do a rigid body registration on an 4GB image around 75 frames.
As you can see in the images pystackreg uses all of the 32GB of memory. You notice it to as the PC is lagging. For the same image Fiji uses a considerable less amount of memory. The usage of CPU and Disk (perhaps because of swapping) is a lot higher with pystackreg too.
Do you have an idea why this is or how to solve this?

Used software:
Windows 10 - Python 3.9 - pystackreg 0.2.7
Windows 10 - ImageJ (Fiji) 1.53t - StackReg 2.0.0

Used code:
I tried it with 'previous' and 'first' as reference but it was the same behaviour.

from skimage import io
from pystackreg import StackReg

image = io.imread(filepath)
sr = StackReg(StackReg.RIGID_BODY)
registered_image = sr.register_transform_stack(image, reference='previous')

pystackreg - memory usage at the start

pystackreg - memory usage later

Fiji - memory Usage at the start

Fiji - memory usage later

Hi glichtner,

I'm trying out your pystackreg package and so far only affine & bilinear transformations seem to work, with the others, like translation, not having an effect on the movable image. I'd be happy to send you the code / image files I'm working with by email!

Ryan

Is there a way to get the tranformation matrix when performing rigid body registration but not the product of rotation*translation as here:
sr = StackReg(StackReg.RIGID_BODY)
tmat = sr.register(im_ref, im)
I noticed that when I put the translation to 0, for example using this:
angle = 45
tmat = np.array([[np.cos(np.deg2rad(angle)), -np.sin(np.deg2rad(angle)),0],
[np.sin(np.deg2rad(angle)), np.cos(np.deg2rad(angle)),0],
[0,0,1]])
imr = sr.transform(im0, tmat)

the rotated image imr leaves the field of view; I guess this is because the (0,0) for the rotation is not the middle of the image but the top left corner?

The reason why I am asking this is because I would like to use the tmat from pystackreg as a tranformation matrix in pytorch (to use it in torch.nn.functional.affine_grid and torch.nn.functional.grid_sample) but in its current form it does not match. It would be possible if we could get the x/y offsets in pixels from the centre of the image.

Hello, Thanks for this great package. It is compatible with 3D + time motion correction? If no, could you suggest another python package for this? Thanks in advance.

Hi
I have almost 17000 frames to be aligned. I am wondering if pystackreg can registersuch images (using first frame as reference, let's say) in parallel ?
Thanks

I am looking for Cross-modal image registration for some of my projects and I am wondering will it support cross-modal images.

Thanks a lot for effort you put into this library. I have two questions which I hope you can give some insight into:

Negative Pixel Values

I realised that often after applying the transform() method to the image, the resulting image contains negative pixel values.

My pipeline looks as follows:

1. I have a reference portrait image where I want to paste a distorted face of the same person onto it
2. I pad the face image with a black border to make it the same size as the reference portrait image (Roughly putting the face at the same position as the reference portrait image to make the algorithm more robust)
3. I split the portrait image into the 3 separate rgb channels, register the two images by using their grayscale-version, then applying the resulting transform onto the three channels. After that I stack the separate channels together to get the final image.

Unfortunately this gives me negative values.

Is there something fundamentally wrong with my approach?

transform_stack glitches

When doing the above mentioned approach with multiple distorted faces on the same reference portrait image, it produces nice results when processing each frame separately.
But when using the register_stack and transform_stack() methods, the resulting frames have glitches such as black lines appearing at random positions or weird transformations (Please see the attached screenshot). Do you have any idea why this might happen?

Hello,
In the Fiji plugin, TurboReg offers the possibility between registering the image either fast or accurate. Is this also offered in the pystack reg package. I looked at the documentation and code, and it seems like it does not have that option. Am I correct, or missing something?

Thank you for your help

Best,

Hi Gregor,

I wish to align (register) an image to its previous timepoint in a timelapse using a StackReg.TRANSLATION. For some reason, the translated image does not seem to be aligned at all, it's rather th opposite: the translation moved the image even further from where it should have been.

import matplotlib.pyplot as plt
import numpy as np
from skimage import io
from pystackreg import StackReg

input_dir = '/Users/ht/Desktop/test_folder_ht/'
img0 = io.imread(input_dir + 'denoised_Lys_C100_Pos0_img_000000003_02-BF_000_xy5_cp_masks.tif')
img1 = io.imread(input_dir + 'denoised_Lys_C100_Pos0_img_000000004_02-BF_000_xy5_cp_masks.tif')

# Binarize the 2 images for alignment:
img0_tmp = np.where(img0 != 0, 1, img0)
img1_tmp = np.where(img1 != 0, 1, img1)

# Register (align) the 2 images and plot superimposed images before and after alignment.
# alignement to the previous image
sr = StackReg(StackReg.TRANSLATION)
translated_img = sr.register_transform(img0_tmp, img1_tmp)

# plot the superimposed images before algnment
plt.figure(figsize=(10, 10))
ax = plt.gca()
im = ax.imshow(img0_tmp+img1_tmp)
plt.suptitle('Superimposed images before alignment', fontsize=16)

# plot the superimposed images after algnment
plt.figure(figsize=(10, 10))
ax = plt.gca()
im = ax.imshow(img0_tmp + translated_img)
plt.suptitle('Superimposed images after alignment', fontsize=16)

test_folder_ht.zip

Hi,

thanks for creating this very useful tool! I want to apply pystackreg to register two microscopy images. The reference image has a high and the second image has a low SNR. However, after alignment, negative pixel values appear. I followed the suggestions in issue #8, but could not get rid of the negative values. Therefore, I would be very grateful for your input on how to fix this!! Attached is my code and two example images.

As the images differ drastically in their pixel value range (high SNR:0-340, low SNR:0-34), I found it to be useful to stretch the pixel values across the entire bit depth (0-65535), learn the registration based on the stretched images, and apply the registration on the non-streched 2nd image. This reduces the number of negative pixel values slightly. Atm, I am clipping negative values to zero, but would prefer if negative values could be avoided in the first place. Without the stretching approach, the registration is unstable and applies an incorrect large shift to the 2nd image.

register_images_negative_values.zip

pystackreg version: 0.2.7

Best,
Johanna

Hello,
I have found that a piece of code in which I use pystackreg delivers completely different (and, I think, wrong) results once I update to the latest Anaconda distribution (2022.05). This is on Windows 10 64bit.

Using a previous distribution, probably a year older or more, still works fine.

I import StackReg and do some image registration.

I suspect that this is due to some python packages versions being updated. I want to help you as much as I can to troubleshoot this issue. Please let me know what information you need from me.

Hi
Thanks for the neat software. I have a simple question. How can I use the transformation matrices derived by pystackreg to move a set of coordinates (ex. contour)? As far as I understood, you can only transform image stacks using SR.transform() function.

Hi there,

I am working with fluorescence microscopy images of beads. The images are acquired on different channels (Red and Green). I would like to transform the red channel using the green as reference, but only on the XY coordinates.

Input: XY coords of beads on the red channel before transformation.
Output: XY coords of beads on the red channel after transformation.

In other words, I would like to know if there is a way on PyStackReg to use the transformation matrix after Registration (e.g AFFINE) to transform XY coordinates instead of the whole image.

Thanks!

Altair

We have two identical images that have been slightly transformed in two different stacks, which we want register one with the other. The images are 1000x1000 pixels

We want to register these two images:

To do the registration we are using the following code:

chunk_last = result[8][j-5]            #image1               
chunk_first = result[9][j]             #image2            
sr = StackReg(StackReg.AFFINE)
tmatrix = sr.register(chunk_last, chunk_first)
chunk_last_transformed = sr.transform(chunk_first, tmatrix)

and got the following images:


As you can see the images are the same but are transformed slightly different in the stacks, image2 (magenta) looks sheared compared to image1.

This is the 2 images superimposed:

Thanks in advance and thank you for the amazing package!