borealisai / noise_flow Goto Github PK
View Code? Open in Web Editor NEWNoise Flow: Noise Modeling with Conditional Normalizing Flows
License: Other
Noise Flow: Noise Modeling with Conditional Normalizing Flows
License: Other
It seems that the FTP server is not available now. When trying to run check_download_sidd
, it throws a connection timeout error.
Hi Abdelrahman,
Very interesting work, thank you for sharing the repo.
I was wondering whether you dequantize the clean image and noise when training noise flow? I couldn't find anything in the code.
Thanks in advance for your feedback.
20:54 kwephis2148189 noise_flow_model E=30 tr=494.6 ts=67.6 tsm=471.8 tv=0.0 T=1034.2 tL=-14214.2 sL=-14450.6 smL=-10295.5 SDr=1.0 SDs=0.9 B=1 0.679,0.785,0.969,0.000
It is my first time get a minus loss, and I donnot know why the absolute value is so big。
Thanks for your greate work, my question is as the title. Clean image, Noise image , Noise as follows(Sorry, I cannnot upload it, )
Hi, I was wondering how to adapt you code to another dataset for training, e.g. SID dataset [1]
[1] Learning to see in the dark
Hello, thanks for the great work of noise modeling.
I want to run your code but get the following error:
FileNotFoundError: [Errno 2] File b'/home/arc/_Code/fourier_flows/experiments/sidd/S6G4/vars.txt' does not exist: b'/home/arc/_Code/fourier_flows/experiments/sidd/S6G4/vars.txt'
I download SIDD_Medium_RAW for training and I think whether other preprocessing steps I should do on the dataset to get VARIANCE_RAW?
Hi, I dive into your code, and find the script provided (see below) seems doesn't match your noise flow model in the paper. Shall I disable the argument of --cam
and --iso
? Shall I add additional arguments e.g. --cond_gt
and --init_sdn
? Thanks~
python3 train_noise_flow.py --logdir noise_flow_model --arch "sdn5|unc|unc|unc|unc|gain4|unc|unc|unc|unc" \
--sidd_path './data/SIDD_Medium_Raw/Data' --cam IP --iso 800 --n_train_threads 16 \
--width 4 --epochs 2000 --lr 1e-4 --n_batch_train 138 --n_batch_test 138 --epochs_full_valid 10 \
--n_patches_per_image 2898 --patch_height 32 --patch_sampling uniform \
--start_tr_im_idx 10 --end_tr_im_idx 12 --start_ts_im_idx 10 --end_ts_im_idx 12
The training's input is the clean image and noise. The training is along the forward direction , "Sdn->A4->Gain->A4", as the figure 3 in paper while all layers use the inverse calculation (train_multithread function in code).
The sampling's input is the clean image with Gauss. The sampling is along the inverse direction (reversed model) while all the layers use the forward calculation (sample_multithread function in code).
I wonder if my understanding above is correct.
Why does the model operate in the forward direction while using the inverse calculation ?
Thank you for sharing your implementation!
It seems like the server hosting the SIDD dataset is down, or is it just me?
Hi,
I'm trying to sample noise directly on sRGB images, but the result seems noisier than expected. Right now, I'm doing as follows:
I think I may be missing something. As the noisy image (second img below) seems much noisier than expected:
My code is as follows
from borealisflows.NoiseFlowWrapper import NoiseFlowWrapper
patch_size, stride = 32, 32 # patch size = [32, 32, 4]
aug_times = 1
scales = [1] # [1, 0.9, 0.8, 0.7]
nf_model_path = 'models/NoiseFlow'
def load_cam_iso_nlf():
cin = pd.read_csv('cam_iso_nlf.txt')
cin = cin.drop_duplicates()
cin = cin.set_index('cam_iso', drop=False)
return cin
# Prepare NoiseFlow
noise_flow = NoiseFlowWrapper(nf_model_path)
# camera IDs and ISO levels related to the SIDD dataset
cam_iso_nlf = load_cam_iso_nlf()
n_cam_iso = cam_iso_nlf['cam_iso'].count()
iso_vals = [100.0, 400.0, 800.0, 1600.0, 3200.0]
cam_ids = [0, 1, 3, 3, 4] # IP, GP, S6, N6, G4
cam_vals = ['IP', 'GP', 'S6', 'N6', 'G4']
def pack_raw(rgb_img):
"""Packs sRGB image to 4 channels (h, w) --> (h/2, w/2, 4)."""
# pack sRGB image to 4 channels
im = np.expand_dims(rgb_img, axis=2)
img_shape = im.shape
h = img_shape[0]
w = img_shape[1]
out = np.concatenate((im[0:h, 0:w, :, 0],
im[0:h, 0:w, :, 1],
im[0:h, 0:w, :, 1],
im[0:h, 0:w, :, 2]), axis=2)
return out
def gen_patches_png(img_file):
# split patches of a given img
img = img_file
img = np.expand_dims(pack_raw(img), axis=0)
_,h, w, c = img.shape
patches = None
# extract patches
for i in range(0, h - patch_size + 1, stride):
for j in range(0, w - patch_size + 1, stride):
x = img[0,i:i + patch_size, j:j + patch_size, :] # first dim will be removed
for k in range(0, aug_times):
if patches is None:
patches = x[np.newaxis, :, :, :] # restore first dim
else:
patches = np.concatenate((patches, x[np.newaxis, :, :, :]), axis=0)
return patches,[h,w]
def recover_full_img(patches,shape):
#Combines patches together into an img
h,w = shape
out_image = np.zeros((h,w,3))
index=0
for i in range(0, h - patch_size + 1, stride):
for j in range(0, w - patch_size + 1, stride):
out_image[i:i + patch_size,j:j + patch_size,0] = patches[index,:,:,0]
out_image[i:i + patch_size,j:j + patch_size,1] = patches[index,:,:,1]
out_image[i:i + patch_size,j:j + patch_size,2] = patches[index,:,:,3]
index = index+1
return out_image
img_clean = imageio.imread(img_file).astype('float32')/255.0
patches,img_shape = gen_patches_png(img_clean)
patches = patches**2.2 ##sRGB to linear
cam_iso_idx = random.choice([3,9]) ## ISO-1600
row = cam_iso_nlf.iloc[cam_iso_idx]
cam = cam_vals.index(row['cam_iso'][:2])
iso = float(row['cam_iso'][3:])
noise = noise_flow.sample_noise_nf(patches, 0.0, 0.0, iso, cam)
patches_noisy = np.clip(patches + noise,0,1)
patches_noisy = patches_noisy**(1/2.2) ##linear to sRGB
recovered_img_noisy = recover_full_img(patches_noisy,img_shape)
imageio.imwrite(cur_img.replace('.png','_NF.png'),recovered_img_noisy)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.