gehelem / als Goto Github PK

Means dark background and mostly red text

Allow user to define a rectangular region inside the image and crop display according to that region

reproduced on develop (c2a3134)

Steps to reproduce :

1. Launch ALS and its scanner
2. Drop at least 1 image is the scanned folder and let ALS process and show the first image
3. Check the SCNR checkbox and change associated slider value to anything
4. Click apply

In als.preprocess.save_tiff function, the following expression (line 206) evaluates to false :

    # if no have change, no process
    if param[0] != 1 or param[1] != 0 or param[2] != 0 or param[3] != limit \
            or param[4] != 1 or param[5] != 1 or param[6] != 1 or any([v != 1 for _, v in param[9].items()]):

So the code flow jumps right to line 271 and no processing is done at all

Is this intentional because it does not make sense to apply SCNR only, or did I miss some other control to activate on the GUI, or is this a bug in this if statement ?

nedd merge process for first image and process for other image

In both README and GitHub issue templates

When raising the value of the black slider, we expect to have more and more black pixels in the image

But we get more and more white :)

Might (or might not?) be worth to premultiply each element (the stack and the new image) by the weight before adding the result in:
https://github.com/gehelem/als/blob/develop/src/als/stack.py#L248

I checked, and the technic I am suggesting is not really usefull to protect against overflow in normal settings, ie:
Maximum value for float32 is 3.402823466e+38 that makes 5.1922965484619143e+33 images with maximum value of 65535 before overflowing.

In terms of numerical stability, I don't see immediate benefit, as we are using floating point arithmetic, we will logically run into issue after some point in both cases, ie when new value will be smaller than current one with a ratio of 2^23 / count_number it will be considered 0 in practice. (cf 23 fractional bits in https://en.wikipedia.org/wiki/Single-precision_floating-point_format)
This is an approximation, as processor using fused multiply add might actually use more bits than that for the intermediate operand.

However there is still one (small) advantage: multiplying stack by (count-1)/count or 1/count beforehand might result in a shorther runtime, depending on the platform.

Here is my attempt at exhibiting this behaviour (around 10% gained in runtime consistently):

In [34]: import timeit 
In [35]: sl=""" 
    ...: import numpy as np 
    ...: a=np.random.rand(2048, 2048).astype(np.float32) 
    ...: b=np.random.rand(2048, 2048).astype(np.float32) 
    ...: def doitlong(x,y): 
    ...:     return (x*0.99+y*0.01)/1.555 
    ...: """                                                                                                                                                                                                                                                                       

In [36]: ss=""" 
    ...: import numpy as np 
    ...: a=np.random.rand(2048, 2048).astype(np.float32) 
    ...: b=np.random.rand(2048, 2048).astype(np.float32) 
    ...: def doitshort(x,y): 
    ...:     return x*(0.99/1.555)+y*(0.01/1.555) 
    ...: """                                                                                                                                                                                                                                                                       

In [37]: timeit.timeit('doitlong(a,b)', setup=sl,number=1000)                                                                                                                                                                                                                      
Out[37]: 20.887005210999632

In [38]: timeit.timeit('doitshort(a,b)', setup=ss,number=1000)                                                                                                                                                                                                                     
Out[38]: 18.9014974380043

Please not that those result may vary from one platform to another depending on memory access performances, availability of some instruction set...

Double click or Ctrl+click on slider handle, for example

Having a keyboard shortcut to hide and show every control
i humbly suggest i = L+P+B
Two possibilities, refering to Ms Michu's use case :

• keep in memory what has been hiden to be able to revert
• that this shortcut would be just a "show all" / "hide all"

When I first came to this project, I downloaded the master branch, simply because it is the default one.

Upon encountering errors, and opening an issue, I was instructed to use the alpha branch.

In order to avoid user confusion, and reduce the support load on the dev team, please make alpha the default branch on Github.

Need code server for see stack picture in live by other.

Do you think it would be a good idea to generate a QR-code from ALS such that one can easily share the url for people to connect to the image server through their phone ?

Using this: https://pypi.org/project/qrcode/ looks not so hard

There is an inconsistency between displayed image bit depth and JPEG saved image
Values end up to high in the jpeg file

For now 16bit JPEG save is not an option, so we could try and reduce data before saving to jpeg

something like : value / 2^ * 2^8 ...

We need to

• specify our dev workflow
• specify what versioning scheme we'll use, and start to use it
• specify how project is released

Refactor project structure with enough tooling for daily development tasks, installation & deployment

The issue is with the call to save_tiff.
We pass the two SCNR and Wavelets QCheckBox'es instead of their checked state. So later on, when testing them (boolean-ly), we get True even when the boxes are not checked.

So both SCNR and Wavelets processing are done. Those are lenghty and freeze the whole app.

First, we need to fix the way save_tiff is called

A global structure redesign will definitely get rid of those possible issues later on

Need add substraction of Dark for image in als code

appears on develop
Steps to reproduce :

• make sure at least one dock is visible
• minimize main window
• maximize window

Expected result : docks that were visible before window was minimized are still visible

Actual result : all docks are hidden

tested on develop branch:

2019-08-21 18:07:18,207 - INFO - __main__ - update GUI image
Qt has caught an exception thrown from an event handler. Throwing
exceptions from an event handler is not supported in Qt.
You must not let any exception whatsoever propagate through Qt code.
If that is not possible, in Qt 5 you must at least reimplement
QCoreApplication::notify() and catch all exceptions there.

Traceback (most recent call last):
  File "/home/user/projects/als/code_utilities.py", line 26, in wrapped
    result = func(*args, **kwargs)
  File "./als.py", line 423, in cb_apply_value
    self.adjust_value(work_folder)
  File "/home/user/projects/als/code_utilities.py", line 26, in wrapped
    result = func(*args, **kwargs)
  File "./als.py", line 448, in adjust_value
    if len(self.image_ref_save.image.shape) == 2:
AttributeError: 'list' object has no attribute 'shape'
Aborted (core dumped)

May start with simple psf estimation + richardson lucy,
Then move on th spatially varying version.

als n not detect init file when als is run since other folder

Must display

• version
• main authors
• link to website (GitHub repo, for now)

We need to identify the past versions of the project :

• v0.1
• v0.2
• v0.3

Currently occurring on develop branch. Step to reproduce :

1. Launch ALS
2. Start scanner
3. Pause scanner
4. Restart scanner
5. Feel the pain

Logs :

2019-09-17 15:33:39,968 - DEBUG - als.als - MainWindow.cb_pause() returned None in 2.268 ms
2019-09-17 15:33:40,927 - DEBUG - als.als - MainWindow.cb_play() called with : (<als.als.MainWindow object at 0x7f1a8fab0a68>,) - {}
2019-09-17 15:33:40,929 - DEBUG - als.als - WatchOutForFileCreations.__init__() called with : (<als.als.WatchOutForFileCreations object at 0x7f1a6009bb88>, '/home/deufrai/als_scan', '/home/deufrai/als_work', True, False, 'Sum', <PyQt5.QtWidgets.QTextBrowser object at 0x7f1a6009b4c8>, <PyQt5.QtWidgets.QSlider object at 0x7f1a6008be58>, <PyQt5.QtWidgets.QSlider object at 0x7f1a6008bc18>, <PyQt5.QtWidgets.QSlider object at 0x7f1a63436b88>, <PyQt5.QtWidgets.QSlider object at 0x7f1a63436f78>, <PyQt5.QtWidgets.QSlider object at 0x7f1a6008b438>, <PyQt5.QtWidgets.QSlider object at 0x7f1a6008b678>, <PyQt5.QtWidgets.QSlider object at 0x7f1a6008b8b8>, <PyQt5.QtWidgets.QPushButton object at 0x7f1a60095288>, <als.als.ImageRefSave object at 0x7f1a54f435f8>, <PyQt5.QtWidgets.QCheckBox object at 0x7f1a60090168>, <PyQt5.QtWidgets.QComboBox object at 0x7f1a600901f8>, <PyQt5.QtWidgets.QSlider object at 0x7f1a60090288>, <PyQt5.QtWidgets.QCheckBox object at 0x7f1a60090558>, <PyQt5.QtWidgets.QComboBox object at 0x7f1a600905e8>, <PyQt5.QtWidgets.QCheckBox object at 0x7f1a60090678>, <PyQt5.QtWidgets.QLabel object at 0x7f1a600908b8>, <PyQt5.QtWidgets.QLabel object at 0x7f1a60090af8>, <PyQt5.QtWidgets.QLabel object at 0x7f1a60090d38>, <PyQt5.QtWidgets.QLabel object at 0x7f1a60090f78>, <PyQt5.QtWidgets.QLabel object at 0x7f1a600951f8>) - {}
2019-09-17 15:33:40,929 - INFO - als.als -  Work folder = '/home/deufrai/als_work'
2019-09-17 15:33:40,930 - INFO - als.als -  Scan folder = '/home/deufrai/als_scan'
2019-09-17 15:33:40,930 - DEBUG - als.als - MyEventHandler.__init__() called with : (<als.als.MyEventHandler object at 0x7f1a6009baf8>,) - {}
2019-09-17 15:33:40,930 - DEBUG - als.als - ImageRefSave.__init__() called with : (<als.als.MyEventHandler object at 0x7f1a6009baf8>,) - {}
2019-09-17 15:33:40,930 - DEBUG - als.als - ImageRefSave.__init__() returned None in 0.005 ms
2019-09-17 15:33:40,930 - DEBUG - als.als - MyEventHandler.__init__() returned None in 0.335 ms
QThread: Destroyed while thread is still running
2019-09-17 15:33:40,939 - DEBUG - als.als - WatchOutForFileCreations.__init__() returned None in 9.131 ms

Process finished with exit code 134 (interrupted by signal 6: SIGABORT)

• not github hosted
• wider sources

In order to efficiently diagnose user issues, we need to put detailed logs in place.
Logs can simply go to stdout and we'll let users redirect to a file if needed

All milestones of app lifecycle must be logged as INFO
All method calls should also be looged (with params and return values) as DEBUG

@deufrai steps forward and volunteers for this improvement

I am using Xubuntu 18.04 LTS, and installed ALS using the instructions on the Github page.

When I try the samples, I get the following output:

someone@astro:~$ python3 ~/als/als.py
/home/someone/als/wrk
/home/someone/als/scan
New image arrive: /home/someone/als/scan/Light_1069.fits
The file does not exist
debayering...
TIFF image create : /home/someone/als/wrk/stack_image.tiff
first file created : /home/someone/als/wrk/stack_ref_image.fit
New image arrive: /home/someone/als/scan/Light_1114.fits
debayering...
alignement and stacking...
0%| | 0/3 [00:00<?, ?it/s]New image arrive: /home/someone/als/scan/Light_1171.fits
100%|█████████████████████████████████████████████| 3/3 [00:00<00:00, 3.32it/s]
OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/imgproc/src/color.cpp, line 9716
Traceback (most recent call last):
File "/home/someone/als/als.py", line 66, in
align_on, save_on, stack_methode))
File "/home/someone/als/als.py", line 80, in created
save_im=save_on, align=align_on, stack_methode=stack_methode)
File "/home/someone/als/stack.py", line 231, in stack_live
save_tiff(work_path, np.array(stack_image), mode=mode)
File "/home/someone/als/stack.py", line 30, in save_tiff
cv2.imwrite(work_path + "/stack_image.tiff", cv2.cvtColor(new_stack_image, cv2.COLOR_RGB2BGR))
cv2.error: /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/imgproc/src/color.cpp:9716: error: (-215) scn == 3 || scn == 4 in function cvtColor

debayering...
Traceback (most recent call last):
File "/home/someone/als/als.py", line 66, in
align_on, save_on, stack_methode))
File "/home/someone/als/als.py", line 80, in created
save_im=save_on, align=align_on, stack_methode=stack_methode)
File "/home/someone/als/stack.py", line 158, in stack_live
raise ValueError("fit format not support")
ValueError: fit format not support
New image arrive: /home/someone/als/scan/Light_1254.fits
debayering...
Traceback (most recent call last):
File "/home/someone/als/als.py", line 66, in
align_on, save_on, stack_methode))
File "/home/someone/als/als.py", line 80, in created
save_im=save_on, align=align_on, stack_methode=stack_methode)
File "/home/someone/als/stack.py", line 158, in stack_live
raise ValueError("fit format not support")
ValueError: fit format not support
New image arrive: /home/someone/als/scan/Light_1264.fits
debayering...
Traceback (most recent call last):
File "/home/someone/als/als.py", line 66, in
align_on, save_on, stack_methode))
File "/home/someone/als/als.py", line 80, in created
save_im=save_on, align=align_on, stack_methode=stack_methode)
File "/home/someone/als/stack.py", line 158, in stack_live
raise ValueError("fit format not support")
ValueError: fit format not support
New image arrive: /home/someone/als/scan/Light_1283.fits
debayering...
Traceback (most recent call last):
File "/home/someone/als/als.py", line 66, in
align_on, save_on, stack_methode))
File "/home/someone/als/als.py", line 80, in created
save_im=save_on, align=align_on, stack_methode=stack_methode)
File "/home/someone/als/stack.py", line 158, in stack_live
raise ValueError("fit format not support")
ValueError: fit format not support

What is going on?

configparser.DuplicateOptionError must be caught on config read

We must also gracefully recover from an unparsable config file

On machines with slow file I/O, watchdog signals the arrival of a new image before the file is fully written to disk.
Subsequent processes fail because they try and access a truncated file

As discussed in the discord, it would be nice to have a feature that would allow to build on the fly, the composited image as a union of the footprint (or extent) of all the previously seen images, in order to enable building mosaic.

Right now, we are using intersection of the footprint(or extent) to build the reconstruction.
To enable this feature, and at the same time, keep computing the average properly on the whole image, we might need to implement a map of image_count, as big as the composite image instead of simple count.

To do so, we might need to use the footprint return value from https://github.com/toros-astro/astroalign/blob/master/astroalign.py#L355

Reproduced on develop
Steps to reproduce :

1. Click START
2. copy images that are not related (different targets) into the scan folder

Actual result : App crash wilth

2019-09-04 07:29:30,088 - INFO - stack - alignement and stacking...
Traceback (most recent call last):
  File "/home/deufrai/work/dev/als/code_utilities.py", line 26, in wrapped
    result = func(*args, **kwargs)
  File "/home/deufrai/work/dev/als/als.py", line 294, in created
    stack_methode=self.stack_method)
  File "/home/deufrai/work/dev/als/code_utilities.py", line 26, in wrapped
    result = func(*args, **kwargs)
  File "/home/deufrai/work/dev/als/stack.py", line 228, in stack_live
    p, __ = al.find_transform(new[1], first_ref[1])
  File "/home/deufrai/work/dev/als/venv/lib/python3.6/site-packages/astroalign.py", line 240, in find_transform
    min_matches)
  File "/home/deufrai/work/dev/als/venv/lib/python3.6/site-packages/astroalign.py", line 418, in _ransac
    raise MaxIterError("Max iterations exceeded while trying to find acceptable transformation.")
astroalign.MaxIterError: Max iterations exceeded while trying to find acceptable transformation.

Expected result :
Image that cannot be aligned is ignored with an explicit message in GUI log

Use simple low frequency background estimation (gradient in french) and substract it.

For now, this is only logged.
URL should be displayed as long as the image server is running

Default language should be english for code readability.

app must detect user language at runtime and provide matching translation + fallback to English if not available

Wanted alternative language for now : French (fr)

Doing so, instead of saving after window is closed, gives us a chance to display a friendly warning box in the unlikely event of config.save() raising an exception

config.save() could be called from inside config._set() but this would force us to check for exceptions each time a setting is set. So we just save on window close

Since the switch from JPEG to PNG default file format for image save, we have a performance issue : as saving to PNG

• takes way more time
• is done in the GUI thread
• is done before even showing the image in the GUI
• is even done when not useful

GUI is frozen for several seconds each time a new stacked image is ready

We need to save newly stacked images only when necessary and do it in the background

utils/copyfits.sh doesn't deal with spaces in source filenames

Amend config.save so it simply raises an exception on save failure.
Client code will decide how to notify user and recover, if possible

Using this :
https://pypi.org/project/wifi-qrcode-generator/
we could easily share hotspot parameters during public sessions