leswright1977 / pyspectrometer Goto Github PK

I suggest adding some sort of averaging from several frames, so that sensor noise doesn't get captured as legitimate spectrum data. Ideally, user should be able to toggle this function on/off as necessary.

I have not looked at the code too much yet, only at a glance. Besides python is not my number one language, but I am willing to help you add something that does the averaging.

Also, if you accept it, I saw some room for improvement with the code.

Me: US based software engineer by trade.

ps: averaging can be done by grabbing data from several rows above and below the "main scan line", instead of waiting for several frames to accumulate.

Hello. I've just found your youtube channel Les. So many interesting videos. Inspired by your posts I've decided to try my hand at creating a Supercontinuum light source that you posted. I was going to buy a spectrometer but then noticed you have designed one using a Pi (but of course you have). I unfortunately am having trouble running the code. I'm somewhat a newbie on the Pi, so perhaps you might be able to see more easily what's wrong.

Here's my setup:
Raspberry Pi 4 running: Debian GNU/Linux 12 (bookworm).
Camera: Raspberry Pi HQ camera V1.0 - (I can run libcamera-hello with no issues except a warning about SDN tuning)
Display: PC Display: 4K display set to 2560 x 1440

Here are my traceback errors:
Traceback (most recent call last):
File "/home/rosst2019/PySpectrometer2/src/./PySpectrometer2-Picam2-v1.0.py", line 261, in
intensity = savitzky_golay(intensity,17,savpoly)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rosst2019/PySpectrometer2/src/specFunctions.py", line 127, in savitzky_golay
window_size = np.abs(np.int(window_size))
^^^^^^
File "/usr/lib/python3/dist-packages/numpy/init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'int'.

---------------------Here's my command line info if you need to see it.
rosst2019@raspberrypi:~/PySpectrometer2/src $ ./PySpectrometer2-Picam2-v1.0.py
[0:21:10.854209711] [3123] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
[0:21:10.901066323] [3129] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:21:10.903741029] [3129] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media4 and ISP device /dev/media2
[0:21:10.903831676] [3129] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:21:10.910147872] [3123] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
[0:21:10.954305816] [3132] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:21:10.956771543] [3132] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media4 and ISP device /dev/media2
[0:21:10.956853894] [3132] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:21:10.963521455] [3123] INFO Camera camera.cpp:1183 configuring streams: (0) 800x600-RGB888 (1) 2028x1520-SBGGR12_CSI2P
[0:21:10.964081226] [3132] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
Loading calibration data...
Loading of Calibration data failed (missing caldata.txt or corrupted data!
Loading placeholder data...
You MUST perform a Calibration to use this software!

Calculating second order polynomial...
2
3.125e-05 x + 0.4375 x + 380
Generating Wavelength Data!

Done! Note that calibration with only 3 wavelengths will not be accurate!
Traceback (most recent call last):
File "/home/rosst2019/PySpectrometer2/src/./PySpectrometer2-Picam2-v1.0.py", line 261, in
intensity = savitzky_golay(intensity,17,savpoly)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/rosst2019/PySpectrometer2/src/specFunctions.py", line 127, in savitzky_golay
window_size = np.abs(np.int(window_size))
^^^^^^
File "/usr/lib/python3/dist-packages/numpy/init.py", line 305, in getattr
raise AttributeError(former_attrs[attr])
AttributeError: module 'numpy' has no attribute 'int'.
np.int was a deprecated alias for the builtin int. To avoid this error in existing code, use int by itself. Doing this will not modify any behavior and is safe. When replacing np.int, you may wish to use e.g. np.int64 or np.int32 to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'inf'?

Hello,

is possible read (avarage) multiple rows?

Thank you

Hi Team, I am working on data transmission using Optical fibre where the center wavelength is in the near IR region. i.e. near 850nm, 1310nm or 1550nm. Kindly advise if the system can work for the light wave with centre wavelength mentioned as well.

First of all, great project!

Currently I'm running a similar setup as the one shown in your youtube video (using the same spectroscope).
I've tried a random camera lens I already had for the Raspberry HQ cam, but the resulting images are really blurry.

I don't know much about camera lenses, but I think you would need a lens with a very short focus length
to get an accurate image. Do you happen to know the technical specs of the lens you used? Much appreciated!

Sorry for opening issue here, I just want to know if I can read absorption through solution using your PySpectrometer?

Could you use this rig to do a demo of LIBS like the instruments on the Mars rovers? Are there low cost lasers capable of forming the plasma? I'm not sure if you can do any useful identification without the ultraviolet spectrum. This demo would be done in the context of high school physics.

It does appear to be possible to do UV imaging using the PI Cam.
https://www.mdpi.com/1424-8220/16/10/1649/pdf

Hello leswright 1977, I am an engineer from China; I currently have a laboratory project that needs to use a spectrometer to measure the near infrared spectrum. The spectral range is 400 to 1100nm;; Check the commercial spectrometer, it is too expensive; One is about 10000 RMB; So I want to DIY myself;

I watched your YouTube video, which only supports the spectrum of 400 to 700nm; I found a spectroscope on Taobao, and found that there was only a spectroscope with the wavelength of 400 to 700nm used to identify gemstones; This can not meet my needs, so I contacted you to inquire if there is a spectrometer that can measure 400nm to 1100nm to meet my project;

My email address is [email protected] , looking forward to your reply! A friend from China

I start with fresh image "Raspberry Pi Os with Desktop" (from 2021-03-04) and got:

pi@raspberrypi:~/PySpectrometer/src $ python3 pyspectrometer-v3.py
Traceback (most recent call last):
  File "pyspectrometer-v3.py", line 46, in <module>
    import PIL.Image, PIL.ImageTk
ModuleNotFoundError: No module named 'PIL.ImageTk'

It seems, that one dependency is missing - I had to add:
sudo apt-get install python3-pil.imagetk

Do you think we can add INDI support? This would open up the ability to use a plethora of astro cameras (some of which are VERY sensitive) with your software.

https://www.indilib.org/what-is-indi/discover-indi.html

Could you make a list of the Raspberry Pies compatible with this project?

Really awesome project!

This is huge (in terms of volume).
I think this would be much more useful as a tool, if it was much more compact (say if you could have this easily added to a trichorder etc).

Does anyone reading this have any idea how it could be made more compact?

The internet has some designs for compact spectrometers, but I would be interrested in something even more compact if possible. You can see one such design here for example: https://img.laserfocusworld.com/files/base/ebm/lfw/image/2016/01/1305lfw01f1.png?auto=format&w=720

Could a prism be used instead of the diffraction grating? That sounds like it could be made more compact then.
A very small motor could be used to move (rotate) the prism about if needed, there are minuscule stepper motors around, and I can help with finding and interfacing those if required.

The design of this project (PySpectrometer) looks a lot like this one, am I correct in my understanding there? https://www.hamamatsu.com/sp/ssd/product/Spectrometers/img06_en.png

Stacking mirrors also might have potential by increasing the number of reflections (still using only two mirrors) and therefore widening the prism/spectrum effect: https://media.springernature.com/m685/springer-static/image/art%3A10.1038%2Fs41467-018-06495-5/MediaObjects/41467_2018_6495_Fig1_HTML.png

Something like this has a similar concept to what I'm describing here : https://www.researchgate.net/profile/Manuel-Cano-Garcia/publication/338279737/figure/fig1/AS:845739789938690@1578651487502/Schematic-draw-of-the-proposed-spectrophotometer-An-Arduino-stage-drives-all-the.png but I would think it might be possible to make this even much more compact somehow ... just racking my brain for an engineering solution and have not found one yet. Curious if somebody else reading this would have an idea.

Maybe it could be possible to take advantage of the fact that we have a 2D sensor (instead of the strictly 1D sensor we actually need for spectra), by having several different prisms stacked on top of each other, creating different lines hitting the sensor on top of one another. This should be relatively easy to manufacture: it is simply a pile of flat panes of glass, on top of one another, with their entries and exists at different/varying angles (and presumably an opaque sheet of something between each layer). I can create a schematic of this if somebody is interrested and/or my description is not clear enough. It would also be very easy to make this very precise, by having two (or more) holes in each pane/plate/prism, and have a "pin" go through all of the holes in the stack for each series/stack of holes, thus ensuring perfect alignment of the prisms.
This way, you would get on your sensor, a series of spectrum lines on top of one another, giving you a 2D array of spectral points instead of a line/1D array of such.

I hope I'm not saying anything too stupid...

Maybe it's not stupid: I found an image that looks like that idea: https://www.osapublishing.org/getImage.cfm?img=dTcqLmxhcmdlLG9lLTI2LTE1LTE5NDU2LWcwMDE

The interresting thing/goal of what I describe here, would be that it would be very compact, which is the objective/what I created the issue for. But if anyone else has any idea of how to make things more compact, I would be very interested to hear about it (I need very compact spectrometers to identify materials on an automated recycling/sorting CNC machine)

Sorry for rambling about this, I hope I'm not disturbing anything, I'm just trying to get a conversation going.

Cheers!

Any suggestion if it is possible to use some cheaper version of Diffraction Spectroscope? Here are few samples
https://www.amazon.com/Handheld-Durable-Small-Diffraction-Spectroscope/dp/B00MNEAMF4
https://www.amazon.com/Gain-Express-Diffraction-Spectroscope-Identification/dp/B017VM4Q86

The only obvious issue - there are no details regarding diffraction grating of lines/mm.
Aliexpress price for them are around $25 - https://www.aliexpress.com/item/32625200308.html

Instead of RPI + camera it should be possible to use $5 USB webcam and attach it directly to PC https://www.aliexpress.com/item/1005001465072245.html
Another $5 USD for the lens with zoom https://www.aliexpress.com/item/4000617323132.html

I am looking for some cheap spectrometer version and initially found webcam + dvd piece + wood box version https://www.youtube.com/watch?v=MgogwcXUIoc&t=131s
I really like PySpectrometer since it looks great. Now I am trying to make a really cheap version but with reasonable quality.

Dear Les,

this is a great project. Thank you for sharing! I found it via hackaday.

I am thinking about if I could use it for astronomy. Because the light sources (stars) are very dim, a long time exposure would be needed. Is there an advantage if the HQ Pi Cam is used instead of the Standard Pi cam for this case?

I just leave my thought for a long time exposure here for discussion. I more people are interested, maybe someone could implement it. Otherwise just close this ticket.

Keep up the great work :)

Where did you buy the black piece that holds the spectroscope (its rectangular with a hole the right size for the spectroscope and with what looks like a brass screw for holding it in place)? It doesn't show it listed in the parts on the github page.

Hi Les, i cloned your nice spctrometer with the small and the big spectroscope. nearly no difference. i made a holder for the pi camera and the small spectrometer. It fits perfect for both, maybe you like to include this in your project:

https://www.tinkercad.com/things/fz8NYdaUxby

https://www.tinkercad.com/things/basXpPEWYCm

i like to build an closed case with battery driven pi, so i can also use it as portable version. i ordered a small 7" display with 1024*600 resolution. it will be nice to see an full screen version. i played a little bit with the code, but i´m not a programmer. i get the main frame and other frames bigger, but not the maingraph. maybe you can help me.

thanks a lot

I have to do a lot of color measurements on different RGBW led strips. It would be awesome to automate this process. It would be great if I could trigger a measurement externally with maybe a TCP of HTTP(S) command and get the measurements back as JSON (or some other format like comma delimited CSV like). Would you please consider this?

np.int is deprecated in bookworm. we need to use np.int32.

127,128c127,128
< window_size = np.abs(np.int32(window_size))
< order = np.abs(np.int32(order))
---
> window_size = np.abs(np.int(window_size))
> order = np.abs(np.int(order))

Hello!

Great project! I am just having a bit of an issue with openCV and GStreamer. Any suggestions would be much appreciated

[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1824) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Failed to allocate required memory.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (914) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (501) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

I just bought the Rpi 4 for wonderful pyspectrometer project.

Unfortunately my Rpi4 cannot boot itself with recommended 2021-01-11-raspios-buster-armhf.
The Legacy and 2021-03-04-raspios-buster does not allow to install scipy.
The computer gets unrensponsive so I have to make a hard reset.

Any help would be very appreciated!

Regards
Andrzej

Hello!

I found link to this project from https://hackaday.com/2021/04/23/pi-based-spectrometer-puts-the-complexity-in-the-software/

Could you please add information about license of this repository — is it allowed to copy, change and use the code and images in any purpose?
It would be great to see any popular open source license.

Hi, great project, congratulations, could you please let me know which Raspberry Pi 4 version did you use for the project? How much memory? What is the average memory utilization when running the software?

When installing on a zero 2 W, the scipy install dies on numpy. It's a memory issue, since you only have 300 some MB. You'll need to increase the swapfile size, at least temporarily. Alternatively install the dependencies on a full size Pi4 and transfer the folders.

also don't forget to run apt get update.

I followed the instructions at:
https://pimylifeup.com/raspberry-pi-swap-file/

to save some clicks, here's the terminal commands

sudo dphys-swapfile swapoff
sudo nano /etc/dphys-swapfile
ctrl-w
CONF_SWAPSIZE=100
---change 100 to 1024 or 2048. I wasted enough time last night. I changed it to 2048 and it finished in about an hour.
ctrl-x
y
sudo dphys-swapfile setup
sudo dphys-swapfile swapon
----then reboot
sudo reboot

This project currently can't be cloned onto a Windows machine due to the colons in the filename of this image:
media/spectrum-09-04-2021-15:19:27.jpg

Renaming the file to something like spectrum-09-04-2021-15-19-27.jpg will solve this issue.

Hi. how far into the UV and IR does the raspberry PI camera go? If any?