does not pick up ir camera when they're two separate devices about howdy HOT 44 CLOSED

Normally during setup you get asked what camera to use by testing them until your emitters turn on.

The library gets a frame from one of your webcams and returns the faces it detects in that frame. Howdy only sends it the frame from the configured /dev/videoDEVICE_ID device (should be 0 in your case). Every working Howdy setup has 2 separate webcams though, this is normal.

I suggest setting your device_id to 0 in sudo howdy config and running sudo howdy test to see what's going wrong. You should see a red circle around your face.

from howdy.

@boltgolt I am on arch, so I just cloned your repo, have had no setup done though.

Have already looked into howdy test. If on device 0 the ir camera picks up a picture (so the green colored one). if on device 2 it picks up the webcam. However on the IR it just does not recognize any faces, but actually if you do not go 2cm before the IR cam you do not really see that much, so that would be expected (you can barely see something by eye but its there though). It would be very hard to find a face in that scenario.

from howdy.

v4l2-ctl --list-devices --all
Integrated IR Camera: Integrate (usb-0000:00:14.0-5):
	/dev/video0
	/dev/video1

Integrated Camera: Integrated C (usb-0000:00:14.0-8):
	/dev/video2
	/dev/video3

Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : Integrated IR Camera: Integrate
	Bus info      : usb-0000:00:14.0-5
	Driver version: 4.16.8
	Capabilities  : 0x84A00001
		Video Capture
		Metadata Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
	Width/Height      : 640/480
	Pixel Format      : 'YUYV'
	Field             : None
	Bytes per Line    : 1280
	Size Image        : 614400
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Limited Range)
	Flags             : 
Crop Capability Video Capture:
	Bounds      : Left 0, Top 0, Width 640, Height 480
	Default     : Left 0, Top 0, Width 640, Height 480
	Pixel Aspect: 1/1
Selection: crop_default, Left 0, Top 0, Width 640, Height 480
Selection: crop_bounds, Left 0, Top 0, Width 640, Height 480
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 30.000 (30/1)
	Read buffers     : 0

from howdy.

Could you try flipping force_mjpeg to true? IR images tend to be black and while, not green.

I'd love to have more builds for other distros besides ubuntu/debian on here, so if you can get it to work there might be a way to package it.

from howdy.

Both video 0 and video 1 are your IR camera? That's very peculiar

from howdy.

Yes have a lenovo t580 here, this has 3 lenses. 1 webcam and the ir with 2, dunno why both webcam and IR create two devices though

from howdy.

video1 and video 3 are not functional though

from howdy.

flipping force_mjpeg does not change anything though :/

from howdy.

And also strange is, the windows hello recogniation lets the two red leds blink rapidly of the IR, here they just blink up once.

Windows hello works fine with the cam, but I do not use windows so that doesn't helps me :D

from howdy.

The emitters should always blink when the IR camera is on because otherwise there won't be enough light to see anything. Sounds like that's what's happening with yours.

There has been an issue on this before though, #19. I seem to have closed that one really quickly but there's not much that can be done here. The emitters aren't controllable directly.

from howdy.

That could be the reason yeah

from howdy.

I just checked out the internal settings of uvc and it turns out the IR has two available resolutions detected. however only the 340x340 works as expected. by default it picked up the 640x480

from howdy.

The question is now how to tell the script that :D

from howdy.

height setting does nothing though

from howdy.

Could you try adding

video_capture.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
video_capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

on line 22 of cli/test.py?

from howdy.

Yup just tested but with 340x340 the one that works and it turns out working after that.

from howdy.

Even the emitters turn on?

from howdy.

yup, and no greenish pic

from howdy.

Very peculiar, i'll add the height and width as a setting in the next release. Do you have any experience with packaging for arch by any chance?

from howdy.

that is quite simple yes

https://wiki.archlinux.org/index.php/makepkg

pkgname=howdy
pkgver=1.0.0
pkgrel=1
pkgdesc='A IR supporting face recognition PAM'
url='https://github.com/Boltgolt/howdy'
arch=('any')
license=('MIT')
source=("https://github.com/Boltgolt/howdy/releases/download/v${pkgver}/howdy-${pkgver}-x86_64.tar.gz"
)
sha256sums=('hash')

And adding some install routines.

from howdy.

I guess your setup for debian is interactive? arch packages (the installation) are usually not.

from howdy.

So that setup would rather be a note being printed after installation to the user that he needs to execute some script to configure.

from howdy.

Yeah, unfortunately Howdy is far from a straight forward installation. The user would need to install python modules, copy files to specific locations, add lines in the PAM and set their config. Might just need to add a sudo howdy unpack command or something.

from howdy.

except maybe for the PAM lines the rest should be easy cake.

from howdy.

I'm looking into pam-auth-update anyway so the /etc/pam.d/common-auth doesn't need to be touched anymore.

from howdy.

ok cool I am just on my way setting the whole stuff up by hand now that it recognizes stuff, will help you on the makepkg if you need help though.

from howdy.

That'd be great! My biggest issue is that i don't have an arch environment to test and build, so might have to look into a travis build there.

from howdy.

since arch is the most raw experience of all distros except gentoo, that shouldn't be that hard :D just use docker for testing purposes
https://hub.docker.com/r/base/archlinux/

from howdy.

only trouble left is the missing pam_python figuring it out :)

from howdy.

https://aur.archlinux.org/packages/python2-pypam/? pam.py runs in python 2 anyway.

from howdy.

seems to be working now, need to integrate it with gdm better.

from howdy.

and pypam needed two patches to compile under the newest gcc

from howdy.

Have it fully working now, also I will make some adjustments b/c I do not want the IR to work without a password when logging in, but it would be fine when unlocking. You don't know by any chance if PAM is capable doing that? Otherwise I will do that with an script in the middle that can recognize this state.

from howdy.

PAM doesn't know what context is calling except for the type (session, auth, etc). You could try disabling pam_sm_authenticate or pam_sm_open_session in pam.py and see if that works but it don't think it will.

from howdy.

As for "python2-pypam" (which is a terrible name for pam-python in my opinion), we could probably ship a patched version if needed, or maybe push it to AUR?

from howdy.

That is probably due to the fact that pam-python is the name of the package that is the reverse. An interface from python to pam.

from howdy.

Concerning the PAM I will have a look if there are any capabilities to have mutliple alternative paths for authenticating. And then checking with loginctl if there is already a user session spawned.

from howdy.

ok there are some returning a number instead of done, ok, irgnore... will make PAM jump over the next modules.

from howdy.

Just quickly thrown in and connected my yubi key:

auth     [ success=ok new_authtok_reqd=ok ignore=ignore default=1 ] pam_u2f.so
auth	   sufficient		pam_python.so /usr/local/lib/howdy/pam.py
auth     include   system-local-login
auth     optional  pam_gnome_keyring.so

That works fine :) So doing the same thing detecting an existent session should be just fine.

from howdy.

Am a bit disappointed about PAM since it is so super limited. Not even parallelism is possible.

So it works, that is at least enough right now. And thank you for putting your work open source :) @boltgolt

from howdy.

Yeah PAM is slightly outdated, at least you got it working

from howdy.

So, is there any config to update for IR on device1 in addition to camera on device0. I have Dell with camera and IR separate. On Windows 10 I don't have problem logging in at night cause that uses IR (night) + webcam.

from howdy.

I created a small script and udev rule that switches the Howdy device depending on the available camera- or disables Howdy alltogether: https://github.com/tuhlmann/select-howdy-source

from howdy.

Thanks @tuhlmann, i've seen a few people ask for exactly this!

from howdy.