This is the official repository and iOS implementation of the role-based mobile applications for attendance checking using facial recognition, UHF RFID and NFC described in the papers "AttendanceKit: ..." in FDSE 2022 and "To Wrap, or Not to Wrap: ..." in SN Computer Science • Volume 4, 729 (2023).

This research was funded by Vietnam National University, Ho Chi Minh City (VNU-HCM) under grant number C2022-28-10 (level C) and is on-going with another grant in level B. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and should not be attributed to their employers or funding sources.

1. Summary
2. News
3. Applications
4. Compatibility
5. Dependencies
6. Directory structure
7. Pre-trained models
8. Training data
9. Performance
10. Authors
11. Reference
12. License

The research project proposes an "AttendanceKit: ..." system that uses real-time Ultra-High Frequency (UHF) RFID and NFC technology combined with face recognition to automatically check students' attendance in offline classes, packaged as a suite of mobile applications for Institution, Lecturers and Students to overcoming the disadvantages of manual inspection.

The backend system will send real-time notifications to students' mobile devices, allowing them to verify attendance. It also features attendance monitoring, allowing instructors to evaluate student status. The system can also compile reports on student learning status, providing insights for lecturers, parents, and institutions.

The diagram depicted above demonstrates the procedural flow of the Custom Admin Module, which addresses Firebase Authentication's limitation by allowing one end-user (Institution) to create accounts for other end-users (Student) using a Node.js module of custom functions.

Our system also takes time and accuracy into account. In addition, the results present a complete performance study of the system with RFID, NFC and genuine mobile devices, as well as a novel machine learning platform that can be deployed on actual devices in reality for commercial.

And also the article "To Wrap, or Not to Wrap: ..." explores the implementation approaches of face recognition models on mobile devices, focusing on their performance and precision. It explores the advantages and disadvantages of the TensorFlow and Core ML model encoding approach, and its impact on the overall success of the AttendanceKit system.

Contribution of this work:

• A set of macOS and iOS role-based attendance checking usable and deployable applications for Apple platforms.
• Assess the performance of the learning outcomes to illustrate the utility of automatic RFID, NFC in improving the quality of learning.
• Custom admin module to modify the behavior of Firebase Authentication, allows an end-user to create accounts for other end-users.
• Algorithms fed with a series of 5-second-long videos containing the face samples of students and and how to optimize the recognition process in real-time on mobile devices's camera.
• Leverage the CNN FaceNet model, implemented in TensorFlow's and how to convert to the native Core ML .mlmodel format prior. Examining the approaches of Create ML, turicreate and coremltools.
• Determined timing and precision by comparing the analysis on the two mobile platforms, and the benefits and drawbacks of each model implementation method (native framework, web API, model wrapping, or model converting) to have a clear picture of which strategy to employ for similar systems.

There are many items in the AttendanceKit set of applications, including: Student, Institution, Lecturer (iOS), and RFID Dashboard (macOS).

Student	Institution	Lecturer	RFID Dashboard
Face recognition is used as biometric security for all students when checking attendance.	To collect student face samples, class scheduling, assign tags, and classrooms.	For lecturers to view attendance reports and change schedules.	For connecting to RFID antennas to read and display tag IDs, act as admin application to compose notifications for mobile devices.

The face recognition module of these applications is heavily inspired by the project enVision from ID Labs.

The code is tested using TensorFlow 1.7 and Core ML 3.0 under iOS 15.0, macOS 12.0 with Swift 5.1, Java 16.0 and Python 3.5.

IMPORTANT: The project must be built with Xcode on a macOS device. While the RFID Dashboard desktop middleware app can be built with IntelliJ.

This project is written in Swift, Objective-C, Objective-C++, Java and Python. Dependencies include:

CocoaPods is a dependency manager for Cocoa projects. If you don't have CocoaPods installed, install it by using this command:

$ sudo gem install cocoapods

To integrate all of the necessary pods into the Xcode workspace using CocoaPods, specify it in your Podfile:

platform :ios, '15.0'
use_frameworks!

workspace 'AttendanceKit'

def pods
  pod 'TensorFlow-experimental'
  pod 'FaceCropper'
  pod 'Alamofire'
  pod 'RealmSwift'
  pod 'Firebase/Core'
  pod 'Firebase/Database'
  pod 'Firebase/Storage'
  pod 'SDWebImage'
  pod 'RxSwift'
  pod 'RxCocoa'
end

target 'General' do
  project 'General'
  pods
end

target 'Student' do
  project 'Student'
  pods
end

target 'Institution' do
  project 'Institution'
  pods
end

target 'Lecturer' do
  project 'Lecturer'
  pods
end

To create the AttendanceKit.xcworkspace, run the following commands in Terminal. Replace <project_folder> with your cloned project root folder:

$ cd /Users/<project_folder>
$ pod install

The macOS RFID Dashboard application uses Gradle as it's project build automation tool. Refresh the dependencies of the dashboard.idea project by running the following command:

$ gradle --refresh-dependencies clean build

IMPORTANT: Download the FaceNet TensorFlow model ~87.4 MB from here and place it into the ML group folders of all .xcodeproj in the AttendanceKit.xcworkspace.

The directory should look like this:

┌── Institution/
  ┌── ML/
    ┌── facenet.pb
    ├── tensorflow_utils.h
    ├── ...
    ├── tfWrap.h
    └── tfWrap.mm
  ├── ...
  └── View/
  
├── Lecturer/
  ┌── ML/
    ┌── facenet.pb
    ├── tensorflow_utils.h
    ├── ...
    ├── tfWrap.h
    └── tfWrap.mm
  ├── ...
  └── View/

└── Student/
  ┌── ML/
    ┌── facenet.pb
    ├── tensorflow_utils.h
    ├── ...
    ├── tfWrap.h
    └── tfWrap.mm
  ├── ...
  └── View/

NOTE: If you use any of the models, please do not forget to give proper credit to me, the FaceNet authors and those providing the training dataset as well.

The CASIA-WebFace dataset has been used for training. This training set consists of total of 453.453 images over 10.575 identities after face detection. Some performance improvement has been seen if the dataset has been filtered before training. Some more information about how this was done will come later. The best performing model has been trained on the VGGFace2 dataset consisting of ~3.3M faces and ~9.000 classes.

The accuracy on LFW for the model facenet.pb is 0.99650±0.00252. A description of how to run the test can be found on the page Validate on LFW.

NOTE: The input images to the model need to be standardized using fixed image standardization (use the option --use_fixed_image_standardization when running e.g. validate_on_lfw.py).

The main contributors include me and my honorable mentor and supervisor, as the principal and main co-investigator of the granted projects:

• Trung-Dung Tran BSc., [email protected]
• Dr. Tu-Nga Ly Ph.D, [email protected]

• Assoc. Prof. Kha-Tu Huynh Ph.D, [email protected]
• Quang-Phu Nguyen MSc., [email protected]
• Cong-Kha Vo BSc., [email protected]

To cite the papers, please use these BibTex:

@inproceedings{tran2022attendancekit,
  title={A set of Role-Based Mobile Applications for Automatic Attendance Checking with UHF RFID Using Realtime Firebase and Face Recognition},
  author={Tran, Trung-Dung and Huynh, Kha-Tu and Nguyen, Phu-Quang and Ly, Tu-Nga},
  booktitle={International Conference on Future Data and Security Engineering},
  pages={432--446},
  year={2022},
  organization={Springer}
}

@article{tran2023wrap,
  title={To Wrap, or Not to Wrap: Examining the Distinctions Between Model Implementations of Face Recognition on Mobile Devices in an Automatic Attendance System},
  author={Tran, Trung-Dung and Ly, Tu-Nga},
  journal={SN Computer Science},
  volume={4},
  number={6},
  pages={729},
  year={2023},
  publisher={Springer}
}

AttendanceKit is open-sourced under the CC0-1.0 license. See LICENSE for more details.