ASL Sign Language Detection Using CNN

Introduction

This project is an implementation of a Convolutional Neural Network (CNN) using TensorFlow and Keras to detect American Sign Language (ASL) letters. The project includes creating a custom dataset, training a CNN model, and developing a real-time detection application.

Prerequisites

Ensure you have Python installed on your system. You can download it from python.org.

Installation

Clone the Repository

git clone https://github.com/CBJdereal/SignLanguageDetectionCNN.git
cd SignLanguageDetectionCNN

Install Required Libraries

Create a requirements.txt file in the src directory with the following contents:

tensorflow
keras
opencv-contrib-python
split-folders

Install the libraries using pip:

pip install -r ./src/requirements.txt

Dataset Preparation

Collecting Data

Run collectdata.py to collect images for the dataset. This script initializes your webcam and allows you to capture images of ASL signs, saving them in a structured directory format (SignImage48x48) with subfolders for each letter and a special folder for variable background images (blank).

python ./src/collectdata.py

Data Splitting

After collecting the data, use split.py to divide the dataset into training and validation sets.

python ./src/split.py

Model Training

Setup Google Colab

Upload the Jupyter notebook trainmodel.ipynb to Google Colab and follow the instructions to mount your Google Drive for model saving and dataset access.

Run Training

Execute the cells in the notebook to train the model. The notebook includes detailed steps for configuring the model, initiating training, and monitoring performance metrics like accuracy and loss.

Epoch Accuracy

Epoch Loss

Model Export

After training, the model is saved as JSON and .h5 files, which you should download to your project directory from your Google Drive.

Real-Time Detection

Setup Detection Script

Modify realtimedetection.py to update the model path and ensure all paths are correctly set relative to your project structure.

python ./src/realtimedetection.py

Run Detection

Execute the detection script to start real-time sign language detection using your webcam.

Troubleshooting

If you encounter issues, you may need to adjust TensorFlow and Keras versions. Here are the versions that were found to be compatible:

• TensorFlow: 2.12
• Keras: Version corresponding to TensorFlow 2.12 compatibility

Expected Output

Ignore my face sorry i was too tired after fixing a bug in the program