Signs: an interactive platform for learning and contributing to American Sign Language fueled by AI technology

General Introduction

Signs is an innovative online platform designed to help users learn American Sign Language (ASL) and contribute to the Deaf community through AI technology. Developed in partnership with NVIDIA, the American Society for Deaf Children (ASDC), and the creative agency Hello Monday/DEPT®, the site combines the concepts of AI-driven real-time feedback and community collaboration. Whether you're a beginner or a skilled sign language user, you can learn ASL interactively through the platform or upload sign language videos to enrich its open source dataset. It not only provides a learning tool, but also strives to build a high-quality ASL video resource library to support the development of more accessible technologies and connect the communication bridge between the deaf and the hearing population. Currently the platform focuses on hand motion capture, and in the future it plans to add recognition of non-hand signals such as facial expressions.

Function List

• ASL Learning Module: Provides instruction in basic sign language vocabulary, and users can watch demonstrations through 3D avatars.
• Real-time feedback system: Use cameras and AI to analyze user gestures and provide instant corrective suggestions.
• Video Contribution Function: Allow users to upload sign language videos to assist in expanding the platform's ASL dataset.
• Open dataset support: Provide developers with validated ASL video resources for AI application development.
• Community Interaction: Connecting learners and the deaf community to facilitate knowledge sharing and cultural exchange.

Using Help

Signs is an online platform that requires no installation to use, simply visit https://signs-ai.com/即可开始体验 via your browser. Below is a detailed user guide to help you get started quickly and take full advantage of its features.

How to get started

1. Access to the website
   Open your browser and enter the URL https://signs-ai.com/ to get to the home page. You will see two main options: Learn and Contribute. You can view some of the content without registering, but it is recommended that you create an account to unlock the full functionality.
2. Create an account
   Click the "Sign Up" button in the upper right corner and enter your email address and password to complete your registration. Once registered, you can log in to save your progress and upload videos.
3. Equipment Requirements
   Make sure your device is equipped with a camera (for real-time feedback) and a stable internet connection. A desktop or laptop computer is recommended for the best experience, cell phones can be used but the smaller screen may interfere with operation.

Main function operation flow

1. Learning American Sign Language (Learn module)

• Go to the learning page
  Click "Learn" on the homepage to enter the sign language learning interface. Common ASL words such as "hello" and "family" are shown here.
• View 3D Headshot Demonstration
  Select a vocabulary word and the system displays a 3D avatar demonstrating the gesture movements for that sign language. You can watch from different angles to make sure you understand the details of each hand pose.
• Practice and get feedback
  Turn on the camera, point it at your hands, and mimic the gestures of your 3D avatar. the AI analyzes your movements in real time and displays feedback on the screen. For example, if your finger position is incorrect, a prompt will appear: "Please raise your little finger" or "Your palm should face left". We recommend practicing in a quiet, well-lit environment to improve recognition accuracy.
• advanced study
  The platform offers a graded learning model that progressively increases in difficulty from beginner vocabulary to complex sentences. After completing each stage, more content can be unlocked for long-term learners.

2. Contribute videos in sign language (Contribute module)

• Go to the Contribute page
  Click on "Contribute" on the homepage to enter the video upload screen. Here you will find a list of words that are currently required by the platform, such as "love" or "school".
• Record Video
  Click the "Record" button next to the target vocabulary word and the system will activate the camera. Follow the prompts to make the corresponding sign language movements and record a 3-5 second video. After recording, you can preview the video to make sure the action is clearly visible.
• Submission and validation
  Click "Submit" to upload your video. All submissions are reviewed by Deaf and certified interpreters for accuracy. Once reviewed, your video will be added to the dataset and you will receive a thank you note.
• caveat
  Keep the background simple when recording, avoid hand obscuration, and make sure the movements are ASL compliant. The platform will provide demonstration videos for reference.

3. In-depth use of the real-time feedback system

• Adjusting the camera angle
  Make sure both hands are in the center of the frame when learning or recording. If the AI recognition fails, adjust the camera height or distance, usually keeping 50-80 cm is appropriate.
• Understanding feedback suggestions
  Feedback is not limited to text, but sometimes visual markers in green (correct) or red (incorrect) are displayed. Repeat the exercise until the green light comes on, indicating that the gesture is completely correct.
• Optimize experience
  If the feedback is delayed, check the network speed or reduce the screen resolution. The platform supports manual switching of sharpness to adapt to different device performance.

Featured Functions Operation Details

• 3D Avatar Interactive
  Click on the avatar to pause, rotate or zoom in for a closer look at the details of the gesture. Beginners can imitate gradually with the slow play function.
• Significance of data set contributions
  Each video upload helped the platform accumulate data, culminating in a target database containing 400,000 video clips. This not only supports individual learning, but also advances the application of AI technology in the field of deafness, such as videoconferencing sign language recognition.
• Community involvement
  The site is regularly updated with contribution rankings to showcase active users. You can also exchange learning experiences or make suggestions with others in the forum area.

Tips for use

• Advice for beginners
  Start with basic vocabulary and practice for 10 minutes a day, gradually familiarizing yourself with the sign patterns.
• Video Uploader Suggestions
  Check the demonstration video before recording to ensure that the movements are standardized and to avoid repetitive modifications.
• Technical Support
  If you have any questions, you can send an email to "Contact Us" at the bottom of the website and the team will respond within 48 hours.

With the above steps, you can easily master the core functions of Signs-ai to improve your ASL skills and contribute to the community at the same time. The website is simple and intuitive for users of all ages, especially for families of children with deafness or sign language enthusiasts, and is an unparalleled platform for learning and sharing.