Text to Sign Language: A Communication System based on RAG

This talk demonstrates a text-to-sign language system converting Bangla text to avatar videos, showcasing a live workflow, evaluation, and a working demo.

FastAPI JavaScript PyTorch RAG Computer Vision

Overview

A text to sign language generation platform that converts typed bangla into a signing avatar video, running through one FastAPI orchestrator with a vanilla-JS browser frontend.
We will show case a live workflow, eval, and a working demo system.

Tech stack

FastAPI

FastAPI is a modern, high-performance Python web framework for building APIs with automatic OpenAPI documentation.

FastAPI is a robust, high-speed Python web framework: it is built on Starlette (for async capabilities) and Pydantic (for data validation and serialization). Leveraging standard Python 3.8+ type hints, the framework automatically generates interactive API documentation (Swagger UI/ReDoc) and enforces data validation, effectively reducing developer-induced errors by an estimated 40%. This architecture delivers performance on par with Node.js and Go, significantly increasing feature development speed (up to 300% faster). It is production-ready, fully supporting OpenAPI and JSON Schema standards for all API specifications.

https://fastapi.tiangolo.com

View projects
JavaScript

A zero-dependency architecture using native browser APIs to deliver sub-100ms load times on static hosting.

This approach replaces heavy frameworks with native DOM manipulation and ES6+ modules. By deploying to global CDNs like Vercel or Netlify, you bypass server-side processing and reduce total bundle sizes by over 90%. It is the optimal choice for high-traffic landing pages or internal tools where speed, security, and long-term maintainability are the primary requirements.

https://developer.mozilla.org/en-US/docs/Web/JavaScript

View projects
PyTorch

PyTorch is the open-source machine learning framework: it provides a Python-first tensor library with strong GPU acceleration and a dynamic computation graph for building deep neural networks.

PyTorch, developed by Meta AI, is a premier open-source deep learning framework favored in both research and production environments. Its core is a powerful tensor library (like NumPy) optimized for GPU acceleration, delivering 50x or greater speedups for complex computations. The key differentiator is its 'Pythonic' design and dynamic computation graph (eager execution), which allows for rapid prototyping and simplified debugging compared to static-graph frameworks. Leveraging its Autograd system for automatic differentiation, practitioners build and train models for computer vision and NLP; major companies like Tesla (Autopilot) and Microsoft utilize PyTorch for critical AI applications.

https://pytorch.org

View projects
RAG

RAG (Retrieval-Augmented Generation) is the GenAI framework that grounds LLMs (like GPT-4) on external, verified data, drastically reducing model hallucinations and providing verifiable sources.

RAG is a critical GenAI architecture: it solves the LLM 'hallucination' problem by inserting a retrieval step before generation. A user query is vectorized, then used to query an external knowledge base (e.g., a Pinecone vector database) for relevant document chunks (typically 512-token segments). These retrieved facts augment the original prompt, providing the LLM (e.g., Gemini or Llama 3) the specific, current, or proprietary context required. This process ensures the final response is accurate and grounded in domain-specific data, avoiding the high cost and latency of full model retraining.

https://en.wikipedia.org/wiki/Retrieval-augmented_generation

View projects
Computer Vision

Computer Vision (CV) uses deep learning and neural networks (e.g., CNNs) to enable machines to interpret visual data: identifying objects, people, and patterns with high accuracy.

CV is a core AI subfield that actively replicates human sight, processing raw images and video via deep learning models to extract meaningful, actionable data. It is critical for major industries: powering object detection in autonomous vehicles, enabling defect detection in manufacturing quality control, and assisting with diagnostics in medical imaging. This technology drives significant operational efficiency and safety across sectors, with the overall market expected to reach $58.29 billion by 2030 (Grand View Research). We leverage CV to automate tasks that traditionally required human visual inspection.

https://azure.microsoft.com/en-us/resources/cloud-computing-dictionary/what-is-computer-vision/

View projects

Text to Sign Language: A Communication System based on RAG

Related talks