There’s a Large Hippo Resting in the Mud

© Bendel / Jovic

Inclusive AI is a young field of research as well as an application domain where AI systems of various kinds are used to support individuals with disabilities and impairments, enabling their participation in different aspects of life. Generative AI plays an important role in this context, particularly in the form of multimodal large language models (MLLMs). These models have also found their way into Animal-Computer Interaction (ACI), for instance in apps that provide behavioral guidance to users encountering animals. In “The VISUAL Project” at the FHNW School of Business, both disciplines were combined to unlock new potential. 

The resulting prototype allows blind and visually impaired individuals to embark on virtual video safaris. It integrates publicly available wildlife webcams from various regions and uses an MLLM to analyze and evaluate the live footage. Different profiles and modes are available to meet various needs. A text-to-speech system delivers descriptions and explanations. The prototype demonstrates the technical feasibility and potential of Inclusive AI and ACI. However, limitations must be considered, such as those associated with commercial MLLMs and occasionally inaccurate descriptions. Further research and development are required to make the experience truly inclusive and autonomous.