Not Searching, But Finding
WhereIsIt as a Memory Aid for Blind and Visually Impaired People
Introduction
The keys are on the dresser, the wallet is on the kitchen table, and the commitment ring is on the nightstand. What seems straightforward when putting something down can become a problem when looking for it later. Blind and visually impaired people often rely on designated storage locations and established routines. If they change or do not follow these routines, or if someone else moves an item, these strategies reach their limits. The WhereIsIt project at the FHNW School of Business addresses this everyday challenge. It investigates how a simple voice-based assistant can record the names of objects and the locations where they are placed and provide this information on request (Huckele 2026).
The project falls within the field of Inclusive AI (Avellan et al. 2020; Shams et al. 2026) – “AI” stands for “artificial intelligence”. One aim is to counter exclusionary phenomena in artificial intelligence, such as bias and hallucinations. Another is to strengthen inclusive applications that help people with disabilities and those who are disadvantaged or excluded (Bendel 2025). Inclusive AI can encompass generative AI as well as other forms of artificial intelligence. What matters is that the people concerned benefit from the applications and retain control over their data and the functions. The broader approach of inclusion aims to include people, reduce disadvantage, and promote equal opportunity (Bendel 2021).
The FHNW School of Business has already investigated various possibilities in this field. The study “How Can Generative AI Enhance the Well-being of Blind?” tested the Be My AI feature of the Be My Eyes app (Bendel 2024). Using a multimodal large language model (MLLM), this feature can analyze and evaluate photos and describe rooms, objects, animals, and people, among other things. The tests demonstrated its potential to improve awareness of the surrounding environment and increase independence. They also revealed recognition errors and restrictions imposed by the provider (Bendel 2024). For example, a tall window was mistaken for a door, and access to fine art in an illustrated book was denied simply because it was erotic. In the VISUAL project, a prototype, also based on an MLLM, analyzes images from publicly available wildlife webcams and delivers the descriptions through speech output. This makes virtual video safaris accessible to blind and visually impaired people (Bendel and Jovic 2025).
WhereIsIt extends these approaches by providing a memory aid for everyday life. It focuses on how information about objects and their storage locations can be saved and retrieved with little effort. Oliver Bendel initiated and led the project, and Damian Huckele worked on it as a project team member. The project was carried out from March to August 2026 as part of a final thesis at the FHNW School of Business (Huckele 2026). It produced a functional prototype in the form of a browser-based application.
From Everyday Experience to Requirements
At the start of the project, the project team member reviewed the literature on assistive technologies and developed requirements for the system. An interview was also conducted with Steve Weidel, the founder of Inklutec, a company specializing in digital accessibility. Blind since birth, he contributed both everyday experience and software development expertise. He described designated storage locations as an important aid to orientation. Difficulties arise particularly when an object is no longer where it is expected to be. Keys, wallets, jewelry, documents, and assistive devices are among the items whose retrieval can take time and cause stress (Huckele 2026).
These foundations led to a deliberately manageable concept. The user should be able to verbally specify an object and a storage location and later retrieve the information with a simple question. Complicated menus and rigid command sequences would reduce its usefulness. Clear feedback, a predictable process, and the ability to correct incorrect input are important. Privacy and local processing were also included as central requirements. The system should fit into existing routines and require as little additional effort as possible (Huckele 2026).
WhereIsIt thus reflects the application- and solution-oriented meaning of Inclusive AI. Forms of artificial intelligence are used to compensate for a specific limitation in access to relevant information. Development focuses on a clearly defined task and the needs of the intended users. At the same time, the other side of the concept becomes apparent: even an application intended to help can create new barriers if it processes speech incorrectly or is not sufficiently accessible to use (Bendel 2025; Huckele 2026).
Recording and Retrieving Storage Locations
The basic use can be illustrated with a set of keys. The user puts them down, activates recording, and tells the system where they are, for example by saying, “My keys are on the kitchen table!”. WhereIsIt processes the statement, identifies the object and its storage location, and saves the association. In response to a later question such as “Where are my keys?” it provides the most recently saved location through speech output. The documented prototype uses an English output voice as a standard (Huckele 2026).
In addition to saving and retrieving information, the prototype supports updating and deleting entries. Up to three storage locations can be saved for an object. If another is added, the oldest is removed. The data remain available even after the application is closed and restarted. If no storage location has been saved for an object, this gap should be addressed constructively. In the corresponding tests, this worked reliably: WhereIsIt pointed out the missing information instead of making something up (Huckele 2026).
It is important to distinguish between a practical memory aid and automatic location tracking. WhereIsIt obtains its information from what the user tells it. It does not observe where the user places an item and does not detect when another person subsequently moves it. Its answer therefore identifies the most recently reported storage location. An extension using Bluetooth Low Energy tags was discussed in the project but not implemented. In the future, such technologies could, for example, trigger an audible signal to help locate an item (Huckele 2026).
Speech Processing on the User’s Own Computer
The prototype was developed as a browser-based application (see Figure 1). A local Flask server connects the user interface to the individual processing components. Speech input is converted into text using Faster Whisper (also written as “Faster-Whisper” and “faster-whisper”). A local language model running through Ollama then identifies the user’s intent, the object, and, where applicable, its storage location. The information is saved in or retrieved from an SQLite database. Piper converts the response into speech (Huckele 2026).
The AI system performs a limited task within a transparent workflow. The language model is intended to identify whether the user wants to save, retrieve, update, or delete something and provide the necessary information in a structured form. The saved names of objects and storage locations come from the user’s input. If the model’s output cannot be interpreted, the user is asked to repeat or clarify the input. The prototype is designed to handle one object–location pair per interaction (Huckele 2026).
Recording is explicitly started and stopped, for example using a button or the Enter key. If it is not stopped manually, it ends after ten seconds. The system is therefore not designed to listen continuously. Temporary voice recordings are deleted after transcription. Once the necessary components have been installed, the system does not require an internet connection to operate. Voice recordings, transcribed requests, and saved objects and storage locations are not transmitted to external services (Huckele 2026).
The interface supplements speech input and output with scalable text, a high-contrast mode, and clearly labeled controls. Semantic HTML and screen reader support are intended to improve accessibility. These measures address different degrees of visual impairment. However, the local browser application is not yet a fully developed mobile app. A portable implementation for use in different rooms and outside the home remains a task for further development (Huckele 2026).
What the Tests Show
More from Dr. Bendel and his Team
The prototype was tested with 12 people ages 15 to 80 using 16 tasks. Five participants had no visual impairment, five had mild visual impairment, and two had severe visual impairment. None of the participants was blind. The tasks included saving and retrieving storage locations, handling incomplete information, deleting entries, and operating the system without visual information (Huckele 2026).
Of 192 test cases, 162 were completed successfully, corresponding to 84.4 percent. A task was considered successful even if it was completed with assistance. Minor assistance was required in 39 cases, and more extensive intervention in 15 cases. The success rate therefore does not demonstrate consistently independent use. Functions that worked well included starting and stopping recording and indicating that no storage location was available for an unknown object whose name had not been saved (Huckele 2026).
More complex workflows performed less well. Retrieving earlier storage locations and the task involving operation without visual information each achieved a success rate of 58.3 percent. Speech recognition errors also led to incorrect database entries. Longer statements and requests mentioning multiple objects could not be processed reliably. These results point to specific improvements: asking users to confirm recognized information before it is saved, making error messages more precise, and making corrections easier. The tests demonstrate technical feasibility and provide initial indications of usability. They do not yet establish reliable suitability for everyday use by blind people (Huckele 2026).
Opportunities Offered by WhereIsIt
WhereIsIt can promote independence in everyday life. Someone who records a storage location independently and later retrieves it independently needs to ask other people for help with this task less often. This could shorten search times and reduce frustration and stress. Such a benefit aligns with the aim of Inclusive AI to strengthen participation and well-being through appropriate applications. Whether these effects actually occur and persist, however, must be examined in further studies with the target group (Bendel 2025; Huckele 2026).
From an information ethics perspective, the privacy-friendly architecture deserves particular attention (Bendel 2019). Information about objects and storage locations can reveal possessions, habits, and living circumstances. WhereIsIt processes this information locally, limits recording to explicitly initiated interactions, and deletes temporary recordings after processing them. In this way, the project supports informational autonomy, meaning the ability to determine how one’s own information is collected and used (Bendel 2019). The ability to delete entries and operate without external cloud services strengthens the user’s control. This design addresses requirements also emphasized in discussions of Be My AI and other Inclusive AI systems (Bendel 2024, 2025; Huckele 2026).
Limiting the system to a simple memory function offers another advantage. Its basic functions require neither attaching tags to objects nor capturing images of homes with cameras. Local processing also reduces dependence on an internet connection and external services, which can likewise be a concern of information ethics (Bendel 2019). This offers an interesting perspective for Inclusive AI: a manageable set of functions can promote accessibility if it reliably supports a frequently performed task (Bendel 2025; Huckele 2026).
Risks of WhereIsIt
The reliability of the information provided remains a central challenge. Misunderstanding an object or storage location can lead to an incorrect reminder. Even correctly saved information can become outdated if someone moves the object. From an information ethics perspective, the application should therefore make clear that it is reporting a location it was told about, rather than verifying the current position. Required confirmations and clear correction options would be important steps toward limiting misplaced trust. The project identifies corresponding weaknesses in speech processing and recommends better safeguards for input (Huckele 2026). The importance of reliable information for blind and visually impaired people was already highlighted in the study of Be My AI (Bendel 2024).
Another limitation concerns the semantic validation of location information. The prototype accepts a reported storage location without checking its plausibility. If, for example, “the moon” is specified as the location, the system saves this information. Such a case would probably occur rarely in practice, but it illustrates that language processing does not yet include checking the content. A plausibility check with a follow-up question function would therefore be useful. WhereIsIt could ask for confirmation or clarification when location information is unusual or unclear in the given context. The system should not reject unusual information on its own, however: “Moon” could, for example, be the name of a box or a personally agreed-upon memory cue. The decision about the intended storage location should remain with the user.
A particular challenge arises when entering unfamiliar apartments and houses. At home, a description such as “on the dresser in the hallway” can refer to a familiar mental map of the space. In an unfamiliar apartment, this may be entirely absent. Someone who does not know where the dresser is or how the hallway is laid out cannot yet derive a route to the object from remembering its storage location. WhereIsIt could help record a location description that has been clarified together with another person. This does not yet provide an orientation aid for unfamiliar indoor spaces, however. This limitation follows from the concept of verbally reported storage locations; automatic capture of the surroundings was not implemented in the project (Huckele 2026).
Local processing does not mean that all privacy issues have been resolved. The database must be protected against unauthorized access, and an audible response can disclose sensitive information to people who are present. In unfamiliar homes, location descriptions may also concern other people’s private circumstances. From an information ethics perspective, the interests of third parties must therefore be considered alongside control over one’s own data. This issue would become more significant with future extensions involving cameras or sensors. Users should be able to choose such extensions deliberately and retain control over them (Bendel 2024, 2025; Huckele 2026).
Ultimately, inclusiveness is determined by actual use. Getting started easily is a development goal; the entire operation must be equally clear and reliable. The weak results for nonvisual use in particular show that speech input and output alone do not guarantee sufficient accessibility. Further tests with blind people in realistic environments and over longer periods are required. Installation, suitable hardware, and technical support can also present barriers to access. The risk of a digital divide therefore remains relevant (Bendel 2019, 2024; Bendel and Jovic 2025; Huckele 2026).
Summary and Outlook
The studies discussed here explore different possibilities for Inclusive AI. The study of Be My AI shows how generative AI can support the description and evaluation of the immediate environment and discusses the associated dependencies and limitations (Bendel 2024). VISUAL gives blind and visually impaired people access to virtual wildlife observation and connects Inclusive AI with Animal-Computer Interaction (Mancini 2011; Bendel and Jovic 2025; Bendel 2026). WhereIsIt addresses a small but recurring and significant task: remembering the storage locations of everyday objects (Huckele 2026).
The project combines natural language, local AI processing, persistent storage, and speech output into a memory aid. Its potential lies in easy-to-use interaction and a privacy-friendly architecture that supports informational autonomy. The prototype shows that the basic functions can be implemented technically. It also makes clear where further work is needed: confirming recognized information, correcting errors, nonvisual dialogue, and portable use. Reliable operation should take priority before additional tracking technologies expand its functionality. Testing with blind people in everyday life is still pending (Huckele 2026).
Further projects at the FHNW School of Business focus on smart glasses, which have drawn criticism regarding their general use and their potential for observation and surveillance. The proposal submitted in summer 2026 by Myriam Rellstab, a master’s student supervised by Oliver Bendel, envisages a study of how AI-powered glasses can accommodate the different needs of blind, visually impaired, deaf, and hard-of-hearing people. A literature review, a comparison of five existing systems, and qualitative interviews are planned. Universal, specialized, and modular design approaches are to be examined and brought together in a conceptual framework (Rellstab 2026). These projects share the question of how AI systems can support people with impairments in everyday life without depriving them of control over information and decisions (Bendel 2025).
References
Avellan, Tero; Sharma, Sumita; Turunen, Markku (2020): AI for all: defining the what, why, and how of inclusive AI. In: Proceedings of the 23rd International Conference on Academic Mindtrek. pp. 142–144.
Bendel, Oliver (2019): 400 Keywords Informationsethik: Grundwissen aus Computer-, Netz- und Neue-Medien-Ethik sowie Maschinenethik. 2nd ed. Springer Gabler, Wiesbaden.
Bendel, Oliver (2021): Inklusion. In: Gabler Wirtschaftslexikon. Springer Gabler, Wiesbaden. Link
Bendel, Oliver (2024): How Can Generative AI Enhance the Well-being of Blind? In: Proceedings of the AAAI 2024 Spring Symposium Series, Symposium “Impact of GenAI on Social and Individual Well-being”. Stanford University, Stanford, California, 25–27 March 2024. The AAAI Press, Washington, DC. Link
Bendel, Oliver (2025): Inclusive AI. In: Gabler Wirtschaftslexikon. Springer Gabler, Wiesbaden. Link
Bendel, Oliver; Jovic, Doris (2025): There’s a Large Hippo Resting in the Mud: Virtual Video Safaris for Blind and Visually Impaired People. In: Wiley Industry News, 10 November 2025. Link
Bendel, Oliver (2026): Tier-Maschine-Interaktion. Springer Gabler, Wiesbaden.
Huckele, Damian (2026): WhereIsIt: An Object Reminder Assistant for Blind and Visually Impaired People. Bachelor Thesis. FHNW School of Business, Olten.
Mancini, Clara (2011): Animal-computer interaction: a manifesto. Interactions 18(4):69–73. Link
Rellstab, Myriam (2026): Inclusive AI for Wearables: Assistive Glasses as a Case Study. Master Thesis Proposal. FHNW School of Business, Olten.
Shams, Rifat Ara; Zowghi, Didar; Bano, Muneera (2026): A question bank to assess inclusive AI. AI and Ethics, 2026, vol. 6, no. 2, pp. 242.
