Project MigrantPal: Harnessing Digital Technology to Improve the Well-being of Migrant Workers in Singapore

Executive Summary

Singapore relies on migrant workers from Bangladesh, India, and China for work in construction, marine, and service. Their integral contributions drive the country’s economic development, underscoring their role in nation building. Despite their contributions, these workers face language and cultural barriers and have limited access to essential resources. There are several nongovernmental organizations (NGOs) in Singapore as well as government resources to address their needs, but more can be done.

We designed a chatbot named MigrantPal, which is less (human) resource intensive than government or NGO personnel would be. It’s available 24/7 and can respond to multiple migrant workers simultaneously. Of course, chatbots cannot substitute for human contact and support but we seek to explore their potential and challenges. This is especially interesting after the rise of Large Language Models (LLMs) such as ChatGPT. In developing our chatbot, we evaluated the effectiveness of LLMs when they’re integrated with more traditional forms of chatbot development.

We engaged migrant workers to test and then provide feedback on using MigrantPal and also analyzed the anonymous responses between the chatbot and workers. The chatbot allows both text and voice inputs for ease of use. Traditional chatbots often had limitations in understanding users’ queries. Because our chatbot was integrated with LLMs, we documented actual conversations so we could evaluate whether LLMs have a more robust ability to understand a diverse range of queries, including informal English and spelling mistakes. This is especially useful for migrant workers who are not proficient in English or who make spelling mistakes. We also analyzed common topics that workers would find useful in a chatbot. While we demonstrated potential in a chatbot’s utility, there remains further work to be done to drive a sustained adoption.

Background: Migrant Workers in Singapore

Migrant workers in Singapore come from diverse backgrounds, often from Bangladesh, India, China, Philippines, and Indonesia. As of December 2023, over 1.25 million (approximately 1,239,200) migrant workers were employed in the country — more than 400,000 in the construction, marine shipyard, and process sectors.¹

Migrant workers typically arrive either through intermediaries or on the recommendation of their friends and relatives. Most come for the potential to earn money to provide a better life for their families back home. They often face challenges related to language barriers, unfamiliarity with local laws, and limited access to resources. Their challenges were specifically brought to light during the pandemic when there were outbreaks of COVID-19 within their dormitories.

Current Resources for Migrant Workers Seeking Assistance and Protection

Ministry of Manpower

During the pandemic, an entire division formed under the Ministry of Manpower (MOM). Called Assurance, Care, and Engagement (ACE), it aims to provide support to migrant workers and dormitory operators. It offers comprehensive medical support to ensure accessible health services and fosters partnerships with stakeholders, deploying Forward Assurance and Support Teams (FASTs) across various dormitory types to support dormitory logistics. It also manages a network of on-site medical centres to provide tailored medical care and recreation centres for migrant workers to enjoy communal sports and recreational facilities. MOM also enforces employment laws to protect migrant workers. These laws address employment conditions, injury compensation, workplace safety measures, and dormitory standards. Migrant workers can walk in to service centres or call a hotline to reach out to MOM. When it comes to digital interventions, MOM has released the FWMOMCare mobile application, which contains features that make it convenient for migrant workers to seek help or find out more information on issues and news affecting them.

Using the FWMOMCare mobile application, a migrant worker can:

• Request to speak to a MOM officer
• Report unsafe workplace practices
• Seek medical assistance through telemedicine
• View infographics on the latest advisories
• Book facilities at selected recreation centres.

This application was built to be a one-stop service portal for migrant workers to access information or services or to request to speak to an officer. It is largely a one-way interaction. By contrast, a chatbot that offers an interactive experience with continuous availability around the clock serves in a highly complementary manner. There is also potential for a chatbot to provide more information compared to what is offered through the mobile application.

Nongovernmental Organizations

There are four prominent NGOs in Singapore that cater to the needs of migrant workers, such as Migrant Workers’ Centre (MWC), HealthServe, Transient Workers Count Too (TWC2), ItsRainingRaincoats (IRR), and Humanitarian Organization for Migration Economics (HOME). These organizations provide a range of services, including legal aid, healthcare, counselling, and skills training. They also engage in public education and advocacy to raise awareness about the challenges faced by migrant workers and promote fair treatment.

Even though prominent NGOs have helplines and are consistent with their services, there remains a limited number of NGOs that may not necessarily cater to the entire migrant worker population. And because not every migrant worker has access to or is willing to approach an NGO, chatbots can potentially complement the reach of NGOs.

Friends and Family

During engagement sessions with migrant workers, we discovered that some have a network of friends or family members either working in Singapore or who previously worked here. This support network is invaluable because they share practical tips and essential information about living and working in the country.

Regular communication with family members back home is also a crucial emotional support pillar. However, despite this support system, some migrant workers choose not to disclose the challenges they face, preferring to keep their families uninformed to avoid causing worry.

Websites

A dormitory coordinator mentioned in an interview that many migrant workers, especially those who are new in the country, are unaware of the laws and regulations in Singapore. Websites in Singapore are generally not designed with the migrant workers’ needs in mind in terms of language and information because they are mainly in English. This can pose an obstacle for migrant workers, many of whom understand only their native languages so they often lack important knowledge about their rights and labour law.

Other Chatbot Use Cases

During the COVID-19 pandemic, a chatbot called SGDormBot was launched to assist migrant workers in dormitories by reminding infected workers to monitor their temperature and oxygen levels in real time.² This chatbot sent automated reminders and basic health checks which saved healthcare professionals’ time and ensured that health protocols were understood and followed, offering multilingual support to workers who might struggle with instructions. In this way, SGDormBot not only aided in early detection of potential COVID-19 symptoms but also contributed significantly to the well-being and safety of migrant workers by providing accessible, understandable, and crucial health guidance. It was used during the pandemic but as of the time of this writing, we are not aware of any chatbots developed specifically for migrant workers in the construction, marine, and process sector.

About This Intervention: The Chatbot’s Purpose

NGOs that serve the vast migrant worker community face common challenges involving limits to their capacity, availability, and ability to reach. Because these NGOs are, to varying extents, run by volunteers, it can be challenging for them to provide services at the exact point in time that migrant workers may need support. For example, NGOs who have the resources to maintain helplines require funds for costs and training. Another challenge is that migrant workers sometimes find it difficult to open up and seek help for their difficulties.

Figure 2 captures a conversation with a migrant worker who was seeking help for his wisdom tooth. After some inquiry, the local contact the worker spoke to found the helpline for Healthserve, an NGO providing dental services to migrant workers. The worker made an appointment and had his wisdom tooth promptly extracted. The local resource person simply passed on the contact for an NGO, a task which could possibly be performed by a chatbot.

Based on this conversation whereby helpful information regarding an NGO’s hotline was shared with a migrant worker, we identified the chatbot’s primary function as a way to disseminate information. Based on secondary research through NGO and government websites, we also identified other topics such as employment issues, culture, and basic information on insurance coverage. Subsequently, we aimed to use focus groups to assess the relevance of these identified topics as well as to surface other potential topics. We did not go in depth into any particular topic, but sought to access the general sentiments toward chatbots as an indication of their potential usefulness.

Beyond addressing common topics, a chatbot can address important gaps that some existing NGO support structures don’t cover, such as providing round-the-clock availability and offering a more discreet way for workers to seek help than in-person exchanges. A chatbot is also extremely cost efficient for NGOs because it typically requires fewer resources to operate and maintain over time compared to human-based solutions. It has the potential to reach a wider audience than NGOs may be able to manage. Therefore, integrating chatbot technology could significantly enhance the support services available to migrant workers.

Since most of the migrant workers can speak only their native languages and struggle with English, the chatbot improves language accessibility, enabling these workers to seek help despite the language barriers. When migrant workers ask questions to the chatbot, it responds in their preferred language.

Here are some samples of the chatbot’s ability to translate from and to their native language:

Example 1:

User Query (in Bangla): “God শ্বর” শব্দটি এমন একটি দে বতাা বা উচ্চতার শক্তি  দেবাঝাায় যাা ক্তিবশ্বজগতেতার স্রষ্টাা এবং শাসক বতে  মতেন করা হয়, প্রাায়শই ধমীয় দেপ্রাক্ষাাপতে  দে াতেকরা উপাসনা বা শ্রদ্ধাা কতের।

User Query (translated to English): What does God mean?

Bot Response (in Bangla): “God শ্বর” শব্দটি এমন একটি দে বতাা বা উচ্চতার শক্তি  দেবাঝাায় যাা ক্তিবশ্বজগতেতার স্রষ্টাাএবং শাসক বতে  মতেন করা হয়, প্রাায়শই ধমীয় দেপ্রাক্ষাাপতে  দে াতেকরা উপাসনা বা শ্রদ্ধাা কতের।

Bot Response (translated to English): The word “God” refers to a deity or higher power that is believed to be the creator and ruler of the universe, often worshipped or revered by people in religious contexts.

Example 2:

User Query (in Tamil): நாான் இங்கே  எந்த விஷயங் ளைை பற்றி அறிந்து கொ ாள்ைலாாம்?

User Query (translated to English): What things can I know about here?

Bot Response (in Tamil): கே ளைலா கொதாடர்பான  ாயங் ள் அல்லாது கேநாாய் ளுக்கு, உங் ளுக்கு கே ளைலா  ாயம் இழப்பீட்டு  ாப்பீடு  ழங் ப்படும், கே லும் நீங் ள் சம்பைம்,  யது அல்லாது குடியுரிளை  ஆகிய ற்ளைைப் கொபாருட்படுத்தா ல், நீங் ள் கேசளை  அல்லாது பயிற்சி ஒப்பந்தத்தில் இருக்கும்  ளைை  ருத்து  கொசலாவு ள்  ற்றும் இழப்பீட்டுக் ான பாது ாப்பு  ழங்குகிைது.

நீங் ள் ஒரு பணி அனு தி அல்லாது பாஸ் ளை த்திருப்ப ைா  இருந்தால், கே ளைலா அல்லாாத  ருத்து  கொசலாவினங் ளுக்கு ( ருத்து  ளைனயில் அனு தித்தல்  ற்றும் கொ ளிகேநாாயாைர் சிகிச்ளைச ள் உட்பட), உங் ள் முதலாாளி உங் ளுக்கு  ருத்து   ாப்பீட்ளைட  ழங்  கே ண்டும். ஆதாைம்: அம் ா இது நீங் ள் கேதடு து இல்ளைலாயா என்பளைத எனக்குத் கொதரியப்படுத்துங் ள்.

Bot Response (translated to English): For work-related injuries or illnesses, you will be provided Work Injury Compensation Insurance and it provides coverage for medical expenses and compensation so long as you are under a contract of service or contract of apprenticeship, regardless of salary, age, or citizenship.

For non-work-related medical expenses (including hospitalization and outpatient treatments).³

To ensure that the migrant workers receive accurate information, we integrated documents and frequently asked questions from the Ministry of Manpower’s website into our chatbot’s knowledge base. This guarantees that the information provided is reliable and up to date. All the information provided by the chatbot caters mainly to migrant workers’ issues.

The chatbot offers ease of use through both text and voice interactions, making it accessible and user friendly for all migrant workers. Although the translations between text and voice are not perfect, it proved to be quite functional.

How We Created the Chatbot

Predefined Responses

We started by designing predefined responses. Using rules and machine learning we aimed to identify a user’s intent through their questions and then designed a corresponding chatbot response that would be returned to the user. In designing these predefined responses, we had to think of the topics that the chatbot could provide more information on. We chose insurance, legal and employment rights, culture and sensitivity, and work life. Addressing these topics ensures that the chatbot will be able to assist the migrant workers in answering some of the questions that they might have without having to go through multiple hoops and processes to get the answer they’re looking for.

One limitation of such an approach is the need for a person with the appropriate knowledge to design these predefined responses, which is akin to providing a curated set of questions and answers. Thus, the depth and range of responses for the selected topics depend on the team’s efforts in curating the question-and-answer pairs from public sources of information on the internet. We aimed to leverage the model’s capacity to handle a broader spectrum of user queries beyond the predefined responses, thereby enriching the user experience (see Figure 3). However, that inadvertently led to certain challenges (which we discuss later).

Widening Breadth of Topics Through Large Language Models (LLMs)

Large Language Models (LLMs) represent advanced computational systems designed to comprehend and generate human language. These models function similarly to sophisticated chatbots, trained on extensive amounts of text from diverse sources, including books, articles, and websites. This extensive training allows them to recognize patterns and structures within the language, enabling them to perform tasks such as answering questions, composing essays, engaging in conversations, and creating narratives.

To illustrate, imagine having access to a vast library and reading every book within it. With this comprehensive knowledge, you would be able to respond intelligently to questions or engage in discussions, drawing on the information you’ve absorbed. LLMs operate on a similar principle but on a much larger scale. While they do not “understand” language in the human sense, they can simulate understanding by predicting subsequent words or sentences based on their training data.

Figure 4 illustrates the practical output of LLMs — AI-generated responses that simulate humanlike language comprehension and generation. These responses showcase how LLMs leverage their extensive training to generate coherent and contextually relevant text across various tasks.

In essence, LLMs are powerful tools capable of performing a variety of language-related tasks, thereby enhancing the naturalness and efficiency of human-computer interactions.

A Predefined Response or a Generative Response

MigrantPal can produce two kinds of responses: a predefined response curated during its development or an AI-generated response. It chooses between the two types of responses by matching the user’s question to the respective questions associated with each predefined response. Based on certain thresholds, if there isn’t a match, an LLM-based AI-generated response will be provided to the user.

The Onboarding Process: Planning for Focus Groups

The engagement team leaders — three dedicated individuals — were responsible for planning, preparing, facilitating, and executing the standard operating procedures for focus groups. They held meetings to organize upcoming focus group sessions, verified attendance, and coordinated with dorm operators to schedule the sessions. Prior to each focus group, they also conducted briefings to ensure that all participants understand their roles and objectives. Focus groups were aimed at gathering feedback on the chatbot’s usefulness.

During each focus group, the engagement leaders demonstrated how to approach migrant workers, including polite introductions of oneself and engaging in small talk to make them feel comfortable, often offering them drinks. They also provided conversational starters for volunteers, such as asking about the workers’ origins, reasons for coming to Singapore, leisure activities, and current tasks. Emphasizing cultural sensitivity, they guided volunteers on topics to avoid, ensuring respectful interactions.

The volunteers, primarily from Singapore Management University (SMU), served as additional hands to introduce the chatbot to migrant workers and gather on-the-ground feedback with Reach Alliance team members. During focus groups, leaders and volunteers built rapport to understand the workers’ needs, even when not discussing the chatbot directly. This relationship-building was crucial for gaining insights into the workers’ challenges and perceptions, which helped to refine the chatbot’s focus and development. The standard operating procedure (SOP) included clear guidelines on the onboarding procedures, which helped to maintain consistency and effectiveness while interacting with the migrant workers. Before the onboarding process, we would ensure that the workers felt comfortable. The onboarding process took place during focus group sessions, and introduced the chatbot to migrant workers, helped them to create a Telegram account, and guided them on how to use the chatbot for day-to-day issues. This structured approach ensured that both engagement leaders and volunteers were wellprepared and aligned in their efforts to support migrant workers.

Figure 6 shows an example of the recently updated simplified SOP developed by one of the engagement team leaders to ensure a more efficient onboarding process that is universally understandable and actionable for the volunteers.

Building relationships with migrant workers fostered trust and encouraged open communication. When workers feel understood and valued, they are more likely to engage with the chatbot, provide honest feedback, and share their experiences. These groups helped us to narrow our focus for chatbot development to what is most essential for the migrant workers.

Chatbots’ Utility: Observations and Challenges

Our engagement sessions in trials of MigrantPal provided insights on whether there is a role for chatbots to support migrant workers and if there is, what the challenges might be. MigrantPal has a certain level of competency to provide useful information, especially considering how traditional chatbots often fail to understand a user’s query because of different keywords they use, misspellings, or lack of context awareness. Our results show how Large Language Models (LLMs) can overcome this.

More Robust Understanding

In the examples listed in Table 1, the chatbot can “understand” the context and intent of the user queries in a more robust way even though there were misspellings and grammatical errors. LLMs are also able to identify the semantic meaning of text even though different keywords are used.⁴ For example, in example 1 of the table, the chatbot was able to detect the user’s concern about not receiving proper compensation even though the word salary was used. Despite not repeating the exact words, the response captures the essence of user’s concern and provides relevant advice, demonstrating a good semantic match.

Migrant Workers’ Feedback About the Chatbot

Migrant workers suggested the future potential use of chatbots to enhance their quality of life.⁵

1. The chatbot can be relied on to answer queries related to migrant workers.
   
   We engaged with migrant workers in brief conversations to gather their feedback after focus groups. For instance, one migrant worker used the chatbot after the focus group and found it very helpful, mentioning that the chatbot was “very helpful to navigate around,” and “answered my queries” (Figure 7).
2. Migrant workers found the interface easy to follow.
   
   Another migrant worker reported using the chatbot a week earlier. He was able to navigate the chatbot easily and had no issue with asking about dormitories, salary, and working hours. He subsequently found that the responses directly answered his questions.
3. The chatbot is a good bridge between their own language and the most common language used in Singapore.
   
   Since many migrant workers are unable to speak and understand English, many of them mentioned that having Bengali included in the chatbot would effectively overcome language barriers, further allowing the chatbot’s ease of use. One person we interacted with told us “Sometimes I need to spend some time typing the questions in English,” referring to the level of difficulty people experience when having to type in English to ask questions. This supports the notion that the chatbot can assist migrant workers in easily voicing their questions and thoughts without struggling to express themselves.
   
   Even when responses in their native languages contain inaccuracies, this did not pose a major problem in our trials.
   
   After the engagement sessions, just under one quarter (i.e., a 23.91 percent adoption rate ) of the workers continued to use or tried to use the chatbot.⁶ Even though these subsequent conversations were not sustained, it suggests the possibility of a longer-term use.

Analysis of the Conversations

We analyzed all the conversations carried out with the chatbot and classified the migrant workers’ queries into topics. We aimed to understand the subjects that the workers were conversing with the chatbot about. We then systematically classified these topics into distinct categories to capture the breadth of subjects that arose during interactions. Figure 8 illustrates the topic categories, showing the predominant themes that emerged from the data. The classification process used LLMs initially, followed by manual refinements by the team to ensure accuracy and relevance to the workers’ queries. Some ambiguous topics were excluded from the pie chart to maintain clarity and focus on the most prevalent themes.

Table 2 provides specific examples of user queries that we categorized into topics during our analysis. Each query contributed to the overall classification of migrant workers’ concerns and interests as represented in Figure 8.

Challenges of Chatbots — Technical Insights

Need for Ongoing Monitoring

There will always be a possibility of inaccurate responses, regardless of whether a chatbot uses a predefined or AI-generated response. With predefined responses, bots sometimes fail to accurately identify the right response. For example, when there was a user query about bank transfers, the chatbot responded with upcoming movie information! By contrast, AI-generated responses typically try to generate an answer even though such an answer may not exist. And, depending on the period that the Large Language Model was trained on, knowledge cutoff dates could result in inaccurate or confusing answers. This challenge is compounded by the fact that policies and other relevant information relating to migrant workers change regularly. The common industry practice is for such systems to inform users of the possibility of inaccurate information.

However, even with this warning of possible misinformation in place, a successful chatbot program needs ongoing monitoring of conversations and regular refinements. Whoever manages the chatbot requires a certain level of expertise. If the monitoring organization is an NGO, resource and capability issues can arise.

Balancing between Predefined and AI-generated Responses

We used both predefined and AI-generated responses to develop the chatbot. We found that AI-generated responses are typically more general and not tailored enough to migrant workers as this sample exchange shows.

Response Provided by LLM

User query: How can I be successful in Singapore?

Bot Response: To be successful in Singapore, it is important to focus on these key areas:

1. Continuous learning and upgrading your skills.
2. Building a strong professional network
3. Adapting to the local work culture and etiquette.
4. Being proactive and taking initiative in your work.
5. Maintaining a positive attitude and perseverance.
6. Being aware of and following local.

In this AI-generated response, the answers don’t align with the context of a migrant worker very well. For example, building a strong professional network (e.g., LinkedIn) may not be as useful as more specific advice such as ensuring workplace safety in construction sites, which is crucial for ensuring their success in their work. A chatbot could provide more specific guidance relating to the importance of wearing protection equipment, following safe work procedures or guidelines on working at heights, on scaffolds, or with machinery.

In contrast, the next exchange demonstrates how a predefined response provides more targeted information suited to the context of a migrant worker. Three different examples led to this response.

Response Provided by Predefined Responses

User Query 1: How to found driver job?

User Query 2: how to get new job for Sposs?

User Query 3: how to apply new job vacancy?

Bot Response: Remember! It is illegal to work for another employer. You must have a valid work pass to work in your job. You can work only in the occupation and for the employer specified in your work pass You must not be self-employed, participate in any business, or hold a second job to earn extra money.

Based on these examples, we see that the choice between predefined and AI-generated responses is not straightforward. Table 3 illustrates some difference between the two approaches.

After the chatbot was developed, methods to reduce the inaccuracies of AI-generated responses such as Retrieval augmented generation (RAG), which provides grounding on more factual information, were developed in both academia and industry. The team did not incorporate RAG in the prototype. However this represents another challenge: the technical expertise required to integrate newer technologies into a chatbot solution, especially with today’s fast-paced technological advancements.

Demonstrated Potential, but Gaps Exist for Sustained Adoption

Many of the workers that we talked to mentioned that the chatbot was “good” and “useful” but its adoption rate beyond the initial engagement sessions was relatively lacklustre with only a few “return customers” throughout the engagement period. While 23.91 percent adoption reveals potential in the usage of a chatbot, MigrantPal has not attained sustained adoption over a longer period or among the larger population of workers.

We provided incentives for some sessions, informing migrant workers that those who most frequently used the chatbot would receive five-dollar supermarket vouchers. Despite these incentives, there was no significant increase in usage after the engagement session.

Challenges Faced during Engagement Sessions

Language Barriers

In our interaction with migrant workers, we experienced some difficulties communicating. There was a gap between the language spoken by student volunteers from SMU (mainly English) and the migrant workers (mainly Bengali). Even though we broke down the chatbot features into lay terms, there was a clear difficulty in establishing a common understanding about our aim in this project. This issue persisted outside of the focus groups. Workers also articulated how there are sometimes misunderstandings between locals and migrant workers in a job setting because of such language barriers. Both sides found it hard to understand each other, including regarding the expectations of the job.

The language barrier varies between dormitories. In one, many migrant workers did not fully understand what we were trying to communicate, based on their responses and body language. The second dormitory had many English-speaking migrant workers, making it easier for us to establish the aims of our project.

Time Constraints

The workers have a busy work schedule, and normally have only a day out of the entire week off. Migrant workers are usually free on Sundays, at certain times of the day. We had only this period of time to interact with them. However, even during this period, we struggled to onboard these workers since many wanted to spend their off day calling their loved ones. We therefore had many rejected invitations regarding the introduction of the chatbot during the focus group.

Skepticism about Chatbots Among Migrant Workers

Many of the workers we interacted with mentioned that the chatbot was “good” but they were not enthusiastic about interacting with it and expressed a certain degree of resistance and skepticism toward it. This resistance or reluctance was compounded by the language barriers, so it was difficult to address their concern. This affected onboarding people to try the chatbot and led to headwinds in the adoption and take-up rates of the chatbot in general.

Efforts to promote MigrantPal to migrant workers in the dormitories were not effective. Our primary strategy involved collaborating with the dormitory operator and putting up posters, but these methods had limited impact on engagement and adoption rates. Due to inconsistent and ineffective marketing, usage rates dropped significantly after the first week of onboarding. While we did manage to reach some users through Telegram after they interacted with the chatbot, many either deleted the app, stopped responding, or blocked us entirely, making it difficult to maintain follow-up engagement.

The Target Audience’s Unfamiliarity with Telegram

While the migrant workers were using the voice-to-text feature of the ChatBot, we observed a sizeable portion of them were not comfortable in using the feature, leading to some incomplete voice messages that hampered the post-event analysis. We eventually attributed this issue to the difference in the user interface between Telegram and WhatsApp. Admittedly the best solution to address this would have been to make use of WhatsApp instead of Telegram, but in the face of technical and logistical constraints behind the usage of WhatsApp for the ChatBot, we decided to stick with using Telegram.

Lessons Learned and Recommendations for Future Projects

Technicalities

The Chatbot Should Have Shorter Replies

Feedback from migrant workers identified how the chatbot’s responses are often lengthy and dense. When the chatbot provides information or answers queries, the responses can be too difficult to digest with whatever time the worker has when they seek information. The chatbot’s shorter replies in plainer language would help prevent users from feeling overwhelmed and enhance their understanding of the chatbot’s responses.

Ensuring Accessibility Through Simplicity

Workers’ onboarding process is roughly as follows:

1. Introduction of the team and the project
2. Download Telegram and key in mobile number
3. Search for the chatbot in order to start a chat with it
4. Answer multiple prompts (i.e., acknowledgement of having their information recorded for research purposes, what language they prefer, voice or text, etc.)

Migrant workers tended to lose focus and interest during the onboarding process. They are entitled to only one day off out of the seven days in a week. They want to spend their time on catching up with their friends and family instead of going through this process which may seem lengthy to them. Lengthy processes could create higher barriers for migrant workers to access the intervention, potentially alienating the very population it aims to assist. We recommend that future interventions should not require so many extra steps to provide the intended functions.

Having a Human Element (a Community Champion) Is Essential

These projects require a human element — a community champion — to be truly effective. A community champion is an individual or a group of individuals who volunteer within their own local community to promote and enhance people’s well-being. A community is more receptive to a community champion because they share a common background and understanding that people outside that community may never fully comprehend. Community champions could streamline the process of explaining the project, helping expand migrant workers’ awareness of the chatbot and its usefulness for improving their well-being. Such a local champion can share the migrant workers’ thought process without concerns about language or unfamiliarity, thereby having a robust two-way communication and gaining a clearer understanding of their needs. They bridge the gap between the chatbot and diverse user contexts. Therefore, by leveraging community champions, we expect migrant workers to be more receptive to understanding the chatbot’s utility, given the endorsement from credible community champions, potentially increasing their adoption of the chatbot.

Designing Deeper Engagements

Our engagement sessions with new groups of migrant workers each time ranged from a few minutes to fifteen minutes long. This made it challenging to engage the workers at deeper levels. To motivate deeper engagement sessions in the future, further engagements with the same group of workers could deepen our understanding of their perspectives toward the chatbot and get better feedback. A notable method of driving deeper engagements with the migrant workers to understand them better is through designing the right questions to ask.

Brainstorming questions before conducting our focus groups facilitated smoother conversations and more productive conversations. This can effectively guide us to our next course of action in terms of how we can improve the chatbot to better suit their needs.

Additionally, some of the migrant workers that we interacted with briefly mentioned that the chatbot was “good,” though many others were unenthusiastic about interacting with it since they did not have a clear understanding of what chatbots are, or what their benefits might be. Most workers expressed a certain degree of resistance and skepticism toward the chatbot. Their reticence was compounded by language barriers we had while interacting with them, so it was difficult for us to address their concerns. It was also more challenging for us to onboard them for our chatbot trials and led to obstacles in the chatbot’s adoption and take-up rates. Hence, learning to ask meaningful questions can significantly improve the efficiency of focus groups despite the language barrier.

Additionally, community engagement techniques could also be employed to facilitate and draw out feedback in a range of activities that would allow the migrant worker to better express themselves. During one of our focus groups, we had an ice-breaker session to get to know the migrant workers better. The simple activity allowed us to experience significantly improved engagements from the migrant workers afterwards. This relaxed setting made the migrant workers more comfortable and willing to share their personal experiences and stories, fostering a more open conversation. This change in the atmosphere was crucial in building trust and introducing the chatbot to them.

Chatbots’ Usefulness Can Differ for Migrant Workers from Different Contexts

Migrant workers vary widely in their cultural and demographic backgrounds. Besides nationality, their age, industry, and even the dormitories they live in can lead to subcultures. So instead of a one-size-fits-all chatbot, future work can consider a more personalized approach.

Because most migrant workers in Singapore tend to have smartphones with data plans for easier communication between them and their employers, our solution of potentially leveraging digital technology, a chatbot, to improve their well-being remains feasible. However, this solution may not be feasible in other countries or environments where migrant workers don’t have access to technology. Future research projects focused on migrant workers should conduct thorough research to determine the feasibility of applying digital technologies within their specific project contexts.

Balance Between Breadth and Depth

Large Language Models (LLMs) have paved the way for more robust chatbots that are able to respond to a wide range of questions. However, even as this floodgate opens, we need to adopt guardrails to manage hallucinations and scrutinize the depth of responses. LLMs are pretrained on general information that may not always be relevant to migrant workers working in Singapore. Hallucinations arise because LLMs are probabilistic models that predict the next word so the issue of hallucination cannot be fully eradicated.

That said, there is still a place for the traditional approaches to chatbot development where specific responses are crafted. Behind the scenes in this research, we started out with a traditional approach and moved onto LLM. In our experience with both approaches we had to determine whether to respond with a preprogrammed response or leverage on the LLM to generate a response in a probabilistic manner — where there will always be a risk of hallucination.

We recommend greater scrutiny in the selection of use cases when balancing between predefined and AI-generated responses. Perhaps a predefined response will suit certain use cases or types of questions better.

Chatbot Maintenance

Nongovernmental organizations (NGOs) looking to implement a chatbot to manage the large volume of needs that migrant workers articulate must recognize the importance of continuously maintaining the chatbot’s usability and effectiveness. These NGOs need to consider who they will engage for the chatbot’s ongoing maintenance and technical support to prevent it from malfunctioning. They should also plan for regular updates and user feedback integration to ensure the chatbot remains responsive and relevant to users’ needs.

Working with Stakeholders

As we mentioned earlier, our main source of publicity in the dormitories involved putting up posters. However, these posters had limited effects in driving engagement and take-up rates. The lack of consistent and effective marketing meant that adoption rates beyond a week from the initial onboarding date were dismal at best. It is crucial to identify the most effective ways to reach the target audience and tailor marketing strategies accordingly, thereby increasing the solution’s adoption rate.

Technology Breakthroughs Do Not Naturally Lead to Adoption

Traditional methods in chatbot development had limited ability to understand human intents expressed through text. LLMs represent a breakthrough where intentions expressed in words that aren’t explicitly programmed can still be understood, leading to appropriate responses from a chatbot. We saw that our chatbot’s technical abilities were insufficient to lead to widespread adoption. While many migrant workers responded to the chatbot warmly during engagement sessions, usage beyond the sessions serves as a more accurate proxy for its true adoption.

Summary of Recommendations

Three issues can increase the chatbot’s usage and effectiveness in assisting migrant workers. First, technical enhancements to the chatbot should include shorter replies, and a more straightforward onboarding process. Second, despite being a technical project, the human element is critical to raising adoption rates, designing deeper engagements, and maintaining relationships with key stakeholders. Third, balancing the breadth and depth of topics the chatbot can address is essential, particularly when managing time and budget constraints. This involves fine-tuning the chatbot’s ability to answer a range of questions while ensuring the depth of each response is sufficient to be useful. Chatbots’ effectiveness may also vary for migrant workers from different countries.

The team successfully developed multiple iterations of a chatbot and conducted several focus groups to attain preliminary findings regarding chatbots’ potential utility within Singapore’s migrant workers community. Even though there was no sustained usage, the conversations that migrant workers had with the chatbot during the engagement sessions reveal the potential for chatbots to play a complementary role alongside support offered by the government and NGOs, especially in terms of availability and reach.

While generative AI demonstrated more robust understanding of the users’ intent, we caution against inaccuracy of information. As generative AI continues to advance, we anticipate new functionalities that could provide further breakthroughs in supporting hard-to-reach communities through digital means.

The study underscores that a truly effective chatbot goes beyond technical utility. A participatory design process that involves migrant workers throughout is essential to creating solutions that address real needs and alleviate specific challenges. A key highlight of this case study is that the advancement of technology does not naturally lead to (nor guarantee) an impactful adoption. These findings provide valuable insights for organizations aiming to develop chatbots for vulnerable or similar communities.

Acknowledgments

We thank Nagalakshmi Magandarn from SMU’s Institutional Review Board for detailed guidance on the ethics review, Devi and Gek Cheng for collaboration with the Singapore Management University (SMU) Centre for Social Responsibility, the MigrantPal student team (Hansen Lim, Leroy Lee, Wei Hua, Chase Lim, Zheng Feng, Dylan Teoh, Zhen Lin, Joshua Sumarlin, Jonas Koh, and Rozanne Goh) for their tireless contributions, and Chee Keong from Minestone Corporation, an operator for dormitories owned by MES Group, for facilitating the engagement groups. Last but not least, thanks to Ying Erh Oey, Yuehan Zhuo, and Fiona Williamson from SMU for their leadership and Marin MacLeod and Moni Kim from Reach for their guidance on the research process.

MASTERCARD CENTER FOR INCLUSIVE GROWTH

The Center for Inclusive Growth advances equitable and sustainable economic growth and financial inclusion around the world. The Center leverages the company’s core assets and competencies, including datainsights, expertise, and technology, while administering the philanthropic Mastercard Impact Fund, to produce independent research, scale global programs, and empower a community of thinkers, leaders, and doers on the front lines of inclusive growth.

MUNK SCHOOL OF GLOBAL AFFAIRS & PUBLIC POLICY

The Munk School of Global Affairs & Public Policy at the University of Toronto brings together passionate researchers, experts, and students to address the needs of a rapidly changing world. Through innovative teaching, collaborative research, and robust public dialogue and debate, we are shaping the next generation of leaders to tackle our most pressing global and domestic challenges.