The constant progress being made in machine learning needs to extend across borders if we are to democratize ML in developing countries. Adapting state-of-the-art (SOTA) methods to resource constrained environments such as developing countries can be challenging in practice. Recent breakthroughs in natural language processing and generative image models, for instance, rely on increasingly complex and large models that are pre-trained on large unlabeled datasets. In most developing countries, resource constraints make the adoption of these breakthrough challenges. Methods such as transfer learning will not fully solve the problem either due to bias in pre-training datasets that do not reflect environments in developing countries or the cost of fine-tuning larger models. This gap in resources between SOTA requirements and developing country capacities hinders a democratic development of machine learning methods and infrastructure.
Practical Machine Learning for Developing Countries (PML4DC) workshop is a full-day event that has been running regularly for the past 3 years at ICLR (past events include PML4DC 2020, PML4DC 2021 and PML4DC 2022). PML4DC aims to foster collaborations and build a cross-domain community by featuring invited talks, panel discussions, contributed presentations (oral and poster) and round-table mixers.
The main goal of PML4DC is to bring together researchers and practitioners (from academia, industry and government agencies) to reflect on aspects of designing, implementing, deploying and monitoring machine learning (ML) solutions that are typical in low resource environments across multiple sectors, such as healthcare, finance, agriculture, or education. Specifically, we encourage contributions that highlight issues related to:
-Advances in algorithms and methods tailored for problems related with data-scarcity, imbalanced representations and limited computational resources
-Industry practices to scale-up ML solutions in low resource settings while balancing performance and latency tradeoffs
-Societal and policy impacts of ML solutions in developing countries obtained via pilot studies, qualitative research, and human-in-the-loop settings.
Fri 12:00 a.m. - 12:15 a.m.
|
The 4th Workshop on practical ML for Developing Countries: learning under limited/low resource settings
(
Welcome Remarks
)
SlidesLive Video » |
🔗 |
Fri 12:15 a.m. - 12:45 a.m.
|
Honoring Kiswahili with Technology and Communit (Talk by Kathleen Simunyu)
(
Keynote
)
SlidesLive Video » This talk gives an indepth look at Mozilla Common Voice’s Kiswahili work – an initiative to bring a vital language of East Africa online, and to make voice technology accessible to Kiswahili speakers. The talk will cover an introduction to the Mozilla Common Voice platform, community building, including how we have worked to address gender challenges and the inclusion of speakers of related dialects and variants of Kiswahili, the challenges and successes of building an open-source Kiswahili speech data set, the research questions we have explored along the model development roadmap and our efforts to disseminate and encourage the use of the resources created in this work. |
🔗 |
Fri 12:45 a.m. - 1:15 a.m.
|
Targeting Cash Transfers in Low-Resources Settings (Talk by Joshua Blumenstock)
(
Virtual Keynote
)
SlidesLive Video » Targeting is a central challenge in the design of anti-poverty programs: given available data, how does one rapidly identify the individuals and families with the greatest need? This talk will discuss recent uses of machine learning, applied to non-traditional data from satellites and mobile phones, in the targeting of anti-poverty programs. It draws on results from several field-based projects -- in Togo, Afghanistan, Nigeria, and Kenya -- that illustrate the promise, as well as some of the potential pitfalls, of this new approach to targeting. Collectively, the results highlight the potential for new data sources to improve humanitarian response in low resource settings, particularly during crises and when traditional data are missing or out of date. |
🔗 |
Fri 1:15 a.m. - 1:30 a.m.
|
Contributed Talks in-person presentation
(
Spotlight
)
link »
SlidesLive Video » |
🔗 |
Fri 1:30 a.m. - 1:45 a.m.
|
Contributed Talks virtual presentation
(
Spotlight
)
link »
SlidesLive Video » |
🔗 |
Fri 1:45 a.m. - 2:00 a.m.
|
Coffee Break
|
🔗 |
Fri 2:00 a.m. - 2:30 a.m.
|
Measuring And Enforcing Diversity In Machine Learning(Talk by Adji Bousso Dieng)
(
Virtual Keynote
)
link »
SlidesLive Video » Diversity is important for many areas of machine learning, including generative modeling, reinforcement learning, active learning, and dataset curation. Yet, little effort has gone into formalizing and understanding how to effectively measure or enforce diversity. This talk will describe the Vendi Score, a new metric for measuring diversity that connects and extends ideas from ecology and quantum mechanics. The Vendi Score is defined as the Shannon entropy of the eigenvalues of a user-defined similarity matrix. It is general in that (1) it can be applied to any domain where similarity can be defined and (2) it doesn't require defining a probability distribution over the collection to be evaluated for diversity. The Vendi Score can therefore be used to measure the diversity of datasets, samples from a generative model, outputs from decoding algorithms, or any collection for which we want to assess diversity. We will showcase the Vendi Score as a diversity evaluation metric in several domains and as a means to improve the exploration of molecular conformation spaces. |
🔗 |
Fri 2:30 a.m. - 3:30 a.m.
|
Virtual Poster Session
(
Poster Session
)
SlidesLive Video » |
🔗 |
Fri 3:30 a.m. - 4:30 a.m.
|
Lunch Break
|
🔗 |
Fri 4:30 a.m. - 5:30 a.m.
|
NLP systems for low resource languages: hype vs reality
(
Panel discussion
)
SlidesLive Video »
|
🔗 |
Fri 5:45 a.m. - 6:00 a.m.
|
Contributed Talks in-person presentation
(
Spotlight
)
link »
SlidesLive Video » Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning: Malte Ostendorff.Ostendorff, Malte*; Rehm, Georg |
🔗 |
Fri 6:00 a.m. - 6:15 a.m.
|
Contributed Talks virtual presentation
(
Spotlight
)
link »
SlidesLive Video » Adaptive Representations for Semantic Search. Aniket Rege, Aditya Kusupati, Sharan Ranjit S, Sham Kakade, Prateek Jain, Ali Farhadi (University of Washington, Allen Institue for AI, Apple) |
🔗 |
Fri 6:15 a.m. - 6:45 a.m.
|
Automated Malaria Detection using Artificial Intelligence (Talk by Nakasi Rose )
(
Keynote
)
link »
SlidesLive Video » Malaria is one of the most significant endemic diseases in Sub-Saharan Africa. In Low developed countries (LDCs), the scourge is further bolstered by the lack of enough skilled lab technologists in health centers to accurately detect the disease using the widely accepted gold standard Microscopy method. Thus, the need for reliable detection interventions. This explains the birth of the Topic Group (TG), Automated malaria detection using Artificial Intelligence (AI). The aim is to harness AI to automate the detection of Malaria in a more fast, accurate, and cost-effective manner. Recently emerging technologies of AI and machine learning that can learn complex image patterns have been successful in different medical image analysis tasks and can improve public health. Therefore, the TG-Malaria under the ITU/WHO Focus Group AI for Health (FGAI4H) aims to develop a standardised benchmarking approach for AI based detection of Malaria. This involves all activities related to the curation of a quality dataset, development of AI models and approaches related to malaria detection, suggestions on scoring metrics, development of a benchmarking framework, and extension of the solution to improve disease surveillance and prediction. |
🔗 |
Fri 6:45 a.m. - 7:00 a.m.
|
Coffee Break
|
🔗 |
Fri 7:00 a.m. - 7:30 a.m.
|
Data-centric ML for low resource settings (Talk by James Zou)
(
Virtual Keynote
)
link »
SlidesLive Video » We will present a data-centric framework for making machine learning more trustworthy in developing countries. We will discuss best practices for data in different stages of the ML pipeline: starting with how to design/curate datasets, followed by how to identify informative data for ML, and then how to audit and debug ML models to ensure reliable application in resource-limited scenarios. |
🔗 |
Fri 7:30 a.m. - 8:00 a.m.
|
In Person Poster Session
(
Poster Session
)
MH1 |
🔗 |
-
|
Towards Federated Learning Under Resource Constraints via Layer-wise Training and Depth Dropout
(
Poster
)
SlidesLive Video »
Large machine learning models trained on diverse data have been successful across many applications. Federated learning enables training on private data that may otherwise be inaccessible, such as domain-specific datasets decentralized across many clients. However, federated learning can be difficult to scale to large models when clients have limited resources. This challenge often results in a trade-off between model size and data accessibility. To mitigate this issue and facilitate training of large machine learning models on edge devices, we introduce a simple yet effective strategy, Federated Layer-wise Learning, to simultaneously reduce per-client memory, computation, and communication costs.We train a machine learning model in a layer-wise fashion, allowing each client to train just a single layer, thereby considerably reducing the computational burden with minimal performance degradation. In addition, we introduce Federated Depth Dropout, a technique that randomly drops frozen layers during training, to further reduce resource usage. Coupling these two designs enables us to effectively train large models on edge devices. Specifically, we reduce training memory usage by 5$\times$ or more, and demonstrate that performance in downstream tasks is comparable to conventional federated self-supervised representation learning.
|
Pengfei Guo · Warren Morningstar · Raviteja Vemulapalli · Karan Singhal · Vishal Patel · Philip Mansfield 🔗 |
-
|
Intermediate Task Fine-tuning of Sequence-Sequence Language Models with Auxiliary Domain Parallel Data for Low-resource NMT
(
Poster
)
SlidesLive Video » NMT systems trained on Pre-trained Multilingual Sequence-Sequence (PMSS) models flounder when sufficient amounts of parallel data is not available for fine-tuning. This specifically holds for languages missing/under-represented in these models. The problem gets aggravated when the data comes from different domains. In this paper, we show that intermediate-task fine-tuning (ITFT) of PMSS models is extremely beneficial for domain-specific NMT, especially when target domain data is limited/unavailable and the considered languages are missing/under-represented in the PMSS model. We quantify the domain-specific results variations using a domain-divergence test, and show that ITFT can mitigate the impact of domain divergence to some extent. |
Shravan Nayak · Sarubi Thillainathan · Surangika Ranathunga · Rikki Hung · Yining Wang · Jonah Mackey · Andrew Ho · Anthony Rinaldi · En-Shiun Annie Lee 🔗 |
-
|
Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models
(
Poster
)
SlidesLive Video » Learned metrics such as BLEURT have in recent years become widely employed to evaluate the quality of machine translation systems. Training such metrics requires data which can be expensive and difficult to acquire, particularly for lower-resource languages. We show how knowledge can be distilled from Large Language Models (LLMs) to improve upon such learned metrics without requiring human annotators, by creating synthetic datasets which can be mixed into existing datasets, requiring only a corpus of text in the target language. We show that the performance of a BLEURT-like model on lower resource languages can be improved in this way. |
Amirkeivan Mohtashami · Mauro Verzetti · Paul Rubenstein 🔗 |
-
|
Addressing multi-centre ICU prediction of heart attack and mortality with a pseudo-dynamic explainable machine learning model
(
Poster
)
SlidesLive Video » Heart attack remain one of the greatest contributors to mortality in the United States and globally. Patients admitted to the intensive care unit (ICU) with diagnosed heart attack (myocardial infarction or MI) are more likely to suffer a secondary episode of MI and are at higher risk of death. The ICU is a setting of low-resource environment with noisy, sparse, and irregular time-series data being produced whose distribution and hence predictive use can vary between hospitals. In this study, we use two retrospective cohorts extracted from the eICU and MIMIC-IV databases, to develop a novel pseudo-dynamic machine learning framework for mortality and recurrent heart attack prediction in the ICU with interpretability. The method provides accurate prediction of both outcomes for ICU patients up to 24 hours before the event and provide time-resolved interpretability results. The performance of the framework relying on extreme gradient boosting was evaluated on a held-out test set from eICU, and externally validated on the MIMIC-IV cohort using the most important features identified by time-resolved Shapley values achieving AUCs of 91.0 (balanced accuracy of 82.3) and 85.6 (balanced accuracy of 74.5) for 6-hour prediction of mortality and recurrent heart attack respectively. We show that our framework successfully leverages time-series physiological measurements by translating them into stacked static prediction problems to be robustly predictive through time in the ICU stay and can offer clinical insight from time-resolved intepretability. |
Munib Mesinovic 🔗 |
-
|
Domain Generalization in Robust Invariant Representation
(
Poster
)
SlidesLive Video » Unsupervised approaches for learning representations invariant to common transformations are used quite often for object recognition. Learning invariances makes models more robust and practical to use in real-world scenarios. Since data transformations that do not change the intrinsic properties of the object cause the majority of the complexity in recognition tasks, models that are invariant to these transformations help reduce the amount of training data required. This further increases the model's efficiency and simplifies training. In this paper, we investigate the generalization of invariant representations on out-of-distribution data and try to answer the question: Do model representations invariant to some transformations in a particular seen domain also remain invariant in previously unseen domains? Through extensive experiments, we demonstrate that the invariant model learns unstructured latent representations that are robust to distribution shifts, thus making invariance a desirable property for training in resource-constrained settings. |
Gauri Gupta · Ritvik Kapila · KESHAV GUPTA · Ramesh Raskar 🔗 |
-
|
S2VNTM: SEMI-SUPERVISED VMF NEURAL TOPIC MODELING
(
Poster
)
SlidesLive Video » Language model based methods are powerful techniques for text classification. However, the models have several shortcomings. (1) It is difficult to integrate human knowledge such as keywords. (2) It needs a lot of resources to train the models. (3) It relied on large text data to pretrain. In this paper, we propose Semi- Supervised vMF Neural Topic Modeling (S2vNTM) to overcome these difficulties. S2vNTM takes a few seed keywords as input for topics. S2vNTM leverages the pattern of keywords to identify potential topics, as well as optimize the quality of topics’ keywords sets. Across a variety of datasets, S2vNTM outperforms existing semi-supervised topic modeling methods in classification accuracy with limited keywords provided. S2vNTM is at least twice as fast as baselines. |
Weijie Xu · Jay Desai · Srinivasan Sengamedu · Xiaoyu Jiang · Francis Iannacci 🔗 |
-
|
Adaptive Representations for Semantic Search
(
Poster
)
SlidesLive Video » Web-scale search systems use a large neural network to embed the query which is then hooked into a separate approximate nearest neighbour search (ANNS) pipeline to retrieve similar data points. Such approaches use a rigid – potentially high-dimensional – representation out of encoder to perform the entire search. This can be far from optimal accuracy-compute trade-off. In this paper, we argue that in different stages of ANNS, we can use representations of different capacities, adaptive representations, to ensure that the accuracy-compute tradeoff can be met nearly optimally. In particular, we introduce AdANNS, a novel ANNS design paradigm that explicitly leverages the flexibility and adaptive capabilities of the recently introduced Matryoshka Representations (Kusupati et al., 2022). We demonstrate that using AdANNS to construct the search data structure (AdANNS-C) provides state-of-the-art accuracy-compute tradeoff; AdANNS powered inverted file index (IVF) is up to 1.5% more accurate or up to 100× faster ImageNet-1K retrieval. We also show that matryoshka representations can power compute-aware adaptive search during inference (AdANNS-D) on a fixed ANNS (IVF) structure and be up to 16× faster for similar accuracy. Finally, we explore the applicability of adaptive representations across ANNS building blocks and further analyze the choice of matryoshka representations for semantic search. |
Aniket Rege · Aditya Kusupati · Sharan Ranjit S · Sham Kakade · Prateek Jain · Ali Farhadi 🔗 |
-
|
VISION HGNN: AN ELECTRON-MICROGRAPH IS WORTH HYPERGRAPH OF HYPERNODES
(
Poster
)
SlidesLive Video » Material characterization using electron micrographs is a crucial but challenging task with applications in various fields, such as semiconductors, quantum materials, batteries, etc. The challenges in categorizing electron micrographs include but are not limited to the complexity of patterns, high level of detail, and imbalanced data distribution(long-tail distribution). Existing methods have difficulty in modeling the complex relational structure in electron micrographs, hindering their ability to effectively capture the complex relationships between different spatial regions of micrographs. We propose a hypergraph neural network(HgNN) backbone architecture, a conceptually alternative approach, to better model the complex relationships in electron micrographs and improve material characterization accuracy. By utilizing cost-effective GPU hardware, our proposed framework outperforms popular baselines. The results of the ablation studies demonstrate that the proposed framework is effective in achieving state-of-the-art performance on benchmark datasets and efficient in terms of computational and memory requirements for handling large-scale electron micrograph-based datasets. |
Rajat sarkar · Sagar Srinivas Sakhinana · Sreeja Gangasani · Venkataramana Runkana 🔗 |
-
|
Combating Harmful Hype in Natural Language Processing
(
Poster
)
SlidesLive Video » In recent years, large multinational corporations have made claims of creating “general purpose” models that can handle many different tasks within natural language processing. Recent works from Meta for example, give theimpression that they have nearly solved machine translation tasks for more than 200 languages including 55 African languages. In this paper, we outline the harms speakers of non dominant languages have experienced due to these grandiose and inaccurate claims, ranging from diverting resources from local startups serving specific communities, to low quality datasets and models from these corporations. We urge the African NLP and machine learning communities to push back against these claims, and support smaller organizations serving their own communities. |
Asmelash Hadgu · Paul Azunre · Timnit Gebru 🔗 |
-
|
Model Compression Beyond Size Reduction
(
Poster
)
SlidesLive Video » With the current set-up, the success of Deep Neural Network models is highly tiedto their size. Although this property might help them improve their performance, itmakes them difficult to train, deploy them on resource-constrained machines, anditerate on experiments. There is also a growing concern about their environmentaland economic impacts. Model Compression is a set of techniques that are appliedto reduce the size of models without a significant loss in performance. Their use isincreasing as models grow with time. However, these techniques alter the behaviorof the network beyond reducing its size. This paper aims to draw attention tothe matter by highlighting present works with regard to Explaniability, NeuralArchitecture Search, and Fairness before finalizing with a suggestion for futureresearch directions. |
Mubarek Mohammed 🔗 |
-
|
CHATGPT IS ALL YOU NEED TO DECOLONIZE SUB-SAHARAN VOCATIONAL EDUCATION
(
Poster
)
SlidesLive Video » The advances of Generative AI with interactive capabilities that have beenachieved in the last few years offer unique opportunities for socio-economic mo-bility. Their potential for scalability, accessibility, affordability, personalizingand convenience consists a first-class opportunity for poverty-stricken countriesto adapt and modernize their educational order. As a result, this position papermakes the case for an educational policy framework that would succeed in thistransformation by prioritizing vocational and technical training over academic ed-ucation in sub-Saharan African countries. We highlight substantial applicationsof Large Language Models, tailor-made to their respective cultural background(s)and needs, that would reinforce their systemic decolonization. Lastly, we pro-vide specific historical examples of diverse states successfully implementing suchpolicies in the elementary steps of their socioeconomic transformation, in order tocorroborate our proposal to sub-Saharan African countries to follow their lead. |
Isidoros Marougkas · Isidora Chara Tourni · Georgios Grigorakis · Konstantinos M. Dafnis · Vasiliki Tassopoulou 🔗 |
-
|
Domain Shift Signal for Low Resource Continuous Test-Time Adaptation
(
Poster
)
SlidesLive Video » Test time domain adaptation has come to the forefront as a challenging scenario in recent times. Although single domain test-time adaptation has been well studied and shown impressive performance, this can be limiting when the model is deployed in a dynamic test environment. We explore this continual domain test time adaptation problem here. Specifically, we question if we can translate the effectiveness of single domain adaptation methods to continuous test-time adaptation scenario. We propose to use the given source domain trained model to continually measure the similarity between the feature representations of the consecutive batches. A domain shift is detected when this measure falls below a certain threshold, which we use as a trigger to reset the model back to source and continue test-time adaptation. We demonstrate the effectiveness of our method by performing experiments across datasets, batch sizes and different single domain test-time adaptation baselines. This can have a significant impact in a variety of applications, from healthcare and agriculture to transportation and finance. As a result, this research has the potential to greatly benefit developing countries by providing new tools and techniques for building more effective and efficient machine learning systems. |
Goirik Chakrabarty · Manogna Sreenivas · Soma Biswas 🔗 |
-
|
JumpStyle: A Framework for Data-Efficient Online Adaptation
(
Poster
)
SlidesLive Video » Research in deep learning is restrictive in developing countries due to a lack of computational resources, quality training data, and expert knowledge, which negatively impacts the performance of deep networks. Moreover, these models are prone to suffer from distribution shift during testing. To address these challenges, this paper presents a novel approach for fine-tuning deep networks in a Domain Generalization setting. The proposed framework, JumpStyle, comprises two key components: (1) an innovative initialization technique that jumpstarts the adaptation process, and (2) the use of style-aware augmentation with pseudo-labeling, in conjunction with a simple and effective test-time adaptation baseline named Tent. Importantly, JumpStyle only requires access to a pre-trained model and is not limited by the training method. The effectiveness of this approach is extensively evaluated through experiments. |
Aakash Singh · Manogna Sreenivas · Soma Biswas 🔗 |
-
|
Do Models see Corruption as we see? An Item Response Theory based study in Computer Vision
(
Poster
)
SlidesLive Video » On a given dataset, some models perform better than others. Can we examine this performance w.r.t. different strata of the dataset rather than just focusing on an aggregate metric (such as accuracy)? Given that noise and corruption are natural in real-world settings, can we study model failure under such scenarios? For a particular corruption type, do some classes become more difficult to classify than others? To answer such fine-grained questions, in this paper, we explore the use of Item Response Theory (IRT) in computer vision tasks to gain deeper insights into the behavior of models and datasets, especially under corruption. We show that incorporating IRT can provide instance-level understanding beyond what classical metrics (such as accuracy) can provide. Our findings highlight the ability of IRT to detect changes in the distribution of the dataset when it is perturbed through corruption, using latent parameters derived from IRT models. These latent parameters can effectively identify annotation errors, informative images, and class-level information while highlighting the robustness of different models under consideration. |
Charchit Sharma · Ayan Pahari · Pranoy Panda · V C Sairam Rebbapragada · Deepak Vijaykeerthy · Vineeth N Balasubramanian 🔗 |