Youssef Elmougy

Selected Publications

1. 

   DISSERTATION: Scalable Asynchronous Actor-based Approaches for Distributed-Memory Parallel Applications

   Youssef Elmougy

   Abstract PDF

   Distributed-memory parallel applications are the backbone of large-scale data analytics and modern computing, forming foundations that enable the efficient distribution and execution of large-scale computations across heterogeneous and distributed systems. Classical primitives for data analytics, including Bulk Synchronous Parallel (BSP) models, focused on moving data to compute. Although these primitives were good enough for past High Performance Computing (HPC) applications, they are highly inefficient for the fine-grained asynchrony and data distribution required by modern large-scale data analytics applications, motivating the need for new architectural approaches for data analytics. Alleviating these challenges inherent in distributed computing environments, the underlying Fine-Grained Asynchronous Bulk Synchronous Parallel (FABS) model in the Actor-based programming system HClib-Actor proposes moving compute to data via asynchronous active messages. This Actor-based approach presents a lightweight, asynchronous computation model that utilizes fine-grained asynchronous actor messages to express point-to-point remote operations, allowing fine-grained, distributed, asynchronous, and scalable executions across systems. We realize the efficacy of the actor-based approach through exploring multiple perspectives and facets of distributed computing, including algorithm design, runtime systems, and system-level optimizations. The Algorithm perspective ("Crafting Scalable Distributed Applications") elucidates novel algorithms tailored for scalable, distributed applications in a wide variety of application domains. Through theoretical analysis and empirical evaluations, we demonstrate the superior scalability and efficiency in the asynchronous actor-based model compared to traditional parallel computing paradigms. The Runtime perspective ("Empowering Distributed Systems with Actors") delves into enhancements made to HClib-Actor, including extending termination protocols, introducing light-weight global termination schemes, and cloud environment deployment. It details optimizations incorporated into HClib-Actor, showing insights that underscore the critical role of runtime systems in facilitating scalable distributed computing. Finally, the System-Level perspective ("Architecture-Aware Optimization and Tuning") signifies the importance of aligning distributed-memory parallel applications with underlying hardware architecture characteristics for achieving peak performance by delving into hardware-specific optimizations and architectural considerations on diverse hardware platforms. We elucidate the importance of architecture-aware bindings and software-level buffer sizes on distributed computing efficiency and bandwidth. Through the realization of the three perspectives and facets of distributed computing, we offer a comprehensive scalable asynchronous actor-based approach for distributed-memory parallel applications. The synthesis of these perspectives, demonstrated through the systematic transformation methodology as an implementation guideline, provides researchers with a practical framework for adopting actor-based approaches in their own applications that are based on traditional models. By integrating these insights, we provide valuable contributions to the advancement of distributed computing, forming foundations for more efficient and scalable distributed and parallel applications in diverse computing landscapes.

2. 

   Asynchronous Distributed Actor-based Approach to Jaccard Similarity for Genome Comparisons

   Youssef Elmougy, Akihiro Hayashi, Vivek Sarkar

   ISC High Performance, 2024

   Abstract PDF Slides GitHub Repo

   The computation of genome similarity is important in computational biology applications, and is assessed by calculating the Jaccard similarity of DNA sequencing sets. However, it's challenging to find solutions that can compute Jaccard similarity with the efficiency and scalability needed to fully utilize capabilities of modern HPC hardware. We introduce a novel approach for computing Jaccard similarity for genome comparisons, founded on an actor-based programming model. Our algorithm takes advantage of fine-grained asynchronous computations, distributed/shared memory, and the Fine-grained Asynchronous Bulk-Synchronous Parallelism execution model. Our performance results on the NERSC Perlmutter supercomputer demonstrate that this approach scales to 16,384 cores, showing an average of 4.94x and 5.5x improvement in execution time at the largest scale and relevant hardware performance monitors at medium scale compared to a state-of-the-art baseline. Our approach is also able to process much larger scale genomic datasets than this baseline. We make our source code publicly available at https://github.com/youssefelmougy/jaccard-selector/.

3. 

   A Distributed, Asynchronous Algorithm for Large-Scale Internet Network Topology Analysis

   Youssef Elmougy, Akihiro Hayashi, Vivek Sarkar

   IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID), 2024. 🏅Won the IEEE TCSC SCALE 2024 Award

   Abstract PDF SCALE 2024 Award Slides Demo Video

   With the growing complexity of modern internet networks, we introduce a distributed, asynchronous, and scalable algorithm tailored for determining the centrality and importance of various levels of the global internet network topology at a massive scale. By utilizing triangle formations and neighborhood densities in these complex networks, our algorithm provides valuable insights into the structural importance of individual nodes and the intricate relationships among interconnected entities. Focusing on the global internet's prime components — routers, IP addresses, and Autonomous Systems — the algorithm provides a scalable solution extending to the global internet infrastructure, datacenters, cloud networks, and private networks. By taking advantage of the parallelism, distributed processing, efficient communication, and fine-grained asynchronous execution through an Actor-based programming system, our algorithm is capable of efficiently processing and performing rapid computations on large-scale internet networks. We perform scalability studies on the NERSC Perlmutter supercomputer and the Georgia Tech HPC PACE cluster using three large-scale real-world network datasets and one large-scale synthetic network dataset, achieving up to 91.7% parallel efficiency scaling out to 2K cores and reducing execution time from 5.7 hours to 18.3 seconds, while performing 105.4x better on average compared to related approaches. Our research contributes significantly to facilitating enhanced network management, fault tolerance and resilience, and security, but as well to improving overall network stability, reducing latency, and optimizing energy consumption across the vast internet network devices environment.

4. 

   Demystifying Fraudulent Transactions and Illicit Nodes in the Bitcoin Network for Financial Forensics

   Youssef Elmougy, Ling Liu

   ACM KDD Conference on Knowledge Discovery and Data Mining (KDD), 2023

   Abstract PDF ArXiv PDF Slides GitHub Repo

   Blockchain provides the unique and accountable channel for financial forensics by mining its open and immutable transaction data. A recent surge has been witnessed by training machine learning models with cryptocurrency transaction data for anomaly detection, such as money laundering and other fraudulent activities. This paper presents a holistic applied data science approach to fraud detection in the Bitcoin network with two original contributions.

   First, we contribute the Elliptic++ dataset, which extends the Elliptic transaction dataset to include over 822k Bitcoin wallet addresses (nodes), each with 56 features, and 1.27M temporal interactions. This enables both the detection of fraudulent transactions and the detection of illicit addresses (actors) in the Bitcoin network by leveraging four types of graph data: (i) the transaction-to-transaction graph, representing the money flow in the Bitcoin network, (ii) the address-to-address interaction graph, capturing the types of transaction flows between Bitcoin addresses, (iii) the address-transaction graph, representing the bi-directional money flow between addresses and transactions (BTC flow from input address to one or more transactions and BTC flow from a transaction to one or more output addresses), and (iv) the user entity graph, capturing clusters of Bitcoin addresses representing unique Bitcoin users. Second, we perform fraud detection tasks on all four graphs by using diverse machine learning algorithms. We show that adding enhanced features from the address-to-address and the address-transaction graphs not only assists in effectively detecting both illicit transactions and illicit addresses, but also assists in gaining in-depth understanding of the root cause of money laundering vulnerabilities in cryptocurrency transactions and the strategies for fraud detection and prevention. The Elliptic++ dataset is released at https://www.github.com/git-disl/EllipticPlusPlus.

5. 

   Highly Scalable Large-Scale Asynchronous Graph Processing using Actors

   Youssef Elmougy, Akihiro Hayashi, Vivek Sarkar

   IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID), 2023. 🏅Won the IEEE TCSC SCALE 2023 Award

   Abstract PDF SCALE 2023 Award Slides Demo Video

   With the accelerating growth of Big Data, real-world graph processing applications now need to tackle graphs with billions of vertices and trillions of edges, thereby increasing the demand for effective solutions to application scalability. Unfortunately, current approaches to implementing these applications on modern HPC systems exhibit poor scale-out performance with increasing numbers of nodes. The scalability challenges for these applications are driven by large data sizes, synchronization overheads, and fine-grained communications with irregular data accesses and poor locality. This paper presents the scalability of a novel Actor-based programming system, which provides a lightweight runtime that supports fine-grained asynchronous execution and automatic message aggregation atop a Partitioned Global Address Space (PGAS) communication layer. Evaluations of the Jaccard Index and PageRank applications on the NERSC Perlmutter system demonstrate nearly perfect scaling up to 1,000 nodes and $64$K cores (one-third of the targeted 3000-nodes for Perlmutter). In addition, our Actor-based implementations of Jaccard Index and PageRank executed with parallel efficiencies of 85.7% and 63.4% for the largest run of 64K cores. This performance represents a 29.6x speedup relative to UPC and OpenSHMEM versions of PageRank.

6. 

   A Fine-grained Asynchronous Bulk Synchronous parallelism model for PGAS applications

   Sri Raj Paul, Akihiro Hayashi, Kun Chen, Youssef Elmougy, Vivek Sarkar

   Journal of Computational Science, 2023

   Abstract PDF GitHub Repo

   The Partitioned Global Address Space (PGAS) model is well suited for executing irregular applications on cluster-based systems, due to its efficient support for short, one-sided messages. Separately, the actor model has been gaining popularity as a productive asynchronous message-passing approach for distributed objects in enterprise and cloud computing platforms, typically implemented in languages such as Erlang, Scala or Rust. To the best of our knowledge, there has been no past work on using the actor model to deliver both productivity and scalability to irregular PGAS applications with large number of small messages.

   In this paper, we introduce a new programming system for PGAS applications, in which point-to-point remote operations can be expressed as fine-grained asynchronous actor messages. In our approach, the programmer does not need to worry about programming complexities related to message aggregation and termination detection. Our approach can be viewed as extending the classical Bulk Synchronous Parallelism model with fine-grained asynchronous communications within a phase or superstep. We believe that our approach offers a desirable point in the productivity-performance space for PGAS applications, with more scalable performance and higher productivity relative to past approaches. Specifically, for seven irregular mini-applications from the Bale Kernels and three graph kernels executed using 2048 cores in the NERSC Cori system, our approach shows geometric mean performance improvements of >= 20x relative to standard PGAS versions (UPC and OpenSHMEM) while maintaining comparable productivity to those versions.

   This is an extended version of the conference paper “A Productive and Scalable Actor-Based Programming System for PGAS Applications” (Paul et al., 2022)[1] from ICCS 2022.

7. 

   Diagnosing the interference on CPU-GPU Synchronization Caused by CPU Sharing in Multi-Tenant GPU Clouds

   Youssef Elmougy, Weiwei Jia, Xiaoning Ding, Jianchen Shan

   IEEE International Performance, Computing, and Communications Conference (IPCCC), 2021

   Abstract PDF Slides

   The GPU-accelerated cloud, enabled by maturing GPU virtualization techniques, has become the most attractive platform for high-performance computing and machine learning workloads. However, it is notoriously challenging to build the multi-tenant GPU cloud where resources, like CPUs and GPUs, can be shared. One well-known and heavily studied reason is that workloads suffer from poor performance isolation and low GPU utilization when GPUs are shared. But little attention has been paid to another fundamental yet under studied problem: how sharing CPUs among GPU instances could affect the workload performance?

   Targeting this problem, the paper conducts experiments to measure the performance slowdown and vGPU utilization decrease under interference from CPU sharing. The results show that GPU workloads suffer from poor and unpredictable performance and heavy vGPU under-utilization because of CPU sharing. We find that such interference is the result of the complex interplay between the characteristics of CPU-GPU interactions and the special behavior of shared vCPUs: vCPU discontinuity. To diagnose how vCPU discontinuity causes the interference, the paper leverages NVIDIA Nsight Systems for fine-grained profiling and has the following findings: 1) vCPU discontinuity causes inefficient CPU-GPU synchronizations; 2) vCPU discontinuity delays task offloading to the vGPU; 3) Polling-based CPU-GPU synchronization suffers from interference more than blocking-based CPU-GPU synchronization; 4) GPU workloads with frequent task offloads and synchronizations are more vulnerable. Based on the findings, the paper proposes a novel polling-then-blocking CPU-GPU synchronization primitive. Evaluation shows that it can improve the performance by 4.2x.

8. 

   Anomaly Detection on Bitcoin, Ethereum Networks Using GPU-accelerated Machine Learning Methods

   Youssef Elmougy, Oliver Manzi

   IEEE International Conference on Computer Theory and Applications (ICCTA), 2021

   Abstract PDF Slides

   Blockchain technology is continually gaining momentum, with applications expanding in sectors beyond digital assets and financial services. With the existence of a public distributed ledger, the validity of transactions and accounts on the blockchain can be easily reviewed. Nevertheless, there are malicious persons that attempt to fraud cryptocurrency holders, undermining the reliability of the blockchain. This study focuses on identifying fraudulent transactions and accounts by detecting anomalies in the Bitcoin and the Ethereum transaction networks, the two largest cryptocurrencies. By leveraging GPU-accelerated machine learning models, including Support Vector Machines, Random Forest, and Logistic Regression, we draw the metadata of over 30 million transactions on the Bitcoin network and confirmed transactions from over 500 thousand accounts on the Ethereum network. We offer insight into feature importance through sensitivity analysis, as well as train accurate models that allow for method adoption in automated fraud detection systems. The trained models achieve an accuracy and recall of 96.9% and 0.987 on the Bitcoin dataset, and 80.2% and 0.835 on the Ethereum dataset. The study of anomaly detection in the cryptocurrency blockchain done in this paper can be generalized to other blockchain networks, including health service blockchains, public sector blockchains, and financial intelligence blockchains.