Our team is at the ready to create a customized plan for you to optimize and scale your business. Contact us today to set up a call.

Curated content and latest news for MangoBoost solutions

Record Breaking Performance for DPU-accelerated NVMe/TCP system submission for MLPerf™ Storage.

Best-performing ethernet-based systems result in MLPerf Storage v2.0.

Near-local SSD performance by accelerating the entire NVMe/TCP stack fully on hardware on our DPU solutions and achieving line-rate performance on 400G network.

Lower TCO and hardware cost for high-performance storage.

Easy to deploy on standard systems. Our DPU solutions provide the standard interface in both hardware and software without modification to the existing infrastructure.

Try it now. If interested in the MangoBoost AI solutions, please contact us at contact@mangoboost.io

In this MLPerf Storage 2.0 round, MangoBoost is proud to announce how our Mango StorageBoostTM addresses the scalability and performance challenges with its patent-pending DPU technology. StorageBoostTM technology offloads infrastructure and data-processing tasks from the host to datacenter-deployable FPGA PCIe cards. By deploying our best-in-class NVMe/TCP Initiator on the host and NVMe/TCP Target on the storage server, we further push the performance of 3D-UNet training on both A100 and H100 configuration making it the leading Ethernet-based storage system for AI training.

DPU-accelerated NVMe/TCP submission for MLPerf™ Storage Solutions

Modern high-performance computing applications, including large-language models, are growing in complexity and size. This trend places significant demands not only on compute and memory, but also on storage systems. Storage I/O bottlenecks can lead to underutilized GPUs. As models increase in size, the need for high-performance, high-capacity storage becomes even more critical. While technologies like NVMe/TCP enable remote block devices for ML systems and help meet capacity demands, these remote storage solutions often experience performance degradation due to the CPU overhead associated with NVMe/TCP software stack.

To address this performance bottleneck, DPU-accelerated Mango StorageBoostTM Initiator and Mango StorageBoostTM Target fully offload NVMe/TCP to the hardware, delivering line-rate performance and allows remote storage to operate at the speed similar to a local high-performance NVMe.

In this round, we have set up our evaluation for MLPerf Storage 2.0 Training workload by connecting the host with Mango StorageBoostTM Initiator DPU with the storage server with Mango StorageBoostTM Target through ethernet switch with a 400G network connection between the host and the storage server as shown in Figure 1. The host then emulates the demand of ML workloads by running multiple copies of emulated A100 and H100 3D-UNet traces on the storage system with the goal of testing the maximum number of GPUs that can be connected to the storage system.

MangoBoost shows the best-performing Ethernet-based system result in MLPerf Storage v2.0 and provides near-local SSD performance.

Measuring the performance efficiency of a remote storage system targeting ML workloads can be done through two key metrics: number of GPUs that the storage system can handle normalized by the number of clients, and the number of GPUs normalized by the network bandwidth. Figure 2 shows the overall GPU scalability per unit relative to other submitters in the Fabric-attached Block Storage category (both NVMe/TCP and NVMe/RDMA).These results show how MangoBoost’s DPU solution delivers the best performance: 6.2x for 3D-UNET A100 and 1.25x-7.5x for 3D-UNET H100, and how offloading NVMe/TCP to hardware allows more GPUs to be connected to the storage system.

Figure 3 shows the performance measured as the number of GPUs that can be connected normalized by the network bandwidth for both NVMe/RDMA and NVMe/TCP submitters. This result highlights the efficiency of our solution in utilizing the available network bandwidth as MangoBoost solutions provide the best performance: 1.57x for 3D-UNET A100 and 1.27x-2.05x for 3D-UNET H100.

Figure 4 also provides further performance breakdown across two of our MLPerf Storage submissions (MLPerf Storage 1.0 and MLPerf Storage 2.0), showing that the DPUs can achieve near line-rate performance regardless of GPU architectures (H100 and A100 architectures).

The combination of the Mango StorageBoostTM Initiator DPU on the host and the Mango StorageBoostTM Target DPU on the target also allows ML workloads to operate remote block storage at performance near that of local storage. This is highlighted in Figure 5 through the performance comparison between MangoBoost solutions against local SSD baselines. Our system deployed four Solidigm D7-PS1030s on the target. We normalized the score by the number of SSDs to compare with other non-Fabric attached submissions using a single SSD.

Industry-leading DPU storage solutions, delivering higher performance and lower TCO

While MangoBoost is the only submitter focusing on DPU solutions, this section now focuses on the performance comparison of Mango StorageBoost DPUs against the most apple-to-apple DPU solution available in the market – the NVIDIA BlueField-3 400G DPU, using the same MLPerf Storage 2.0 methodology but replacing MangoBoost DPU with NVIDIA BlueField-3 as shown in Figure 6.

To highlight our performance and cost-efficiency, we rerun MLPerf Storage 2.0 on 3D-UNET benchmarks using the same methodology as the official MLPerf Storage 2.0. Figure 7 compares the scores of BlueField-3 running NVMe/TCP and NVMe/RDMA with host CPU and MangoBoost NVMe/TCP full offloading. The results demonstrate that Mango StorageBoost significantly outperforms BlueField-3 NVMe/TCP and even shows higher performance than BlueField-3 NVMe/RDMA.

Additionally, we also ran the Llama3-8B checkpointing workload on the same setup (this result is not included in v2.0, we will submit the checkpointing results in the next round). Figure 8 shows the comparison of checkpointing performance for different storage configurations. The setup used only one client and did not fully saturate the given bandwidth. The I/O pattern of MLPerf Storage v2.0 checkpointing is not optimized for throughput; shorter storage system latency is crucial for achieving higher throughput on a single client. The results demonstrate that Mango StorageBoost significantly outperformed BlueFIeld-3 (NVMe/TCP) and achieved near local SSD performance.

The superior performance advantage directly translates into TCO reduction. Figure 9 shows the TCO reduction of MangoBoost DPU compared to BlueField-3; as more storage is deployed, the TCO gap widens. For more details, please check our 

, which provides further analysis on throughput and cost breakdown.

Mango StorageBoost™ Traditional Storage, JBoF, JBoD and EBoF Solutions.

Mango StorageBoost™ is a high-performance vendor-agnostic easy-to-use solution that allows AI systems running on any host and GPUs to scale their ML workloads.

Solution 1: Mango StorageBoost™- NVMe/TCP Initiator (NTI).

Mango StorageBoost™- NVMe/TCP Initiator (NTI) is a high-performance NVMe-oF initiator solution that unlocks the full potential of storage disaggregation. By offloading the entire NVMe/TCP stack into hardware, NTI delivers unmatched full-duplex line-rate performance with zero CPU consumption.

Mango StorageBoost™- NVMe/TCP Initiator (NTI) integrates seamlessly into NVMe/TCP initiator servers. By exposing the DPU as a standard NVMe PCIe device, existing storage systems can leverage NTI without the need for any SW modifications. Furthermore, NTI can connect to any NVMe/TCP target servers over a standard TCP/IP network.

Solution 2: Mango StorageBoost™- NVMe/TCP Target (NTT).

Mango StorageBoost™ - NVMe/TCP Target is a groundbreaking NVMe-oF solution, powering the storage system with outstanding performance. It enables disaggregated storage servers to be connected to the network through the standard TCP/IP protocol stack. By fully offloading and accelerating both the TCP/IP and NVMe-oF processing, it entirely eliminates the CPU intervention from the I/O path.

On top of Mango NetworkBoost™ - TCP/IP, NTT decapsulates the TCP packets into NVMe-oF commands and translates them into standard NVMe commands. By utilizing PCIe peer-to-peer technology, NTT directly sends those commands to the peer NVMe SSDs without any CPU consumption. Additionally, its modular and flexible architecture also enables more dynamic customer-driven storage systems such as data reduction and reliability.

Solution 3: Mango StorageBoost™- GPU Storage Boost (GSB).

Mango StorageBoost™- GPU Storage Boost (GSB) is an essential software solution designed to dramatically enhance the efficiency of GPU data movement, enabling unprecedented performance capabilities. By establishing a direct data path for DMA (Direct Memory Access) transfers between GPU memory and storage, GSB eliminates the need for bounce buffers through the CPU, and bypasses CPU involvement entirely. It facilitates peer-to-peer communication within GPU systems, regardless of whether the storage is local or remote. Furthermore, GSB offers a nearly identical syntax and semantic structure to traditional POSIX File APIs, making integration straightforward. This allows for efficient, filesystem-based data management with minimal adaptation, enabling users to harness the full power of the GPU systems without sacrificing ease of use.

When combined with Mango StorageBoost™- NVMe/TCP Initiator (NTI), GSB enables highly efficient GPU data movement, even over remote storage systems based on the TCP/IP transport layer, without any CPU involvement. Typically, handling TCP/IP protocols and the NVMe/TCP stack requires CPU intervention and memory copying, which introduces latency and reduces overall efficiency. NTI addresses these challenges by offloading these tasks to dedicated hardware, allowing the host to leverage the DMA engine as if it were directly connected to an NVMe PCI device. NTI effectively removes the performance and efficiency constraints associated with CPU resources during remote storage access, while GSB eliminates similar limitations for GPU data movement. By utilizing both solutions together, users can unlock unparalleled synergies, dramatically improving data flow efficiency and reducing system bottlenecks.

With the three solutions combined, MangoBoost DPU not only can be used for a traditional remote block storage, but can be used to manage headless storage network including Just-a-bunch-of-Flash (JBoF), Just-a-bunch-of-Disk (JBoD), and Ethernet-bunch-of-Flash (EBoD) to further lower the TCO of high-performance storage system.

Interested in how our DPU solution can deliver higher performance and lower your TCO, try our solutions today by contacting us at 

MangoBoost Demonstrates Record-Breaking Performance in AI Training Storage Solution with MLPerf® Storage v2.0 

MangoBoost, a provider of cutting-edge system solutions designed to maximize AI data center efficiency, has set a new industry benchmark with its latest MLPerf Training v5.0 submission. The company’s Mango LLMBoost™ AI Enterprise MLOps Software together with Mango GPUBoost™ RoCEv2 RDMA solution has demonstrated unparalleled performance on AMD Instinct™ MI300X GPUs, delivering the first-ever MLPerf multi-node results on AMD GPUs.

RoCEv2 RDMA solution on AMD Instinct™ MI300X servers.

First-ever multi-node MLPerf submission using AMD Instinct™ GPUs

, achieving the fastest Llama2-70B-LoRA fine-tuning on AMD hardware at just 

Customizable RoCEv2-based DPU-powered communication stack

 ensures seamless multi-node scalability—purpose-built for high-throughput AI and cloud infrastructure.

 (95–100% efficiency) on additional models like Llama2-7B and Llama3.1-8B, validating real-world scalability beyond benchmark tuning.

Model-agnostic, hardware-flexible, and deployment-ready

 across public cloud or on-premise—MangoBoost enables enterprise-grade training anywhere.

End-to-End Software-Hardware Stack for Performance at Scale

At the heart of this achievement is MangoBoost’s integrated platform, illustrated in Figure 1:

, featuring patent-pending technology for auto-tuning, model parallelism, batch scheduling, and memory management.

, enabling customizable, low-latency, high-throughput communication across nodes using RDMA over converged Ethernet.

First-Ever Multi-Node MLPerf Submission on AMD GPUs

Llama2-70B-LoRA fine-tuning across four AMD Instinct™ MI300X nodes (32 GPUs)

. This breakthrough enabled us to achieve:

The fastest Llama2-70B-LoRA fine-tuning time on AMD GPUs

 using MangoBoost’s end-to-end AI stack

 that demonstrate the maturity of AMD MI300X for real-world AI workloads

first-ever multi-node MLPerf Training result on AMD Instinct™ GPUs

high-performance large-scale AI training is now possible beyond vendor-locked platforms

, our AMD MI300X runs—spanning 8, 16, and 32 GPUs—exhibit 

scaling behavior comparable to prior NVIDIA H100 submissions in MLPerf Training v4.1

MI300X can deliver competitive multi-node performance

 backed by our software-hardware co-design.

This milestone was also recognized by industry leaders:

"

I'm excited to see MangoBoost's first MLPerf Training results, pairing their LLMBoost AI Enterprise MLOps software with their RoCEv2-based GPUBoost DPU hardware to unlock the full power of AMD GPUs, demonstrated by their scalable performance from a single-node MI300X to 2- and 4-node MI300X results on Llama2-70B LoRA. Their results underscore that a well-optimized software stack is critical to fully harness the capabilities of modern AI accelerators.

We congratulate Mangoboost on their MLPerf 5.0 training results on AMD GPUs and are excited to continue our collaboration with them to unleash the full power of AMD GPUs. In this MLperf Training submission, MangoBoost has achieved a key milestone in demonstrating training results on AMD GPUs across 4 nodes (32 GPUs). This showcases how the AMD Instinct™ MI300X GPUs and ROCm™ software stack synergize with MangoBoost's LLMBoost™ AI Enterprise software and GPUBoost™ RoCEv2 NIC.

, Fellow, AI Performance Design Engineering, AMD

As our first multi-node MLPerf Training submission on AMD Instinct™ MI300X, this result already demonstrates strong performance and scalability. With continued tuning and deeper hardware-software co-optimization, we expect even stronger results in future rounds.

Demonstrating Near-Linear Scaling on Llama2-7B and Llama3.1-8B

MangoBoost has benchmarked additional large language models—including 

 of its LLMBoost™ platform. Unlike our MLPerf Training v5.0 submission, these results are from 

, as MLPerf currently does not support these two models.

As shown in Figure 3, both models exhibit 

 as GPU count increases, confirming that our 

MLOps software and RDMA communication stack

 can efficiently orchestrate training across a variety of cluster sizes and workloads.

architecture-aware scheduling and optimized RDMA communication

, adapting well to diverse real-world training scenarios.

MangoBoost’s LLMBoost™ platform is production-ready, scalable, and optimized for modern AI training workloads—regardless of model size or structure.

Mango GPUBoost™ RDMA: Unlocking Multi-Node Training at Scale

Multi-node training isn’t just about adding more GPUs—it's about ensuring those GPUs communicate 

 that outpace standard RNICs in the market.

Up to 3.2Tbps RDMA (RoCEv2) throughput in total (up to 400Gbps per NIC)

Peer-to-peer communication between RNIC and GPU (i.e., ROCmRDMA support)

Customized congestion control and load balancing

Workload-specific optimization hooks using RoCEv2 extended headers

High connection scalability: Maintains line-rate throughput at thousands of QPs—3.4× higher than standard RNICs under heavy load

MangoBoost’s RDMA stack ensures that your training isn’t bottlenecked by the network—one of the biggest pain points in distributed deep learning today.

Collaboration with AMD: Unlocking the Full Potential of MI300X GPUs

MangoBoost’s ground-breaking MLPerf performance was made possible through deep collaboration with our partner 

 with industry-leading compute density and massive memory bandwidth.

Together, AMD’s ROCm platform and MangoBoost’s LLMBoost stack deliver an 

AI training and finetuning solution that is fast, scalable, and easy to deploy

—whether on a single node or across a multi-node cluster. MangoBoost’s GPUBoost RoCEv2 RDMA solutions further pushes the performance and scalability of the GPUs, and allows training across multiple MI300X nodes.

Train Anywhere with Mango LLMBoost™: Model-Agnostic, Hardware-Flexible, and Deployment-Ready

Modern AI infrastructure demands flexibility—not lock-in. MangoBoost’s LLMBoost AI Enterprise MLOps Software is built to support 

enterprise-grade LLM training wherever you need it

, whether that’s on public cloud, private datacenters, or hybrid environments.

MangoBoost supports a broad range of open and custom LLM architectures out of the box:

Whether you're training from scratch or fine-tuning with adapters, MangoBoost’s stack adapts to your model with minimal configuration.

Our full-stack platform is optimized for multiple GPU backends, including:

 — deeply optimized with ROCm and RDMA acceleration

 — compatible via standard CUDA/NCCL support

Future support for other AI accelerators via our abstraction layer

 and can make hardware choices based on price, performance, or availability.

☁️ Cloud & On-Premise Deployment Ready

MangoBoost offers seamless deployment modes:

: Preconfigured Docker images for AWS, Azure, and GCP

: Hardware-aware deployment scripts and RDMA stack support

: Easily integrated into your existing scheduling and orchestration frameworks

—any model, any hardware, any environment.

MangoBoost AI Infrastructure Hardware Solutions based on DPUs

Beyond LLMBoost software and GPUBoost RDMA solutions, MangoBoost offers hardware acceleration solutions based on Data Processing Unit (DPUs) for AI and cloud infrastructure, such as:

 – Offloads TCP/IP stack to free up CPU resources

 – High-performance NVMe initiator/target stack for scalable AI storage and JBoF solutions

This is just the beginning. Our R&D team is already implementing next-generation communication optimizations, hybrid parallelism strategies, topology-aware scheduling, and application-specific hardware acceleration to push multi-node performance even further.

If your organization is building large-scale AI models or exploring alternative GPU architectures, MangoBoost is your partner in unlocking full-stack efficiency and scalability.

To learn more or request a demo, contact us at 

Let us help you deploy high-performance, cost-efficient, and vendor-flexible AI infrastructure—powered by MangoBoost’s advanced MLOps software and communication stack.




The performance claims in this document are based on the internal cluster environment. Actual performance may vary depending on the server configuration. Software and workloads used in performance tests may have been optimized for performance only on MangoBoost products. Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. Results that are based on pre-production systems and components as well as results that have been estimated or simulated using MangoBoost reference platform for informational purposes only. Results may vary based on future changes to any systems, components, specifications, or configurations. Statements in this document that refer to future plans or expectations are forward-looking statements. These statements are based on current expectations and involve many risks and uncertainties that could cause actual results to differ materially from those expressed or implied in such statements. MangoBoost does not guarantee any specific outcome. Nothing contained herein is, or shall be relied upon as, a promise or representation or warranty as to future performance of MangoBoost or any MangoBoost product. The information contained herein shall not be deemed to expand in any way the scope or effect of any representations or warranties contained in the definitive agreement for MangoBoost products.

The information contained herein may not be reproduced in whole or in part without prior written consent of MangoBoost. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. MangoBoost assumes no obligation to update or otherwise correct or revise this information and MangoBoost reserves the right to make changes to the content hereof from time to time without any notice. Nothing contained herein is intended by MangoBoost, nor should it be relied upon, as a promise or a representation as to the future.

MANGOBOOST MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION.

MangoBoost Sets a New MLPerf Training v5.0 Standard for Multi-nodes Llama2-70B-LoRA on AMD Instinct™ MI300X GPU!

 in its ground-breaking MLPerf Inference v5.0 submission on 

, beating all other MLPerf inference results to date

 vs. 32x H100 GPUs published MLPerf result

, hardware-flexible, deployable anywhere (cloud/on-prem)

🛠️Beyond LLMBoost software, MangoBoost also offers 

 is proud to announce a ground-breaking MLPerf inference V5.0 submission: we are the 

first-ever multi-node MLPerf submission using AMD Instinct™ GPUs, 

the highest result in Llama2-70B offline inference category in history, beating all other submitted results

, MangoBoost has not only showcased the scalability of 

AMD Instinct for large-scale AI workloads

robustness and high performance of our MLOps enterprise software - LLMBoost

🔧 LLMBoost: Easy to Use, High Performance, Scalable, and GPU-flexible

 is an easy-to-use AI Enterprise MLOps software that offers 

open 50+ models like LLaMA, Qwen, DeepSeek

, with no custom orchestration required. Summary of LLMBoost highlights are as follows:

or full control, security, and performance

 – Up to 138× faster than Ollama and 74× faster than HuggingFace

significantly more efficient than vLLM and HuggingFace TGI

LLMBoost integrates several patent-pending technologies that further enhance the performance and scalability, including:

 – Efficiently distributes large models across GPUs and nodes

– Optimizes runtime parameters based on workload characteristics

– Dynamically adapts LLM’s memory usage to maximize GPU utilization

– Flexible deployment across multiple inference stages

No vendor lock-in. No complex setups. Just performance, efficiency, and simplicity

🤝 MangoBoost MLPerf Inference Result: Powered by AMD MI300X, Supercharged by LLMBoost Software

MangoBoost record-breaking MLPerf performance was made possible through deep collaboration with 

AI inference solution that is fast, scalable, and easy to deploy

—whether on a single node or across a multi-node cluster.

🚀 LLMBoost on 32x MI300s Outperforms 32x H100s — At Lower Cost

LLMBoost delivers a 24% higher throughput

 than the previous best multi-node MLPerf result from Juniper Networks using 

 which reported 82,749 tokens per second (TPS) on Llama2-70B offline inference.

LLMBoost with AMD Instinct MI300X achieved a leap to 

surpassing the performance of a 32x H100 setup. 

With each MI300X GPU estimated price of around $15,000–$17,000, compared to $32,000–$40,000 per H100— LLMBoost on AMD Instinct MI300X 

offers up to 62% savings in GPU cost while delivering superior inference throughput.

the LLMBoost with MI300X system delivers approximately 2.8× more inference throughput per $1,000 spent than the H100-based system

, making it the clear choice for high-performance, budget-conscious deployments.

📈 Highest Llama2-70B Offline Inference Results in MLPerf v5.0

MangoBoost achieved the highest LLaMA2-70B offline inference performance

 in MLPerf v5.0 compared with all other submitters (e.g., Nvidia, Intel, Google). Compared to the previous 8-GPU MI300X submission,

 MangoBoost’s latest MLPerf results showcase 

from a single-GPU to a distributed multi-node, multi-GPU production cluster.

Whether you're running a handful of inference threads or deploying a high-throughput LLM service, 

LLMBoost ensures you get the maximum efficiency at any scale

Want to replicate our MLPerf v5.0 results? Try it yourself on 

] or follow the step-by-step instructions in our 

🔧 Additional Result Highlights on 8x Nvidia A100 GPUs on AWS

Beyond the specific MLPerf results mentioned above, LLMBoost has been extensively evaluated with many platforms and GPUs. As an example, in our evaluation using an 

8×NVIDIA A100 GPU server from AWS, LLMBoost delivers up to 138× faster inference compared to Ollama

outperforms both HuggingFace TGI and vLLM across all model sizes tested

-R1-Distill-Qwen-32B, and LLaMA3.1-8B. In terms of cost-efficiency, LLMBoost also leads the pack with 

Whether you're optimizing for performance, cost, or deployment simplicity, 

LLMBoost offers unmatched efficiency and scalability across a wide range of LLMs.

🧪 From Research to Production in Minutes

 bridges the gap between experimentation and deployment, allowing developers and data teams to serve powerful open-source models 

, not days. Whether you're testing a quantized 

a consistent, streamlined experience at every scale

Getting started is as simple as a running single command:

⚡MangoBoost AI Infrastructure Hardware Solutions based on DPUs

Beyond LLMBoost software, MangoBoost offers hardware acceleration solutions based on Data Processing Unit (DPUs) for AI and cloud infrastructure, such as:

 – RDMA acceleration for multi-node inference/training via RoCEv2

 – High-performance NVMe initiator/target stack for scalable AI storage

References
[1] Benchmark Suite Results for MLPerf Inference: Datacenter
•https://mlcommons.org/benchmarks/inference-datacenter/
[2] Tom's Hardware – H100 vs. MI300X Pricing
•https://www.tomshardware.com/tech-industry/artificial-intelligence/nvidias-h100-ai-gpus-cost-up-to-four-times-more-than-amds-competing-mi300x-amds-chips-cost-dollar10-to-dollar15k-apiece-nvidias-h100-has-peaked-beyond-dollar40000
[3] Mango LLMBoost for AMD MI300x on Azure Marketplace
•https://azuremarketplace.microsoft.com/en-us/marketplace/apps/mango_solution.mango_llmboost_amd?tab=Overview
[4] MangoBoost’sMLPerfGitHub README
•https://github.com/mlcommons/inference_results_v5.0/blob/main/closed/MangoBoost/README.md



MangoBoost Sets The Highest MLPerf Inference v5.0 Result in History for Llama2-70B Offline!

 unlocks the power of state-of-the-art GPUs for AI training systems with multi-GPUs. As an example, MangoBoost demonstrates linear scaling using our RDMA solution, for MLperf Training v4.1 Llama2-70B-LoRA workload (BF16), using 24x AMD MI300X GPUs across multiple nodes.

 to allow direct communication across GPUs to scale AI systems. Furthermore, MangoBoost HW-based 

in-line customizable congestion control and custom headers

 enable even more optimal solutions at scale.

Mango GPUBoost RDMA™ is ready to ship. Pre-orders are open for customers who would like to accelerate their AI infrastructure.

 and plans to offer an Ultra Ethernet solution in the near future.

Efficient GPU-to-GPU communication across multiple nodes is essential for large-scale AI training workloads, where dozens to thousands of GPUs work together to train complex models. 

 has emerged as a key technology for enabling this scalability, as it allows a direct data transfer between GPU memory on different nodes without the help of the CPU host significantly reducing the latency of GPU-to-GPU data transfers. Among various RDMA solutions,

has been shown to be a promising solution that offers unique advantages over traditional Infiniband for large-scale AI training environments[1].

One of RoCEv2’s primary advantages is its 

. Because it uses an IP header for routing, RoCEv2 can scale over larger, more complex network topologies than Infiniband. This IP-based routing capability makes RoCEv2 especially appealing in hyperscale AI infrastructures, where scalability is crucial for managing massive AI models and multi-node GPU configurations.

, which directly impacts the physical layout of the AI networking infrastructure. Ethernet switches typically support more ports than Infiniband switches of the same generation, reducing the level of switches that introduces latency and potential oversubscription. With RoCEv2’s use of high-port-count Ethernet switches, system architects can create

Blog

Ready for
a boost?

Blog

MangoBoost Demonstrates Record-Breaking Performance in AI Training Storage Solution with MLPerf® Storage v2.0

MangoBoost Sets a New MLPerf Training v5.0 Standard for Multi-nodes Llama2-70B-LoRA on AMD Instinct™ MI300X GPU!

MangoBoost Sets The Highest MLPerf Inference v5.0 Result in History for Llama2-70B Offline!

MangoBoost Demonstrates Mango GPUBoost RDMA™ with MLPerf Training v4.1 on a Multi-Node System Powered by AMD MI300X GPUs

MangoBoost Demonstrates Turn-Key Storage Solution for AI Training with MLPerf® Storage v1.0

MangoBoost Demonstrates a Turn-Key LLM Inference Serving Solution on AMD Instinct™ MI300X GPUs with the MLPerf® Inference v4.1 Llama Benchmark

Ready for a boost?

Ready for
a boost?