Mitigating communication bottlenecks during parameter exchange in data-parallel DNN training

Number of patents in Portfolio can not be more than 2000

United States of America Patent

PATENT NO 11868880
APP PUB NO 20200160171A1
SERIAL NO

16276250

Stats

ATTORNEY / AGENT: (SPONSORED)

Importance

Loading Importance Indicators... loading....

Abstract

See full text

An interconnect topology for communication between GPUs in a computing system is determined. A quantity of directed spanning trees are generated for transmitting data between the GPUs using the interconnect topology and packed. The directed spanning trees define the connections between GPUs that are to be utilized for the transmission and the amount of data to be transmitted on each connection. Program code is generated for implementing the data transfer defined by the directed spanning trees. When the program code is executed, the directed spanning trees are used to pipeline the transmission of chunks of data, such as model parameters used during data-parallel DNN training, between the GPUs. The program code can also determine an optimal chunk size for data to be transferred between the GPUs.

Loading the Abstract Image... loading....

First Claim

See full text

Family

Loading Family data... loading....

Patent Owner(s)

Patent OwnerAddress
MICROSOFT TECHNOLOGY LICENSING LLCONE MICROSOFT WAY REDMOND WA 98052

International Classification(s)

Inventor(s)

Inventor Name Address # of filed Patents Total Citations
Phanishayee, Amar Redmond, US 20 227
Rangarajan, Nikhil Devanur Seattle, US 15 110
Thelin, Jorgen Redmond, US 4 4
Venkataraman, Shivaram Redmond, US 7 15
Wang, Guanhua Redmond, US 8 6

Cited Art Landscape

Load Citation

Patent Citation Ranking

Forward Cite Landscape

Load Citation

Maintenance Fees

Fee Large entity fee small entity fee micro entity fee due date
3.5 Year Payment $1600.00 $800.00 $400.00 Jul 9, 2027
7.5 Year Payment $3600.00 $1800.00 $900.00 Jul 9, 2031
11.5 Year Payment $7400.00 $3700.00 $1850.00 Jul 9, 2035
Fee Large entity fee small entity fee micro entity fee
Surcharge - 3.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge - 7.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge - 11.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge after expiration - Late payment is unavoidable $700.00 $350.00 $175.00
Surcharge after expiration - Late payment is unintentional $1,640.00 $820.00 $410.00