Token-position handling for sequence based neural networks

Number of patents in Portfolio can not be more than 2000

United States of America Patent

PATENT NO 11544537
APP PUB NO 20210319288A1
SERIAL NO

16848748

Stats

ATTORNEY / AGENT: (SPONSORED)

Importance

Loading Importance Indicators... loading....

Abstract

See full text

Embodiments of the present disclosure include a method for token-position handling comprising: processing a first sequence of tokens to produce a second sequence of tokens, wherein the second sequence of tokens has a smaller number of tokens than the first sequence of tokens; masking at least some tokens in the second sequence to produce masked tokens; moving the masked tokens to the beginning of the second sequence to produce a third sequence; encoding tokens in the third sequence into a set of numeric vectors in a first array; and processing the first array in a transformer neural network to determine correlations among the third sequence, the processing the first array producing a second array.

Loading the Abstract Image... loading....

First Claim

See full text

Family

Loading Family data... loading....

Patent Owner(s)

  • MICROSOFT TECHNOLOGY LICENSING, LLC.

International Classification(s)

Inventor(s)

Inventor Name Address # of filed Patents Total Citations
Bharadwaj, Sujeeth Subramanya Milpitas, US 5 14
Kulkarni, Saurabh Mohan Redmond, US 6 24
Mitra, Tiyasa San Jose, US 20 18
Tremblay, Marc Bellevue, US 272 6044
Wagner, Andrew Cupertino, US 31 562

Cited Art Landscape

Load Citation

Patent Citation Ranking

Forward Cite Landscape

Load Citation

Maintenance Fees

Fee Large entity fee small entity fee micro entity fee due date
3.5 Year Payment $1600.00 $800.00 $400.00 Jul 3, 2026
7.5 Year Payment $3600.00 $1800.00 $900.00 Jul 3, 2030
11.5 Year Payment $7400.00 $3700.00 $1850.00 Jul 3, 2034
Fee Large entity fee small entity fee micro entity fee
Surcharge - 3.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge - 7.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge - 11.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge after expiration - Late payment is unavoidable $700.00 $350.00 $175.00
Surcharge after expiration - Late payment is unintentional $1,640.00 $820.00 $410.00