Skip to main content Skip to secondary navigation

SQuAD Training

Main content start
 Submission DateModelTime to 0.75 F1Cost (USD)Max F1 ScoreHardwareFramework

Mar 2019

FastFusionNet

Wu et al. (Cornell, SayMosaic, Google)

source

0:18:46N/A0.82361 NVidia GTX-1080 TiPytorch v0.3.1

Dec 2018

DrQA

Runqi Yang, Facebook ParlAI, Brett Koonce

source

0:27:07N/A0.78291 NVidia 2080 RTX (dev box)Pytorch 1.0.0

Apr 2018

QANet

Google

source

0:45:56N/A0.76371 TPUv2TensorFlow v1.8

Dec 2018

DrQA

Runqi Yang, Facebook ParlAI, Brett Koonce

source

0:50:21$0.760.76061 T4 / GCPPytorch 1.0.0

Dec 2018

DrQA

Runqi Yang, Facebook ParlAI, Brett Koonce

source

0:56:43$0.570.76201 P4 / GCPPytorch 1.0.0

Sep 2018

DrQA

Runqi Yang, Facebook ParlAI, Brett Koonce

source

1:00:35$3.090.75691 V100 / AWS p3.2xlargePytorch 0.4.1

Sep 2018

DrQA

Runqi Yang, Facebook ParlAI, Brett Koonce

source

1:21:55$1.230.75841 K80 / AWS p2.xlargePytorch 0.4.1

Oct 2017

BiDAF

Stanford DAWN

source

7:38:10$6.870.75331 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge])TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

7:51:22N/A0.75301 P100 / 512 GB / 56 CPU (DAWN Internal Cluster)TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

8:43:40$8.440.75241 K80 / 30 GB / 8 CPU (Google Cloud)TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

10:50:22$5.780.757960 GB / 16 CPU (Google Cloud [n1-standard-16])TensorFlow v1.2