Skip to main content Skip to secondary navigation

SQuAD Inference

Main content start
 Submission DateModel1-example Latency (milliseconds)10,000 batch answering cost (USD)Max F1 ScoreHardwareFramework

Jul 2019

PA-Occam-Bert

Ping An Technology Occam Platform

source

7.5790N/A0.75891 NVidia Tesla V100Tensorflow 1.13.0

Feb 2019

FastFusionNet

Wu et al. (Cornell, SayMosaic, Google)

source

7.9000N/A82.52091 NVidia GTX-1080 TiPytorch v0.3.1

Oct 2017

BiDAF

Stanford DAWN

source

100.0000$0.150.757960 GB / 16 CPU (Google Cloud [n1-standard-16])TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

590.0000$1.580.75241 K80 / 30 GB / 8 CPU (Google Cloud)TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

638.1000N/A0.75301 P100 / 512 GB / 56 CPU (DAWN Internal Cluster)TensorFlow v1.2

Oct 2017

BiDAF

Stanford DAWN

source

705.9000$1.760.75331 K80 / 61 GB / 4 CPU (Amazon EC2 [p2.xlarge])TensorFlow v1.2