Benchmarks for TorchScript models
For most stable results, do the following:
cset shieldwhen running benchmarks.
Some of these scripts accept command line args but most of them do not because I was lazy. They will probably be added sometime in the future, but the default sizes are pretty reasonable.
Test the fastrnns benchmarking scripts with the following:
python -m fastrnns.test or run the test independently:
python -m fastrnns.test --rnns jit
python -m fastrnns.bench
should give a good comparison, or you can specify the type of model to run
python -m fastrnns.bench --rnns cudnn aten jit --group rnns
python -m fastrnns.profile
should generate nvprof file for all models somewhere. you can also specify the models to generate nvprof files separately:
python -m fastrnns.profile --rnns aten jit
Use Linux for the most accurate timing. A lot of these tests only run on CUDA.