Agile And Efficient Inference Of Quantized Neural Networks