.onnx Model inference in Unreal Engine
I'm currently learning how to use Unreal Engine's Neural Network Engine (NNE) to leverage some machine learning work in game in real-time.
Enable NNE plugin
So the first entry point is, use NNE plugin in Unreal Engine.
By following this article, we can discover how to enable the plugin and write a simple demo.
[!warning] NNE plugin name has changed after Unreal Engine 5.4.
NeuralNetworkEngine + NNERuntimeORTCpu -> NNERuntimeORTThe abbreviation
ORTstands foronnx runtime, as well as a model file format named.ort.
mnist-8 ONNX Model and IR version
The official mnist-8 model in onnx model zoo is based on IR version 3, which will throw an exception to no graph input.
ValidationError: ConvAddFusion_Add_B_Parameter6 in initializer but not in graph input
Since IR version 3 has been deprecated, the only method to fix this is upgrade the model (train it again or modify the ir_version value), or download another one.
import onnx
model = onnx.load("mnist-8.onnx")
print(f"IR_VERSION: {model.ir_version}")
model.ir_version = 7
print(f"IR_VERSION: {model.ir_version}")
onnx.save(model, "updated_mnist-8.onnx")
Now you can import updated_mnist-8.onnx into Unreal Engine as normal.
Reference
- minst:
- MNIST handwritten digit database, Yann LeCun, Corinna Cortes and Chris Burges
- models/validated/vision/classification/mnist at main · onnx/models (github.com)
- CNTK/Tutorials/CNTK_103D_MNIST_ConvolutionalNeuralNetwork.ipynb at master · microsoft/CNTK (github.com)
- ONNX graph optimization cause ONNX model checker to fail · Issue #4143 · onnx/onnx (github.com)
- Unreal Engine NNE:
- onnx: