UnrealCV: Connecting Computer Vision to Unreal Engine. Computer graphics can not only generate synthetic images and ground truth but it also offers the possibility of constructing virtual worlds in which: (i) an agent can perceive, navigate, and take actions guided by AI algorithms, (ii) properties of the worlds can be modified (e.g., material and reflectance), (iii) physical simulations can be performed, and (iv) algorithms can be learnt and evaluated. But creating realistic virtual worlds is not easy. The game industry, however, has spent a lot of effort creating 3D worlds, which a player can interact with. So researchers can build on these resources to create virtual worlds, provided we can access and modify the internal data structures of the games. To enable this we created an open-source plugin UnrealCV (this http URL) for a popular game engine Unreal Engine 4 (UE4). We show two applications: (i) a proof of concept image dataset, and (ii) linking Caffe with the virtual world to test deep network algorithms.
Keywords for this software
References in zbMATH (referenced in 2 articles , 1 standard article )
Showing results 1 to 2 of 2.
- Matthias Müller, Vincent Casser, Jean Lahoud, Neil Smith, Bernard Ghanem: Sim4CV: A Photo-Realistic Simulator for Computer Vision Applications (2017) arXiv
- Weichao Qiu, Alan Yuille: UnrealCV: Connecting Computer Vision to Unreal Engine (2016) arXiv