Utilizing MidJourney generated AI artworks to create a 2.5D location for an adventure game
Jussi Kemppainen, the VFX lead at Mainframe Industries, has recently demonstrated the power of combining cutting-edge tools to create an immersive 2.5D location for a point and click adventure game. By harnessing the potential of the new Photoshop beta generative fill and Blender, Kemppainen successfully blended multiple MidJourney generated AI artworks to bring an intriguing virtual world to life.
In a recent blog post, Kemppainen provides insight into the development process of the project. Kemppainen highlights the use of the Photoshop beta generative fill. This new feature leverages the power of artificial intelligence to intelligently generate content based on the existing artwork. By utilizing this cutting-edge tool, Kemppainen was able to create stunning landscapes and intricate details that formed the foundation of the game's virtual world.
In the kitchen scene, Jussi took advantage of the existing angle of the image, which lent itself well to being seen from another room. This allowed for a smooth transition between the scenes. However, to hide the other room behind a wall in the kitchen location, Kemppainen strategically positioned the back wall of the kitchen to be visible from the lobby/bedroom without revealing the entire lobby. This attention to detail ensured that the location would hold up when viewed from different angles.
Before moving on to the 3D modeling step, Kemppainen focused on creating the shadow passes for the original images. By using a custom shader in Photoshop, the shadow version of the location images was rendered using 3D shadows cast by real-time 3D lights. This allowed for precise control over the shadow appearance, ensuring that the shadows blended seamlessly with the environment.
Using fSpy and Blender, Kemppainen proceeded to model the locations in 3D. By reverse engineering the camera in fSpy and aligning the scenes in Blender, Kemppainen created a 3D mesh of the location for accurate 3D projection. This step required attention to detail, especially due to the parallax shift visible to the player, which necessitated additional modeling to achieve a natural-looking transition.
Once the 2D and 3D steps were complete, Kemppainen brought all the elements together in Unity. The scenes were meticulously set up, with the rooms matching each other in color and shape. Kemppainen added various visual enhancements, such as painted shadows, depth of field, grain, ambient lighting, and fog, to create a visually appealing and immersive experience for the players.
To further enhance the transition between the rooms, invisible cubes were used for navigation instead of the actual room meshes. This allowed for precise control over the player's movement and proximity to different walls and objects in the tight spaces. Camera movements, such as dolly and pan, were triggered by hit-boxes at the doorways, providing predetermined transitions and maintaining a static camera for better point-and-click gameplay.
Interactions within the scene were also implemented, with the player being able to open the fridge. This was achieved by animating only the fridge door and using inverse kinematics (IK) to attach the player's hand to the handle. Adventure Creator's tools were utilized for scripting, and custom code was integrated through plugins or additions for Adventure Creator, allowing for seamless integration and easier updates in the future.
Overall, the creation of this immersive 2.5D location was a successful experiment that showcased the potential of combining multiple 2D backgrounds into a larger location. Kemppainen gained valuable insights and confidence in adding more complex scenes in the future, further pushing the boundaries of game development. The next challenge on the horizon involves seamlessly blending open spaces with two camera-projected scenes, taking the experimentation to new heights and uncovering new possibilities in the realm of game design.
