Wagner James Au / New World Notes

Author, strategist, virtual world reporter

recent posts

about

Midjourney Metaverse? CEO David Holz Confirms AI Startup is Building “Real-Time Open World Holodeck”

Midjourney Metaverse holodeck

Update, 11:35PT: Hours after running this post, I received a late-night reply from Midjourney CEO David Holz himself, in answer to my email with the subject line, "Confirm Midjourney holodeck announcement?", which cited the below quote attributed to him, then asked if that characterization was accurate.

Holz's one sentence reply: "roughly correct quote yes". 

So it does indeed look like Midjourney is aiming to build some kind of real-time open world holodeck in 2024! 

In a follow-up, I asked Holz if Midjourney advisor (and Second Life founder) Philip Rosedale is involved in this effort. Will further update this post if that's also confirmed! I've also amended the original post title accordingly.

Original post (first published 4:33pm PT) below: 

I've reached out to official channels to confirm this (or not), but according to some attendees (as here and here), Midjourney CEO David Holz made a pretty amazing announcement during the startup's open office hours earlier this week.

Specifically, he apparently suggested Midjourney is preparing to deploy a metaverse platform of some kind.

As attendee Nick St. Pierre on Twitter/X quotes Holz:

[H]e thinks they “can get to the holodeck” by 2024.

“We’re gonna build a lot of stuff this year. I think we’ll build more stuff than I’ve ever built before…By the end of 2024 hopefully we have real-time open worlds.”

Normally I would be instantly skeptical about this as hearsay or a misinterpretation. For one thing, while Midjourney outputs 2D images and video, creating a dynamic, immersive virtual world isn't simply a matter of generating 3D graphics, for reasons explained here.

But here's the thing: Second Life founder Philip Rosedale is a lead advisor at Midjourney.

He's listed as such on the startup's hompage. But this may be a coincidence, of course, and Holz's comments this week might have been mischaracterized; hopefully I get some kind of official reply on this.

https://platform.twitter.com/widgets.js

And if you attended this week's Midjourney office hours, dear reader, please recount what was said in Comments!

Please support posts like these by buying Making a Metaverse That Matters!

Posted in

7 responses to “Midjourney Metaverse? CEO David Holz Confirms AI Startup is Building “Real-Time Open World Holodeck””

  1. Nadeja Avatar
    Nadeja

    Consider that the models don’t exactly need to create mesh objects to render a 3D environment and items.
    This is what was already possible half a year ago with NERFs:
    https://www.youtube.com/watch?v=31oxj6mcsOM
    Also, a different thing, video generators such as the one offered by Pika AI are starting to produce interesting short clips:
    https://www.youtube.com/watch?v=IYGbQpqTYxE
    In about another year, by the end of 2024 there is a chance that not only the generation of 3D photo-realistic animations with a prompt would be feasible, but also a constant stream that may be used as a real-time open world. If it doesn’t happen by then, it will happen eventually.

    Like

    Reply
  2. Nadeja Avatar
    Nadeja

    In Star Trek, a way to start the holodeck simulations was to describe the scenes, the characters and the objects to generate. It would be fun if we will eventually create our avatars the way we create images with Midjourney. And also interactively “I’d like my haircut more elaborate and with flowers, a little more, yes, like that”.

    Like

    Reply
  3. Lex4art Avatar
    Lex4art

    Totally unexpected, I’m intrigued! Oo

    Like

    Reply
  4. Tad Avatar
    Tad

    If Phillip Rosedal Is at the helm, I’d be concerned they will be scraping Second Life for their 3D model data. (And possibly VR Chat too). If such a thing is possible, I hope users have a way go protect their work and intellectual property. Sounds scary tbh

    Like

    Reply
  5. Ryan Schultz Avatar
    Ryan Schultz

    Ooh, things are getting interesting!

    Like

    Reply
  6. Nadeja Avatar
    Nadeja

    Eventually, on December 2 2024, Google Deepmind introduced Genie 2
    https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
    “Today we introduce Genie 2, a foundation world model capable of generating an endless variety of action-controllable, playable 3D environments for training and evaluating embodied agents. Based on a single prompt image, it can be played by a human or AI agent using keyboard and mouse inputs.
    […]
    Genie 2 is a world model, meaning it can simulate virtual worlds, including the consequences of taking any action (e.g. jump, swim, etc.). It was trained on a large-scale video dataset and, like other generative models, demonstrates various emergent capabilities at scale, such as object interactions, complex character animation, physics, and the ability to model and thus predict the behavior of other agents.”

    Like

    Reply
  7. Nadeja Avatar
    Nadeja

    December 4

    Like

    Reply

Leave a comment