As technology continues to evolve, gaming has become a crucial part in how we communicate and spend time together. But in an increasingly interconnected world keeping experiences in sync and delivering the best experience for everyone has become an ever increasing battle requiring new tech and innovations at every turn to keep the game in play.
David A. Smith is the founder and CTO of metaverse OS Croquet and an industry veteran with experience from the very early dawn of computing.
With the advent of Croquet for Unity we caught up with the creator of countless innovations we now take for granted to talk about the early days of multi-user tech and how Croquet forms the basis of today’s high-speed multiplayer mobile experiences and is ready for what’s to come.
You were there at the dawn of networking and multi-user set-ups. What was the tech like back then and foundations of Croquet?
David A. Smith: Croquet started from a discussion between Alan Kay and myself. Alan is a Turing Award winner and is considered to be the father of the modern personal computer, having not only coined the term, but also having led the team at Xerox PARC to create the Xerox Alto – an extraordinary machine that directly influenced virtually every aspect of modern computer systems today.
If I sat you down in front of an Alto from 1975, you would already know how to use it, as that machine provided the foundation for not just the Macintosh and Windows computers today, but most of the interface we take for granted on mobile devices. Xerox had hundreds of interconnected Alto computers connected by Ethernet. Altos provided email, file sharing, most of the things you take for granted on your PC, but one thing it did not have, and this is still mostly true of modern systems, was a way to do deep collaboration.
So despite their being a network there wasn’t really a way for users to work and play together?
The capability was demonstrated by Doug Engelbart at a computer conference in 1968 in what has become known as the “Mother of All Demos”. There he really demonstrated the future of computing. There was the first demonstration of the mouse, which he co-invented with Bob English. There was hypertext… video conferencing… but most importantly, peer-to-peer collaboration where every participant could engage with each other instantly and fully interactively.
Engelbart and his team were able to do this because they were brilliant, but also because they were all working on the same time-sharing computer, so access to each other and their workspace was relatively simple. Doing the same thing with the Xerox Alto was another matter entirely. Live sharing between these systems was impossible so Alan and I decided to address this long standing missing protocol problem.
He really demonstrated the future of computing. There was the first demonstration of the mouse, which he co-invented. There was hypertext… video conferencing…
David A. Smith
There he really demonstrated the future of computing. There was the first demonstration of the mouse, which he co-invented with Bob English. There was hypertext… video conferencing…
We started the Croquet Project at HP Labs to make this idea a reality. This first system was created using Smalltalk which proved to be essential – both due the incredible speed, but also because it was a live programming system written in itself. Smalltalk used a bitmap image of memory to maintain its state – you literally save the entire state of the Smalltalk virtual machine and can then load it again at any time and not miss a single thing. This enabled a new user to instantly join a Croquet session and be perfectly synchronised with the other participants no matter in what state the other users were in.
Later, after I left Lockheed Martin (where I was a Senior Fellow and leading a team to develop AR and VR technology for training), Alan convinced me to take another shot at the problem. We spent about three years to solve the remaining problems of making Croquet not just powerful, but extremely accessible and simple to use.
JavaScript, which was greatly influenced by Smalltalk along with “C” and Lisp, proved to be an excellent vehicle for this effort. Alan noted that the browser would make an excellent platform for creating and delivering a full virtual machine based operating system and indeed that is what the Croquet OS is.
Tell us more about Croquet for Unity, and what makes it different from other similar frameworks?
Croquet OS utilised a model/view architecture (also invented at Xerox PARC). What this means is that the developer can write any kind of simulation, including physics, AI for bots, real time interactions, and that simulation is guaranteed to operate exactly the same on everyone’s system. The first implementations of the view side of the Croquet system were done using web visualisation technologies like using the DOM with a 2D canvas or 3D with WebGL. This was aided by such incredible tech as Three.js. This proved to be an excellent platform for the development of multiplayer WebXR applications, and Croquet won both the 2022 Augmented World Expo Startups to Watch and the 2023 WebXR Poly award for best WebXR Platform.
However, we had many of our customers wanting to develop and deploy their applications with a native rendering system. In particular, there was a great interest in our supporting Unity and Unreal. We were able to figure out how to bridge between the Croquet virtual machine and the Unity rendering engine in a way that was extremely efficient, but still enabled a perfectly synchronised simulation between users. We knew this would be a very big deal and of course it is.
What are the biggest problems facing companies working in the multiplayer space?
Multiplayer has been an unsolved problem since Xerox PARC yet multiplayer games are rapidly becoming the centre of the gaming universe, and rightly so. There is nothing more interesting than empowering a human being to be a full participant in a shared virtual world. It’s essential to the next generation of games. Croquet literally changes the game. It’s enabling a new class of interactions that hasn’t been possible before while making the development of these new experiences as easy as creating single user applications.
The Croquet OS doesn’t just level the playing field between the independent game developers and the AAA studios that have their own multiplayer infrastructure. It gives the indies a powerful tool that surpasses what the AAA studios can provide. Game on.
Croquet was named as “Startup to watch” at the AWE Conference in May 2022. What put you ahead of the competition?
Croquet directly addressed a core requirement of the idea and ideals of the metaverse. First, Croquet is fully open and extensible. Most of our frameworks are open source today and the rest will be made open source over time. But more importantly, Croquet addresses the metaverse directly as a communication medium.
One way to think about this is that language is like a virtual machine running on top of the hardware that makes up your brain. Language enables you to think and understand things that your primitive brain could never reach. You think in terms of language. And not just speech.
Any musician will tell you that music is a language – it enables you to share thoughts and feelings that aren’t possible otherwise.
Mathematics is an incredibly powerful language that we use to understand and control the possibilities of the universe. It was essential for us to create the world we live in today. But again, it is a virtual machine running on our brains.
The metaverse is an incredibly powerful and rich language that enables an augmented conversation where the ideas we express are amplified and made real by the computer. There’s simply no limits to our ability to explore, share and understand new ideas, … or to create new worlds and new realities.
It comes down to this. You are defined more than anything by how you communicate. The metaverse will redefine what we are and what it means to be human and Croquet is helping to open the next frontier in human/computer/human interaction.
What would you like to see more of in the gaming sector?
I created one of the first, if not the first, adventure shooter games – The Colony in 1987. I also helped create the Rainbow Six game franchise with Tom Clancy. What has been incredibly frustrating to me is that the mechanics and ‘vocabulary’ of gameplay has not evolved much since those games were created. This is particularly true of multiplayer games.
In the case of multiplayer though, it is more a case of technological limitations instead of a lack of creativity or desire. What we need to see is entirely new classes of game ideas that are empowered by a platform like Croquet where there are no longer limitations of client/server or trying to wire up netcode. Games are a combination of simulation and user interaction with that simulation. Our goal is to make this as easy in multiplayer as it is in single player. Adding another human to the next generation simulations will literally change the game.
What can we expect next from Croquet?
Croquet uses a kind of server that we call a reflector. Reflectors have no application state – in fact Croquet event messages are encrypted end-to-end, so reflectors can’t see the contents of these messages. All they can do is add a time-stamp to the event and redistribute it to all of the participants in a world. This means that we can deploy reflectors anywhere and instantly – they are tiny but are very, very fast. Our goal is to ensure that everyone everywhere can enjoy extremely low-latency to reflectors no matter where they are. For example I get a less than 10 msec latency at my apartment in Los Angeles because we have a reflector fleet nearby.
With that in mind, we are looking at how we can fully virtualise reflectors so that anyone can deploy them onto any system – allowing access to anyone that might need it in the area. As an example, I have a reflector deployed on my phone. The other part of this equation is that anyone that does deploy a reflector should be compensated for its use – and of course this is an ideal application for crypto currency and smart contracts. Next year, people will be able to deploy their own fleet of reflectors that will be available for anyone in their neighbourhood to utilise, totally anonymously and safely, but also providing appropriate reward back to the host of the reflector.